Introduction

Acupuncture, as a part of Traditional Chinese Medicine (TCM), can be traced over a few thousand years. The theoretical system of acupuncture was gradually established and further developed in long-term clinical practice. In recent decades, acupuncture has become increasingly popular for multiple applications and less adverse reactions in many countries.1–3 However, because of educational and cultural differences, it is important to recognize that clinical acupuncturists as well as acupuncture researchers may apply different traditions regarding diagnosing, operating, and probably influencing clinical decisions on acupuncture treatment4,5. For clinicians and health-carers unfamiliar with traditional acupuncture, it is also worth considering how to make clinical decisions about acupuncture.

Clinical decisions regarding acupuncture are highly operator-dependent and uncertain. From the perspective of Evidence-Based Medicine, researchers typically conduct randomized controlled trials, observational studies, meta-analyses, bibliographic reviews, and other methods as a reference for clinical decision making. Although these methods can provide concrete and credible evidence, they still have inherent disadvantages. First of all, clinicians and researchers may face the challenge of making decisions under the condition that there is insufficient information.6 Clinical trials and experience may be limited due to the rarity of specific diseases.7 Furthermore, the clinical decision-making process can be misled by previously published poor-quality evidence.8 Secondly, conducting scientific and comprehensive clinical research demands significant manpower and material resources, consuming a considerable amount of time. This may cause delays in clinical decisions.

More importantly, due to the characteristics of acupuncture, whether in scientific research or in treating diseases, the clinician experience is one of the most vital factors in the efficacy of acupuncture, with great variation and uncertainty. Unlike precision medicine which requires sophisticated tools and platforms, acupuncture decision making relies more heavily on experience, such as accurate selection of acupoints, familiarity with operations, communication with patients, and appropriateness of syndrome differentiation and treatment. In China, new acupuncture theories and methods formulated by professors based on their clinical experience have taken shape, such as Jin’s three-needle therapy,9 “Xingnao Kaiqiao” acupuncture technique.10 At present, acupuncture studies are gradually incorporating the clinical acupuncture experience as a necessary factor.11–15

Meanwhile, as a health care therapy, acupuncture involves doctor-patient interactions.16 Acupuncture clinicians pursue the happening of “De Qi” which is crucial for acupuncture to react to different nerve conduction.17 “De Qi” refers to the subjective sensation from patients during the needling process, such as soreness, distension, or heaviness, which arises as an interaction between patient and clinician. Thus, patient preferences and personal requirements are gradually becoming important components to be considered and can be achieved through investigation of patients when making decisions. Canadian Task Force on Preventive Health Care (CTFPHC) has involved patients and the public in guideline development to guide clinical decision making.18,19 Therefore, a combination of practical and professional suggestions from participants is needed to assist in acupuncture clinical decision making.

Delphi is a group facilitation technique that involves a repeated iterative multi-stage process to convert opinions into group consensus, providing a flexible and modifiable method in medicine and social sciences.20 Several recommendations and clinical guidelines regarding acupuncture, as well as instruments, have been developed based on the Delphi method,21,22 demonstrating strong applicability and flexibility. The Delphi method can take into account subjective factors such as expert experience and preference of patients, which is especially applicable to acupuncture. Indeed, the precision and standardization of the Delphi process are critical factors affecting the clinical decision making of acupuncture. A non-standard Delphi process can mislead the final clinical decision. To promote standardization, the standard for conducting and reporting Delphi studies (CREDES) was developed by Saskia Jünger, which is a methodological checklist originally intended to be used in palliative medicine.23 Currently, evaluation of the Delphi consensus process in acupuncture for clinical decision making is lacking. It is unclear whether the Delphi implementation process is standardized. Therefore, we conducted overall descriptions and quality assessments in acupuncture for the Delphi process based on the CREDES standards, including content and implementation, and made suggestions with the aim of facilitating standardization and rigor for further clinical decision making in the field of acupuncture.

Methods Search Strategy

Comprehensive research on reports utilizing the Delphi consensus process in acupuncture, ranging from database inception to Mar 26, 2024, was sourced from the following databases: PubMed, Embase, Cochrane Library, China National Knowledge Infrastructure (CNKI), WanFang, and VIP database. The Chinese search terms were primarily determined by “Acupuncture” and “Delphi” as a combination: (Delphi OR Delphi research OR Delphi technology) AND (acupuncture and moxibustion OR acupuncture and moxibustion treatment OR electro-acupuncture OR body acupuncture OR hand acupuncture). An example search strategy is provided in Table 1 and Supplementary Material S1. Our research was limited to reports published in either English or Chinese.

Table 1 Search Queries

Eligibility Criteria

All articles utilizing a Delphi consensus procedure and acupuncture were under consideration, regardless of the type of research. Acupuncture should be the main focus, including manual acupuncture, electronic acupuncture, and body acupuncture. Moxibustion, acupressure, acupoint injection, and massage were excluded. Given our goal to systematically evaluate the standardization of the Delphi consensus, it is essential that articles provide a detailed account of the process rather than just mentioning it. Articles were excluded if they met the following exclusion criteria: (a) duplicate literature; (b) conference abstracts, meta-analyses, reviews, comments, editorials, etc.; (c) published in languages other than English or Chinese; (d) unavailable documents. In addition, if the content of the articles was similar but differed in either language or expression, only the article with a more detailed description and a higher quality of the Delphi consensus process was included.

Data Extraction

Screening, data extraction, and bidirectional checking were performed by two independent researchers. Firstly, all retrieved articles were entered into EndNote X7 (version: EndNote X7.2 Bld 8156). Duplicates were removed through a combination of electronic and manual checks. Secondly, records such as conference abstracts, comments, and editorials were removed after reviewing the titles, abstracts, and keywords of the remaining articles. According to the inclusion and exclusion criteria, the full text of the articles was read to determine whether they should be included, and the clearly irrelevant articles were deleted. The data was collected from the included documents using Microsoft Excel 2010 and entered into a self-designed collection table. The extracted data comprised the following general characteristics from each eligible record: year of publication, country of the first author, financial support, research topic and category, Delphi rounds, working group initiation, and additional quality details based on CREDES standards. Prior to analysis, researchers cross-checked and sorted the collection.

Assessment of Reporting Quality

Standardized methods were used to extract and analyze data from appropriate datasets. First, two researchers and additional research assistants underwent systematic training to ensure a thorough understanding of each item prior to formal analysis. A pilot test of the CREDES standards using the 5-Likert scale for 3 records was then conducted by two independent evaluators (WPF and SYY) and they met to discuss and resolve the discrepancies. Next, the CREDES standards, consisting of 16 items categorized into 4 domains, were then applied to assess the reporting quality of the Delphi consensus process in all documents. Depending on whether the authors report or not, the reported items included in full in the collection table were rated “yes”. As for the 4 domains, we performed the quality assessment using a 5-point Likert scale,24 with the narrative quality of the report expressed in descending order from 5 to 1, respectively. If any item was not mentioned at all or the quality of the report was poor, 1 point was awarded. In contrast, items with fairly high quality were awarded 5 points. As soon as there were differences of opinion in the assessment and decision, the two staff should resolve these through discussion and then reach a consensus or seek consultation. If necessary, a third-party assessor (YGR) should be involved in the assessment without conflicts.

Bias Control

ICC-statistics (Intraclass Correlation Coefficient, ICC) were evaluated to assess agreement and consistency.25,26 The value of the ICC approached 1, as consistency between raters increased. Published criteria were applied in our study. Moderate variability was defined as 0.5–0.75, good variability as 0.75–0.9, and excellent variability as more than 0.9.27

Data Analysis

Descriptive analysis was applied to summarize the general characteristics. The positive reporting rate of each item was used for qualitative analysis, as well as quality assessment in four domains using the Likert scale, which was employed for quantitative analysis. The percentage for each scaled domain needs the addition of evaluators’ ratings and numbers of items in each domain and scaling by maximum and minimum possible domain scores before converting the value into a percentage. The scaled domain percentages were generated for inter-domain comparison, and the average appraisal scores and scaled domain percentages for each record were applied for comparison. The basic quality assessment idea in this study was to calculate in advance the minimum and maximum values based on the input parameter configuration of a 5-point Likert scale, and use those extremes as the normalization limits, keeping the values within the [0,1] interval, which enables it more robust:28–30

Microsoft Excel 2010 (Microsoft Corporation, Redmond, WA, USA) was used for data extraction and management, while R 4.3.3 was used to analyze the data. Results were presented in an overall description and visualization.

Results Search Results

According to the retrieval strategy, a total of 328 records were identified. Of these, 94 duplicate records were removed. After reading the title and the abstract, 83 full texts remained to assess for eligibility. Finally, 37 articles were included in the final analysis (Figure 1,31). A PRISMA checklist was provided in Supplementary Material S2.32

Figure 1 The PRISMA flow chart.

Notes: Flow chart adapted from Liberati A, Altman D, Tetzlaff J, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. Journal of clinical epidemiology. 2009;62(10). Creative Commons.31

General Characteristics

A total of 37 articles were published ranging from 2011 to 2023 (Figure 2). The total number of papers appeared to be increasing: two in 2011 (5.41%), one in 2012 (2.70%), zero in 2013 (0.00%), three in 2014 (8.11%), zero in 2015 (0.00%), zero in 2016 (0.00%), zero in 2017 (0.00%), one in 2018 (2.70%), two in 2019 (5.41%) and 2020 (5.41%), nine in 2021 (24.32%), ten in 2022 (27.03%), seven in 2023 (18.92%). The number of publications before 2020 was only 11 for almost ten years. However, the number has increased sharply since 2021, in which 26 relevant papers were published in the last 3 years, indicating that the Delphi method has become a hot topic of acupuncture. 37 datasets were published in China (n=28), Australia (n=3), Germany (n=2), the USA (n=2), South Korea (n=1) and the United Kingdom (n=1).

Figure 2 Number of annual publications (n=37).

We categorized the included literature into several topics depending on the research purpose and overall content: clinical survey, indicator research, methodology research, operation standards, the establishment of rating scale/statement/instrument, specific disease research, treatment regime, and other studies. The top 5 categories were treatment regime (43.24%), establishment of rating scale/statement/instrument (18.92%), operation standards (10.81%), specific disease research (10.81%), and indicator research (8.11%), respectively (Figure 3). Researchers have given more importance to treatment regimes and methodological guidelines in acupuncture using the Delphi method. Regarding funding, 11 articles (29.73%) did not mention financial support. In other articles (67.57%) that received financial support, the number of projects and sponsor names were reported in the text (Supplementary Material S3).

Figure 3 Proportion of types and objectives in articles.

Details of Delphi Consensus

A total of 34 literatures reported rounds of the Delphi survey, with 15 (40.54%) consisting of two rounds of Delphi, and 17 (45.95%) consisting of three rounds. Only 2 (5.41%) studies were composed of 4 rounds.33,34 For the working panel, most studies (67.57%) indicated the composition of the working panel, which mainly consisted of a steering panel, a working panel, and an expert panel. Two studies used four subgroups designated by a steering committee, an expert consensus group, an evidence review group, and an academic secretariat group, respectively (Table 2 and Supplementary Material S4).

Preliminary investigations before the Delphi survey and consensus meeting were mentioned in 28 (75.68%) studies. The main methods contained a question investigation, a Delphi survey, a semi-structured interview and a pilot observation study. 10–30 experts participated in 23 Delphi studies, corresponding to a ratio of 62.16% across all included documents. Meanwhile, 30–50 experts took part in 4 studies (10.81%), and in total there were 5 studies (13.51%) with fewer than 10 experts. Only 2 studies (5.41%) involved more than 100 experts to obtain comprehensive opinions.

Table 2 Descriptive Information in Delphi Consensus

Most studies (75.68%) elicited professional views mainly through online software and email. Regarding the response and enthusiasm of the experts, most studies (67.57%) gained positive feedback with a description of the response rate or the number of questionnaire collections. Information about the experts involved in the Delphi survey was listed, which was reported in 33 articles (89.19%). In our summary, we found that the specific disease research contained a moderately incomplete description in terms of general details. The Delphi method has been mainly used in the development of most clinical treatment regimes for acupuncture, and the general details have been well supplemented.

Reporting Quality Based on CREDES Items

Based on the CREDES standards, the positive reporting rate of 37 documents was used for summary and calculation (Table 3 and Supplementary Material S5). The item “External validation” was reported as the lowest positive rate by 32.43%. And 48.65% subsequently reported the item “Prevention of bias”. The item “Adequacy of conclusions”, “Definition and attainment of consensus”, and “Discussion of limitations” were reported with a modest positive ratio of 62.16%, 64.86%, and 67.57%, respectively. The other items were reported by more than 70%. Among them, 4 items were reported with a significant proportion of more than 90% as follows: Purpose and rationale, Expert panel, Description of the methods, and Procedure.

A total of 13 articles were estimated by two independent researchers using the Likert scale with an average score of over 80%. As shown in Table 4, the domains “Planning and design” and “Reporting” performed better than “Rationale for the choice of the Delphi technique” and “Study conducting”. The averages for “Rationale for the choice of the Delphi technique”, “Planning and design”, “Study conducting”, and “Reporting” in all included documents were, respectively, at scores of 65.54%, 68.75%, 45.10%, and 66.07% (Supplementary Material S6).

Table 3 Reporting Items of Delphi Consensus Based on CREDES Standards

Table 4 Scaled Domain Percentages for Appraisers of Each Record

Discussion

This study first demonstrated the reporting quality of acupuncture reports regarding the Delphi process in strict adherence to the CREDES standards. The Delphi method is an important approach to clinical decision making and is particularly recommended as the best method for developing clinical practice guidelines.68,69 A high-quality Delphi process not only contributes to the inclusion of multiple opinions but also balances benefits and harms from optimal decisions. Therefore, a high quality Delphi process has a positive impact on clinical decision making in acupuncture. After evaluating the 37 included studies, we found that a significant number of item descriptions were inadequate. Compliance with the CREDES checklist improves the quality of reports as well as helps to some extent in decision making for clinical purposes.

Since 2020, the number of relevant publications has increased moderately but is still insufficient. Research reports with funding are significantly much more rigorous and could be carried out with sufficient cost and time. When classifying the articles according to the purpose or type of research, we found that the objective mainly focused on acupuncture treatment regimens in trials, followed by the development of scales and instruments. At this stage, the Delphi technique still aims to develop a clinical therapeutic strategy for acupuncture but lacks the expansion of other functions. A recent online Delphi study, made up of patients and the public, demonstrated that digital interventions improve mental well-being.70 Additionally, a shared modified Delphi consensus was utilized to assess the safety of nine physical agent modalities (PAMs) practices such as electrical stimulation neuromodulation and electromagnetic therapy.71 In order to optimize the clinical strategy, appropriate studies can be promoted, for example, types of intervention can be determined based on personal preferences of patients and clinicians. The safety assessment of acupuncture treatment can be carried out in the Delphi consensus but has not yet been researched. The application of the Delphi technique should be expanded more widely in acupuncture.

Based on the CREDES standards, 26 articles mentioned the rationale for the Delphi technique (Table 3). Among them, details were poorly described based on the definition, without the narrative being consistent with the study. Evidence-Based Medicine combines clinical expertise with the best available clinical evidence from systematic research to make clinical decisions. Because solving the clinical problems is uncertain, the Delphi technique is necessary to produce systematic published guidance, especially when randomized controlled trials cannot be used for validation. It is applicable to create clinical guidance that cannot be developed using quantitative results.23,72 The Delphi method involves a larger number of participants to obtain the best evidence by combining expert opinion with evidence.73 It must be emphasized that the Delphi technique is only considered an appropriate approach to make clinical decisions only if the suggestions of participants and current evidence are equally reliable. Therefore, clarifying the certification of the Delphi technique selection can help determine whether it meets the research as the best approach.

The domain “Planning and design” does not reach satisfactory levels (68.75%). Unless the major methodological process involved the Delphi technique, description of it in these reports was limited. The Delphi process includes selection of experts, appointment, gathering of current data, presentation of statements and recommendations, appropriate grading of evidence and then repeat until there is a convergence of opinion.69,74,75 Reasonable protocols and preliminary experiments are required to ensure usability. Different degrees of modification should also be determined systematically and strictly based on reasonable references and reasons as much as possible to avoid subjective arbitrariness.20 Furthermore, a preliminary and precise definition of consensus is crucial. This involves handling the existing items in the next round, the required threshold of ending process, the following procedure after iterations. It is necessary to plan consensus building in advance, taking uncertainty into account, as the Delphi method may also not achieve perfect consensus. Therefore, more exploratory research and programs are needed to enrich paradigms.

The domain “Study conduct” was assessed at the lowest average ratio of 45.10%. To our surprise, the item “Information input”, including execution of preliminary investigation and pilot studies was reported with a positive ratio of 75.68%. The reports include the following: investigation, pilot observational study, expert consultation meetings, and literature reviews. A preliminary pilot study will assess usability and preliminary effectiveness.76 Systematic reviews can provide evidence for previous studies, while pilot observations and investigations can predict applicability to promote dissemination. A questionnaire prepared based on investigation and literature evidence can provide more comprehensive information and reference aspects for final discussion and review, rather than just forming the basis of expert experience. The preliminary studies can not only serve as prior data to optimize implementation but also targeted improvement measures can be taken to minimize the impact of the information provided on individual judgments. To ensure confidentiality and independence, 18 studies (48.65%) took measures to control bias, mainly through strengthening training, anonymous questionnaires, independent working groups responsible for the collection, or the use of database software to avoid manual input. Xin-Lin Li performed training for surveyors and a pilot study for preventing bias.40 Xin-Tong Su introduced the entire flow by teleconference before the formal consensus meeting, especially responded the experts’ queries, which may minimize the distortion of information among experts and items.48 Questions given for the same data might lead to different answers from different observers.77 Due to the subjectivity of experts and respondents in their understanding of the item, interpretation is crucial prior to formal determination. The CREDES standard states that consensus may not imply that the truth has been found,23 but should highlight differences of views. A clinical guideline proposes that the current form is not stable and needs to be updated for several years as new evidence emerges in the future.22 Giese, N. and M. K. Heirs pointed out that even a panel with a reasonable level of expertise could reach different conclusions due to differences in backgrounds.39 Many checklists are required to “describe” certain topics or “details” should be given or choices “explained”.78,79 Research suggests that if explanations are given for certain terms or items, people can assume that such projects are sufficiently compliant, but they can also be more stringent and require more detailed descriptions.80 Researchers have supposed that consensus or non-consensus should be critically reflected because it provides informative insights and highlights personal opinions regarding complex issues.81 Hence, the results and conclusions obtained from the consensus and the Delphi survey should be interpreted cautiously and adequately.

Concerning external validation, the reporting rate of item “external validation” was at an inferior ratio of 32.43%, indicating that few researchers have taken measures to verify the valuable conclusions that were obtained during the Delphi consensus process. Three studies conducted pilot clinical trials to verify the consensus. Pilot testing of the proposed consensus may demonstrate feasibility.82 A potential threat to the applicability and external validity of the results may occur without this step.83 The establishment of external review committees is applied to review and finalize the draft by consensus, aiming to further clarify publication and dissemination value, although this might require significant time and financial support. Researchers should pay more attention to external validation to ensure generalizability.

As for reporting, the quality assessment is generally acceptable Most studies have pointed out the necessity of the Delphi technique and clearly stated the objective with a positive ratio of 91.89%. In particular, the aim of the studies was described concretely, including gaps in research, lack of literature, and conditions. However, the description of the Delphi method in present studies focuses mainly on its advantages and characteristics, rather than on the interpretation of its applicability. Therefore, researchers should describe more based on the actual situation such as limited available data for choosing Delphi as a demand to provide decision-makers with more comprehensive information. We found that two Delphi surveys consisted of more than 100 experts from different disciplines, which can promote universality and specialization in consensus development. However, several studies used steering groups and expert consulting groups without clear distinctions. Independence in these Delphi surveys might be reconsidered. In addition, few studies included patients and medical staff as participants in the investigation. Researchers proposed that it is vital to consider the individual preferences of healthcare providers and patients and personalize treatment based on disease severity, prognosis, and individual risk factors while adhering to guidelines.84 In addition to specialists, the opinions of healthcare workers, patients, methodologists, as well as nurses, should be taken into account when making medical decisions.

The item “Description of the methods” was reported positively, but the quality was insufficient. The Delphi method is a structured process for collecting views from a group of experts.46 Currently, a combination of modified Delphi technique and semi-structured interviews could be conducted as a better method to avoid inherent limitations. A semi-structured interview can ask predetermined questions for discussion, ask spontaneous questions and develop targeted responses,85 but unstructured processes are more restrictive. Due to the complexity of the overall flows, if no corresponding flowchart is available, the control of progress and the presentation of the results in the individual steps can often differ. Meanwhile, deletion of items requires reasonable interpretation to prevent ignoring important content. The results of each round of the Delphi survey should also be presented more clearly in numbers or details.

Regarding limitations reported in the included studies, a common deficiency is the lack of clinical evidence or the majority of evidence being low certainty.35 Actually, new evidence will be updated, and experts’ understanding of the diseases will also change as a result. Expert opinions therefore do not provide information about the efficacy of a particular therapy and should be viewed as temporary guidance.39 Further studies with high evidence-based quality are required, either in the preparatory phase or in the consensus testing phase. Secondly, the diversity of acupuncture and lack of uniformity in assessment may result in opinions with professional limitations or biases, where professors tend to view from the perspective of individual views and experiences.21,47 The composition of the expert panel and personal opinions may cause bias of all expert consensus, thereby excluding particularly valuable suggestions from the consensus process.86 It is important to choose a considerable number of qualified experts with caution to ensure generalizability and comprehensiveness of clinical decision making. On the other hand, a lack of rigor in the implementation process can lead to inaccuracies and exaggerations. Hence, due to the limitations mentioned, justification of the reached consensus requires more scientific trials and tests to determine applicability and feasibility, and not just reliance on opinions and current evidence.

The CREDES standard provided the minimum requirements of rigor and transparency that Delphi research in the field of medical research should possess, in terms of the rationale for the choice of the Delphi technique, planning and design, study conduct and reporting. To our knowledge, this current quality assessment is the first report to adopt quantitative methods based on the CREDES standards to evaluate the reporting quality of acupuncture articles for the Delphi process. After identifying deficiencies in the Delphi process, this study has contrapuntally proposed some suggestions through quality assessment and comprehensive review to improve the reporting quality in the Delphi process for further clinical acupuncture decisions. Several limitations exist in our study. Reports published in English and Chinese are included from six databases, which may limit the scope of retrieval. This study was primarily a preliminary descriptive study, and understanding between researchers may cause bias.

Conclusion

The reporting quality of articles in acupuncture for the Delphi consensus was generally moderate. But it must be emphasized that the low reporting quality may result from the insufficient promotion and dissemination of the CREDES checklist. Improvement in complete reporting of the Delphi process and application of the CREDES checklist is still in demand. Although documents concerning the Delphi method in acupuncture were insufficient, it is quite promising that it will become a mainstream method for the development of clinical guidelines and decision making in acupuncture. This study was a preliminary descriptive study, and understanding between researchers may vary. Further standardization, including clearer checklists or study reports, should be developed and strengthened to guide clinical decisions in acupuncture.

Abbreviations

CREDES, the Standards for Conducting and Reporting Delphi Studies; PRISMA-ScR, Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews Checklist; CNKI, China National Knowledge Infrastructure; ICC, Intraclass Correlation Coefficient; PAMs, Physical Agent Modalities.

Data Sharing Statement

All data generated or analysed during this study are included in this published article and its Supplementary Information Files.

Ethics Approval and Informed Consent

This study involved a systematic review of peer-reviewed literature only; it did not require ethics approval or consent to participate.

Acknowledgments

The research was supported by the National Key R&D Program of China (No.2022YFC3500403). The funders did not play any role in the study design, data collection, interpretation, and analysis, the preparation of the manuscript, or the decision to publish.

Author Contributions

All authors made a significant contribution to the work reported, whether that is, in the conception, study design, execution, acquisition of data, analysis, and interpretation, or in all these areas; took part in drafting, revising, or critically reviewing the article; gave final approval of the version to be published; have agreed on the journal to which the article has been submitted; and agree to be accountable for all aspects of the work.

Disclosure

The authors report no conflicts of interest in this work.

References

1. Li H, Darby JE, Akpotu I, et al. Barriers and facilitators to integrating acupuncture into the U.S. health care system: a scoping review. J Integr Complement Med. 2024;2024:1. doi:10.1089/jicm.2023.0603

2. Chen L, Michalsen A. Management of chronic pain using complementary and integrative medicine. BMJ. 2017;357:j1284. doi:10.1136/bmj.j1284

3. Luxey X, Lemoine A, Dewinter G, et al. Acute pain management after vaginal delivery with perineal tears or episiotomy. Reg Anesth Pain Med;2024. rapm–2024–105478. doi:10.1136/rapm-2024-105478

4. Bauer M, McDonald JL, Saunders N. Is acupuncture dose dependent? Ramifications of acupuncture treatment dose within clinical practice and trials. Integr Med Res. 2020;9(1):21–27. doi:10.1016/j.imr.2020.01.003

5. Duan Y, Xu Z, Li X, et al. Reporting and data-sharing level of acupuncture randomised controlled trials: a cross-sectional study protocol. BMJ Open. 2023;13(6):e070545. doi:10.1136/bmjopen-2022-070545

6. Jones J, Hunter D. Consensus methods for medical and health services research. BMJ. 1995;311(7001):376–380. doi:10.1136/bmj.311.7001.376

7. Pijnenburg MW, Rubak S, Skjerven HO, et al. Optimizing care for children with difficult-to-treat and severe asthma through specialist paediatric asthma centres: expert practical experience and advice. BMC Pediatr. 2024;24(1):218. doi:10.1186/s12887-024-04707-0

8. Huang J, Wu M, Liang S, et al. A critical overview of systematic reviews and meta-analyses on acupuncture for poststroke insomnia. Evid Based Complement Alternat Med. 2020;2020:2032575. doi:10.1155/2020/2032575

9. Li M. The efficacy and safety of Jin’s three-needle therapy vs. placebo acupuncture on anxiety symptoms in patients with post-stroke anxiety: a study protocol for a randomized controlled trial. Front Psychiatry. 2022;13:941566. doi:10.3389/fpsyt.2022.941566

10. Zhao W, Li J, Wang Y, et al. Efficacy and safety of the “Xingnao Kaiqiao” acupuncture technique via intradermal needling to treat postoperative gastrointestinal dysfunction of laparoscopic surgery: study protocol for a randomized controlled trial. Trials. 2017;18(1):567. doi:10.1186/s13063-017-2319-3

11. Tsai CY, Liao W-L, Wu H-M, et al. Acupuncture improves neurological function and anti-inflammatory effect in patients with acute ischemic stroke: a double-blinded randomized controlled trial. Complement Ther Med. 2024;82:103049. doi:10.1016/j.ctim.2024.103049

12. Wen Q, Hu M, Lai M, et al. Effect of acupuncture and metformin on insulin sensitivity in women with polycystic ovary syndrome and insulin resistance: a three-armed randomized controlled trial. Hum Reprod. 2022;37(3):542–552. doi:10.1093/humrep/deab272

13. Jin Y, Yu X, Hu S, et al. Efficacy of electroacupuncture combined with intravenous patient-controlled analgesia after cesarean delivery: a randomized clinical trial. Am J Obstet Gynecol MFM. 2023;5(2):100826. doi:10.1016/j.ajogmf.2022.100826

14. Tu JF, Cao Y, Wang L-Q, et al. Effect of adjunctive acupuncture on pain relief among emergency department patients with acute renal colic due to urolithiasis: a randomized clinical trial. JAMA Network Open. 2022;5(8):e2225735. doi:10.1001/jamanetworkopen.2022.25735

15. Chen L, Qu Y, Cao J, et al. The increased inter-brain neural synchronization in prefrontal cortex between simulated patient and acupuncturist during acupuncture stimulation: evidence from functional near-infrared spectroscopy hyperscanning. Hum Brain Mapp. 2023;44(3):980–988. doi:10.1002/hbm.26120

16. Lee YS, Chae Y. Powerful effects of placebo needles. Acupunct Med. 2018;36(3):197–198. doi:10.1136/acupmed-2017-011516

17. Si X, Han S, Zhang K, et al. The temporal dynamics of eeg microstate reveals the neuromodulation effect of acupuncture with deqi. Front Neurosci. 2021;15:715512. doi:10.3389/fnins.2021.715512

18. Julian S, Rashid A, Baker R, et al. Attitudes of women with menstrual disorders to the use of clinical guidelines in their care. Fam Pract. 2010;27(2):205–211. doi:10.1093/fampra/cmp090

19. Yan L, Hu J, Yu Z, et al. The perceptions and experience of developing patient (version of) guidelines: a descriptive qualitative study with Chinese guideline developers. BMC Health Serv Res. 2023;23(1):789. doi:10.1186/s12913-023-09591-5

20. Hasson F, Keeney S, McKenna H. Research guidelines for the Delphi survey technique. J Adv Nurs. 2000;32(4):1008–1015. doi:10.1046/j.1365-2648.2000.t01-1-01567.x

21. Sun L-Q, Luo F-L, Chen S, et al. Acupuncture as an adjunctive therapy for gastric ulcer: a modified Delphi consensus study. Complement Ther Med. 2023;79:102997. doi:10.1016/j.ctim.2023.102997

22. Chen HY, Yeung W-F, Yang M-X, et al. Guideline Acupuncture for low back pain: a clinical practice guideline from the Hong Kong taskforce of standardized acupuncture practice. J Tradit Chin Med. 2022;42(1):140–147. doi:10.19852/j.cnki.jtcm.2022.01.009

23. Jünger S, Payne SA, Brine J, et al. Guidance on Conducting and REporting DElphi Studies (CREDES) in palliative care: recommendations based on a methodological systematic review. Palliat Med. 2017;31(8):684–706. doi:10.1177/0269216317690685

24. Cho J, Yoo S, Lee EE, et al. Impact of a nationwide medication history sharing program on the care process and end-user experience in a tertiary teaching hospital: cohort study and cross-sectional study. JMIR Med Inform. 2024;12:e53079. doi:10.2196/53079

25. Jiang Z, Sun W, Xu D, et al. The feasibility of half-dose contrast-enhanced scanning of brain tumours at 5.0 T: a preliminary study. BMC Med Imaging. 2024;24(1):88. doi:10.1186/s12880-024-01270-z

26. Pizones J, Moreno-Manzanaro L, Pupak A, et al. Reliability of a new digital tool for photographic analysis in quantifying body asymmetry in scoliosis. J Clin Med. 2024;13(7):2114. doi:10.3390/jcm13072114

27. Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med. 2016;15(2):155–163. doi:10.1016/j.jcm.2016.02.012

28. Cuesta-Frau D, Kouka M, Silvestre-Blanes J, et al. Slope entropy normalisation by means of analytical and heuristic reference values. Entropy. 2022;25(1):66. doi:10.3390/e25010066

29. Gao Q, Liu Y, Li H, et al. Comparison of several chemometric methods of libraries and classifiers for the analysis of expired drugs based on Raman spectra. J Pharm Biomed Anal. 2014;94:58–64. doi:10.1016/j.jpba.2014.01.027

30. Peng Y, Huang B, Luo Y, et al. Cross-sectional reference values of cerebral ventricle for Chinese neonates born at 25-41 weeks of gestation. Eur J Pediatr. 2022;181(10):3645–3654. doi:10.1007/s00431-022-04547-z

31. Liberati A. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ. 2009;339:b2700. doi:10.1136/bmj.b2700

32. Tricco AC, Lillie E, Zarin W. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Internal Med. 2018;169(7):467–473. doi:10.7326/M18-0850

33. Yoon SH, Lee H, Kwon C-Y, et al. Development of a survey form through Delphi study about adverse events associated with the miniscalpel needle, for application in prospective observational studies regarding safety of miniscalpel needles: study protocol. Medicine. 2018;97(41):e12736. doi:10.1097/MD.0000000000012736

34. Liu T, Research and formulation of WFAS standard technical benchmark of acupuncture-moxibustion: cupping (In Chinese); 2022.

35. Ge L, Wang Q, He Y, et al. Acupuncture for cancer pain: an evidence-based clinical practice guideline. Chin Med. 2022;17(1):8. doi:10.1186/s13020-021-00558-4

36. Wang Q, Li N, Li J, et al. A protocol of a guideline to establish the evidence ecosystem of acupuncture. Front Med Lausanne. 2021;8:711197. doi:10.3389/fmed.2021.711197

37. Ortiz M, Witt CM, Binting S, et al. A randomised multicentre trial of acupuncture in patients with seasonal allergic rhinitis--trial intervention including physician and treatment characteristics. BMC Complement Altern Med. 2014;14:128. doi:10.1186/1472-6882-14-128

38. Cotchett MP, Landorf KB, Munteanu SE, et al. Effectiveness of trigger point dry needling for plantar heel pain: study protocol for a randomised controlled trial. J Foot Ankle Res. 2011;4:5. doi:10.1186/1757-1146-4-5

39. Giese N, Heirs MK. Development of provisional acupuncture guidelines for pelvic pain in endometriosis using an e-delphi consensus process. J Integr Complement Med. 2023;29(3):169–180. doi:10.1089/jicm.2022.0659

40. Li XL, Cao H-J, Zhang Y-J, et al. Attitude and willingness of attendance for participating in or completing acupuncture trials: a cross-sectional study. Patient Prefer Adherence. 2019;13:53–61. doi:10.2147/PPA.S173202

41. Ma P, Liu X, Liu Z, et al. The SHARE: sHam Acupuncture REporting guidelines and a checklist in clinical trials. J Evid Based Med. 2023;16(4):428–431. doi:10.1111/jebm.12560

42. Nielsen A, Ocker L, Majd I, et al. Acupuncture intervention protocol: consensus process for a pragmatic randomized controlled trial of acupuncture for management of chronic low back pain in older adults: an NIH HEAL initiative funded project. Glob Adv Health Med. 2021;10:21649561211007091. doi:10.1177/21649561211007091

43. Nielsen A, Olson J, Quesada M, et al. Acupuncture intervention for acute pain in the Emergency Department trial: a consensus process. Acupunct Med. 2022;40(4):339–346. doi:10.1177/09645284221076507

44. Rotter G, Ahnert MW, Geue AV, et al. Acupuncture and osteopathic medicine for atopic dermatitis: a three-armed, randomized controlled explorative clinical trial. Clin Exp Dermatol. 2022;47(12):2166–2175. doi:10.1111/ced.15340

45. Smith CA, Grant S, Lyttleton J, et al. Using a Delphi consensus process to develop an acupuncture treatment protocol by consensus for women undergoing Assisted Reproductive Technology (ART) treatment. BMC Complement Altern Med. 2012;12:88. doi:10.1186/1472-6882-12-88

46. Smith CA, Zaslawski CJ, Zheng Z, et al. Development of an instrument to assess the quality of acupuncture: results from a Delphi process. J Altern Complement Med. 2011;17(5):441–452. doi:10.1089/acm.2010.0457

47. Su XT, Wang L-Q, Li J-L, et al. Acupuncture therapy for cognitive impairment: a delphi expert consensus survey. Front Aging Neurosci. 2020;12:596081. doi:10.3389/fnagi.2020.596081

48. Su XT, Wang L-Q, Zhang N, et al. Standardizing and optimizing acupuncture treatment for irritable bowel syndrome: a Delphi expert consensus study. Integr Med Res. 2021;10(3):100728. doi:10.1016/j.imr.2021.100728

49. Li JL, Wang L-Q, Zhang N, et al. Acupuncture as an adjunctive therapy for arrhythmia: a Delphi expert consensus survey. Cardiovasc Diagn Ther. 2021;11(5):1067–1079. doi:10.21037/cdt-21-201

50. Wang X, Chen Y, Liu Y, et al. Reporting items for systematic reviews and meta-analyses of acupuncture: the PRISMA for acupuncture checklist. BMC Complement Altern Med. 2019;19(1):208. doi:10.1186/s12906-019-2624-3

51. Zhang N, Wang L-Q, Li J-L, et al. The management of sciatica by acupuncture: an expert consensus using the improved delphi survey. J Pain Res. 2021;14:13–22. doi:10.2147/JPR.S280404

52. Bai Y, Hong Y-L, Chen B, et al. [Construction of a core outcome set in clinical research of acupuncture and moxibustion for treatment of adhesive capsulitis]. Zhongguo Zhen Jiu. 2023;43(6):701–705. Dutch. doi:10.13703/j.0255-2930.20220506-k0013

53. Cui Y. Evaluation of the quality of clinical acupuncture evidence based on the GRADE and research on the construction of core index set: primary depression (In Chinese); 2022.

54. Cui C. A study on the knowledge graph, diagnosis and treatment rules, and application standardization of the twelve mu points based on modern literature (In Chinese); 2022.

55. Deng Y, Formulating the diagnosis and treatment technical standard scheme of acupuncture and moxibustion in the treatment of primary dysmenorrhea based on the literature analysis and delphi expert consultation method; 2021.

56. Du S-H, Yang C, Chen S, et al. Evidence based Acupuncture-Moxibustion Key questions and outcomes for the clinical practice guideline on acupuncture and moxibustion for allergic rhinitis. World J Acupunct. 2022;32(2):101–108. doi:10.1016/j.wjam.2022.01.001

57. He YH, Xu N-G, Zhang H-B, et al. [Acupoint selection for cancer pain: based on current evidence and Delphi method]. Zhongguo Zhen Jiu. 2021;41(10):1161–1165. Dutch. doi:10.13703/j.0255-2930.20200831-k0002

58. Huang X. The study of acupuncture therapeutic strategy development for pre-diabetes based on evidence-based evidence and delphi method (In Chinese); 2021.

59. Li J, Grierson L, Wu MX, et al. [Quantitative research on operation behavior of acupuncture manipulation]. Zhongguo Zhen Jiu. 2014;34(3):247–251. Dutch

60. Li X. Establishment of core outcome indicator set for clinical research on acupuncture treatment of migraine (In Chinese); 2020.

61. Liu L. Identification of clinical questions for the development of clinical practice guideline on acupuncture and moxibustion for nonspecific low back pain using the modified delphi method (In Chinese); 2023.

62. Ran N. Study on technical operation standard of scalp acupuncture (In Chinese); 2021.

63. Shi L. Evidence-based research of an instrument for assessing the adequacy of acupuncture protocols in randomized controlled trials (In Chinese); 2023.

64. Yang C, Du S-H, Wang S-Z, et al. [Using modified Delphi method to determine the clinical questions in the Practice Guideline of Acupuncture and Moxibustion for Urinary Incontinence in Women]. Zhongguo Zhen Jiu. 2022;42(8):927–931. Dutch. doi:10.13703/j.0255-2930.20211007-k0003

65. Yang J. The study of the use of experts’ consensus in the evaluation scale of treating cervical spondylotic neurodynia with acupuncture and moxibustion (In Chinese); 2014.

66. Yue L. Development of patient reported outcome scale for premature ovarian insufficiency and test of clinical application of acupuncture (In Chinese); 2023.

67. Zhang H. Research on the formulation of acupuncture diagnosis and treatment plan for spasmodic torticollis (In Chinese); 2023.

68. Nair R, Aggarwal R, Khanna D. Methods of formal consensus in classification/diagnostic criteria and guideline development. Semin Arthritis Rheum. 2011;41(2):95–105. doi:10.1016/j.semarthrit.2010.12.001

69. Benzon HT, Joshi GP, Gan TJ, et al. Development, reporting, and evaluation of clinical practice guidelines. Anesth Analg. 2019;129(6):1771–1777. doi:10.1213/ANE.0000000000004441

70. Brown J, Cox L, Mulligan K, et al. Gaining consensus on emotional wellbeing themes and preferences for digital intervention type and content to support the mental health of young people with long-term health conditions: a Delphi study. Health Expect. 2024;27(2):e14025. doi:10.1111/hex.14025

71. Gianola S, Bargeri S, Pellicciari L, et al. Evidence-informed and consensus-based statements about SAFEty of physical agent modalities practice in physiotherapy and rehabilitation medicine (SAFE PAMP): a national Delphi of healthcare scientific societies. BMJ Open. 2024;14(3):e075348. doi:10.1136/bmjopen-2023-075348

72. Bradley CT, Brasel KJ. Developing guidelines that identify patients who would benefit from palliative care services in the surgical intensive care unit. Crit Care Med. 2009;37(3):946–950. doi:10.1097/CCM.0b013e3181968f68

73. Knottnerus JA, Tugwell P. Evidence-based medicine: achievements and prospects. J Clin Epidemiol. 2017;84:1–2. doi:10.1016/j.jclinepi.2017.02.006

74. Sinha IP, Smyth RL, Williamson PR. Using the Delphi technique to determine which outcomes to measure in clinical trials: recommendations for the future based on a systematic review of existing studies. PLoS Med. 2011;8(1):e1000393. doi:10.1371/journal.pmed.1000393

75. Verhagen AP, de Vet HCW, de Bie RA, et al. The Delphi list: a criteria list for quality assessment of randomized clinical trials for conducting systematic reviews developed by Delphi consensus. J Clin Epidemiol. 1998;51(12):1235–1241. doi:10.1016/S0895-4356(98)00131-0

76. Weber F, Kloek C, Stuhrmann S, et al. Usability and preliminary effectiveness of an app-based physical activity and education program for people with Hip or knee osteoarthritis - a pilot randomized controlled trial. Arthritis Res Ther. 2024;26(1):83. doi:10.1186/s13075-024-03291-z

77. El Naqa I, Boone JM, Benedict SH, et al. AI in medical physics: guidelines for publication. Med Phys. 2021;48(9):4711–4714. doi:10.1002/mp.15170

78. Liu X, Cruz Rivera S, Moher D, et al. Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension. Lancet Digit Health. 2020;2(10):e537–e548. doi:10.1016/S2589-7500(20)30218-1

79. Norgeot B, Quer G, Beaulieu-Jones BK, et al. Minimum information about clinical artificial intelligence modeling: the MI-CLAIM checklist. Nat Med. 2020;26(9):1320–1324. doi:10.1038/s41591-020-1041-y

80. Hurkmans C, Bibault J-E, Clementel E, et al. Assessment of bias in scoring of AI-based radiotherapy segmentation and planning studies using modified TRIPOD and PROBAST guidelines as an example. Radiother Oncol. 2024;194:110196. doi:10.1016/j.radonc.2024.110196

81. Guzys D, Dickson-Swift V, Kenny A, et al. Gadamerian philosophical hermeneutics as a useful methodological framework for the Delphi technique. Int J Qual Stud Health Well-Being. 2015;10:26291. doi:10.3402/qhw.v10.26291

82. Hyatt A. Development of consensus quality indicators for cancer supportive care: a Delphi study and pilot testing. BMC Health Serv Res. 2024;24(1):377. doi:10.1186/s12913-024-10876-6

83. Kennedy HP. Enhancing Delphi research: methods and results. J Adv Nurs. 2004;45(5):504–511. doi:10.1046/j.1365-2648.2003.02933.x

84. Hassoun HK. Iraqi experts consensus on the management of relapsing remitting multiple sclerosis in adults. Curr Med Res Opin. 2024;2024:1–20.

85. Dolunay A, Temel AC. The relationship between personal and professional goals and e