Qin Wang1†
Nian Li2†
Juan Li3
Ying He4
Yuxi Li3
Dongling Zhong3
Xiaobo Liu3
Jin Fan3
Rongjiang Jin3
Deying Kang1*
Yonggang Zhang1,4*- 1Department of Evidence Based Medicine and Clinical Epidemiology, West China Hospital, Sichuan University, Chengdu, China
- 2Department of Medical Affairs, West China Hospital, Sichuan University, Chengdu, China
- 3School of Health Preservation and Rehabilitation, Chengdu University of Traditional Chinese Medicine, Chengdu, China
- 4Department of Periodical Press and National Clinical Research Center for Geriatrics, Nursing Key Laboratory of Sichuan Province, West China Hospital, Sichuan University, Chengdu, China
This is a protocol for developing a guideline to establish the evidence ecosystem of acupuncture. It describes all steps that will be followed in line with the World Health Organization Handbook for Guideline Development and the Reporting Items for practice Guidelines in Healthcare (RIGHT). The key steps included guideline protocol development, guideline registration, systematic review of acupuncture evidence issues, systematic review of methods for establishing evidence ecosystem, survey of acupuncture stakeholders regarding potential acupuncture evidence issues, development of potential items for guidelines, Delphi method for guideline item development, consensus meeting, drafting guideline, peer review, approval, and publishing. This future guideline will help to establish evidence ecosystem of acupuncture, which will facilitate the application of acupuncture in clinical practice.
Introduction
The United Nations Educational, Scientific, and Cultural Organization (UNESCO) had placed acupuncture on the Representative List of the Intangible Cultural Heritage of Humanity (1) and this therapy had been deemed safe and effective. Acupuncture is an important Chinese medicine treatment method suitable for a wide spectrum of diseases (2). More than 60,000 randomized controlled trials (RCTs), 6,000 systematic reviews, and 1,000 recommendations had been published, while some studies were published in top journals (3, 4), which promoted the use of acupuncture worldwide.
Acupuncture clinical practice should be based on high-quality evidence, which could help in decision-making. Thus, acupuncture research should provide sufficient data to enable funders, reviewers, and steering committees to appraise the scientific and methodological rigor of the studies, and for the researchers to replicate and implement these studies (5). However, many RCTs were of low quality and were characterized by incorrect random allocation, vague statistical analyses, and methodological issues (6). Meanwhile, effect sizes do not provide policymakers with information on how an intervention might be replicated in their specific context, or whether trial outcomes will be reproduced. In addition, some systematic reviews were also of low quality, containing insufficient comparisons and out-of-date, thereby resulting in redundancy and research gaps (7). Furthermore, some acupuncture guidelines contained insufficient interpretations and explanations, and the majority was in the Chinese language (8, 9), thereby decreasing the promotion of acupuncture. High-quality and up-to-date systematic reviews require substantial time and resources, and although the synthesis of evidence is directly affected by the quality of primary research (10), there has been little effort to connect evidence generation to evidence synthesis. These issues had been highlighted and exacerbated with an increasing use of acupuncture. Relevant, accessible, up-to-date, and trustworthy syntheses of high-quality evidence are urgently needed in clinical practice. Although thousands of RCTs had been conducted, the authors often rushed to publish or to communicate through non-peer-reviewed preprints. With the development of acupuncture, it is also worthy to pay attention to the component issue of acupuncture (11). Meanwhile, process evaluation within trials, which has been used to assess fidelity and quality of implementation, clarify causal mechanisms, and identify contextual factors associated with variation in outcomes should also be paid attention to. Still, there were limited studies about the process evaluation within acupuncture trials. A new model was needed to address these challenges and to help connect evidence generation, synthesis, and decision-making. Therefore, the development of an evidence ecosystem is a possible solution.
The evidence ecosystem (7, 10, 12–14) was a concept developed by the MAking GRADE the Irresistible Choice (MAGIC) foundation (14) to promote the continuous cycle of evidence production, evidence synthesis, and guidance generation. Up-to-date evidence is needed to flow efficiently between different stakeholders. These stakeholders included evidence producers (conducting primary research), evidence synthesizers (summarizing the research into systematic reviews or evidence syntheses), evidence processors and disseminators (producing evidence-informed decision products such as health systems guidance and policy briefs), and evidence implementers [individuals responsible for implementing evidence-informed decisions within health systems (7, 10, 15–18), such as program managers and policymakers, and those involved in delivering and using health services, such as service providers and citizens]. In addition, there is no acupuncture evidence ecosystem, so it is urgent to establish it. Establishing an acupuncture evidence ecosystem (14) involved different stakeholders, such as researchers, funders, and reviewers in literature reviews, process evaluation, workshops, and discussions at conferences and seminars. Thus, a guideline should be developed. The guideline should not only include treatment methods of acupuncture, but also realized the integration and utilization of acupuncture evidence-based decision-making information resources from different sources. To ensure a high-quality development, a protocol is needed (Figure 1). The current study was the protocol which introduced the key steps for the development of the guideline. It would increase transparency and objectivity of the guideline.
Figure 1. The process of establishing an acupuncture evidence ecosystem.
Methods
Guidance of Guideline
The guideline will be evidence-based. It will be developed according to the World Health Organization Handbook for Guideline Development and reported according to the Reporting Items for practice Guidelines in Healthcare (RIGHT). It has been registered on the Global Practice Guidelines Registry Platform (registration number: IPGRP-2021CN106).
Guideline Group
The guideline project was launched by West China Hospital of Sichuan University in April 2021. Chinese Evidence-based Medicine Center or Cochrane China would provide technical support. The guideline working groups will be established in mid-2021 and will comprise four groups: a guideline steering committee, a guideline formulation group, a guideline secretary group, and an external review group.
The steering committee group will comprise a clinical chairman, a methodology chairman, an executive chairman, and acupuncture experts. Their responsibilities will be as follows: determine the scope according to PICO strategy; establish a guideline secretary group, evidence evaluation group, and consensus expert group; manage the statement of conflict of interest; supervise the process of guideline formulation; approve the recommendations; and monitor and evaluate the revised guideline.
Experts of evidence-based medicine, acupuncturists, editors, policymakers, and computer scientists will be invited as the guideline formulation group. Their responsibilities will be as follows: perform a rapid and high-quality retrieval of the relevant evidence, evaluation, synthesis, and quality classification; generate a summary table and draft recommendations; draft, modify, and improve the guideline; and promote and disseminate the guideline.
Students majoring in evidence-based medicine and acupuncture will be selected as the guideline secretary group. Their responsibilities will be as follows: coordinate and organize the work during guideline formulation; investigate the essential clinical problems and outcomes; organize three Delphi expert consensus conferences (online or face-to-face); draft, modify, and improve the guideline; and conduct data analysis and evidence synthesis.
The external review group will consist of acupuncture stakeholders, including evidence-based medicine experts, acupuncture experts, TCM experts, editors, policymakers, computer scientists, and patients. Their responsibilities will be as follows: evaluate guideline scope and the issues covered and review the draft guideline.
We will use a systematic approach to design the whole evidence ecosystem of acupuncture, including establishing clear descriptions of ecosystem theory, processing evaluations in acupuncture clinical trials, and identifying of key process questions in ecosystem. In addition, each process in the ecosystem will be different, and the guidance facilitates planning and conducting a process ecosystem.
All members involved in the guideline development will declare their interests and relevant relationships, and also complete a statement of interest form. If conflicts of interest exist, the expert will be excluded from participation in evidence evaluation and recommendation.
Problems With Guideline Construction
It is important to develop a high-quality and useful acupuncture evidence ecosystem. We will define the problems in guideline construction using the Population, Intervention, Comparator, Outcome strategy (PICO).
Literature Search
We will search RCTs, systematic reviews, meta-analyses, overviews of systematic reviews and clinical practice guidelines, clinical decision analyses, health technology evaluations, and health policy research methods using the following electronic databases: PubMed, EMbase, CBM, CNKI, VIP, and WanFang Data. We will search the gray literature using the following websites to collect potential guidelines: the National Guideline Clearinghouse (Agency for Healthcare Research and Quality), Canadian Medical Association InfoBase, Guidelines International Network, PEDro, Trip Database, American College of Physicians Clinical Recommendations, Australian Government, National Health and Medical Research Council, Health Services/Technology Assessment Texts, Institute for Clinical Systems Improvement, National Institute for Health and Clinical Excellence (NICE) Guidance, NICE Pathways, New Zealand Guidelines Group, Scottish Intercollegiate Guidelines Network (SIGN), and the WHO guidelines approved by the Guidelines Review Committee. Search terms are shown in Table 1.
Table 1. Search strategy used for PubMed database.
Study Inclusion and Exclusion Criteria
The following inclusion criteria will be used: RCTs, systematic reviews, meta-analyses, overviews of systematic reviews and clinical practice guidelines, clinical decision analyses, health technology evaluations, and health policy research methods in acupuncture. The following exclusion criteria will be used: non-Chinese and non-English documents; conference abstracts, reviews, and so on; complete unavailable data; old versions of guidelines, consensus, standards, procedures, and specifications formulated by the same institution; organization or management-related guidelines, standards procedures, and specifications formulated and released by non-associated societies and groups; interpretations, translations, plans, reviews, and other articles related to guidelines, consensuses, standards, procedures, and norms; research on compliance with guidelines, consensuses, standards, procedures, and norms; lastly, guidelines, standards, processes, and normative after-effect evaluation research, etc.
Study Selection
Two independent reviewers (QW and YH) will screen titles and abstracts to identify potentially eligible documents, which will be retrieved in full text for final review. Duplicate records will be removed. Any disagreements will be discussed and resolved by a third reviewer (YGZ).
Data Extraction
We will design a data extraction form based on study characteristics and conduct a pre-experiment. The following content will be extracted: author, publication year, published journal, disease, sample size, location, and quality of evidence. If extra information about the study is required, we will contact the author.
Quality Assessment of Included Studies
We will assess the methodological quality of the included studies using several tools (Table 2). We will assess the quality of evidence using the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) (15–17) system. We will evaluate the methodological quality and reporting quality of the included guidelines using the Appraisal of Guidelines for Research and Evaluation (AGREE) II tool (18–21) and the RIGHT statement (22), respectively. The quality of evidence will be divided into four levels: high (A), medium (B), low (C), and very low (D) basing on GRADE system. We will use internationally agreed standards to make transparent recommendations (Table 3).
Table 2. Document types and evaluation tools.
Table 3. Factors that determine the direction and strength of a recommendation.
Data Synthesis and Presentation
A synthesis of qualitative research will be conducted as part of the scope of the guideline and this will also help to define and to refine the questions. A mixed-methods synthesis will combine a meta-analysis of quantitative data and a qualitative analysis. We will summarize the recommendations for the acupuncture evidence ecosystem. The main outcomes will be as follows: author, title and subtitle, type and version, implementation status, target disease, population, user, country or region, literature search method, funding, and patient preferences. The outcome of qualitative research is patient preferences. We will evaluate the effects by using “Preferences about Therapy Questionnaire” on a 5-point Likert scale. We will perform a systematic review of the main principles, basic methods, and procedures for formulating key core technical items of the evidence ecosystem.
Development of Recommendations
Once the evidence is retrieved, synthesized, and assessed, the evidence will be used to develop recommendations. The recommendations will be surveyed by 3 rounds of Delphi methods. We will conduct an online-based Delphi process. It is an accepted method in medical research to gain consenting information from a group of experts. The anonymous multi-step survey comprised three rounds, starting with open-ended questions and ending with a consensus-scoring assessment of the guideline. Design and analysis methods will be adopted and modified from a recent guideline for the World Health Organization Handbook for Guideline Development. The online Delphi process is conducted using the survey tool Questionnaire star. We will submit them to the guideline-steering committee for recommendations. Then, the draft of the guideline will be developed. A full draft will recirculate for further review before being revised and approved.
Peer Review
The guideline will undergo peer review. The review and response processes will be recorded. The guideline will be revised after discussion with the peer review comments. Peer review opinions will be submitted to the guideline steering committee for further assessment. The next version of the guideline will be developed.
Dissemination of Guidelines
Dissemination involves making guidelines accessible, advertising their availability, and distributing them widely. The guidelines will be disseminated through association. The guideline will be submitted to peer review journals, such as the Journal of Evidence-based Medicine or other journals.
Establishment of the Evidence Ecosystem on Acupuncture
The guideline is used for establishing the evidence ecosystem of acupuncture. After the approval of the guideline, the evidence ecosystem will be established.
Discussion
This is a protocol for the development of a guideline, which will be used to establish an acupuncture evidence ecosystem. Several challenges of developing the guideline should be considered. First, sustainability is a potential issue in generating the data. Thus, an appropriate strategy should be considered when developing the guideline. Seeking assistance from the central government or industry may be useful in developing the ecosystem. A potentially good example from the Cochrane Collaboration may be helpful. We should move to a long-term, inexpensive, and sustainable structure with a website that is more accessible and useful to end-users. Collaboration with associations or researchers would be useful. In addition, the ecosystem will be used to strengthen clinical practice, which we will provide a guidance for carrying out the process evaluation of acupuncture clinical trials. Therefore, efficient flow between evidence producers, evidence synthesizers, evidence processers and disseminators, and evidence implementers in the evidence ecosystem should be well defined. International scientists should also be invited to contribute to the ecosystem and assist with decision-making. Third, because the quality of evidence ecosystem of acupuncture is still uncertain, the recommendations should be carefully considered (23, 24). Fourth, the ecosystem should be considered a living system. Thus, the development of a living guideline should be considered, which will benefit the development of the evidence ecosystem. Fifth, the ecosystem should further consider the development of acupuncture discipline, such as acupuncture manipulation metrology which has occupied an important strategic position (25).
The acupuncture evidence ecosystem model will illustrate how evidence is transferred between different key stages to strengthen health systems and inform care. It will show the importance of “closing the loop” between evidence producers, synthesizers, disseminators, and users. It will also help to process evaluation and to provide potential progress in developing high-quality acupuncture trials. Although the acupuncture evidence ecosystem regarding evidence generation, synthesis, and translation contains thriving areas, others remained unclear. Recognition of these strengths and weaknesses, and the identification of future paths to a more robust system would allow health care and public institutions to focus on the real problems with acupuncture evidence rather than the purported problems highlighted by universal critics. A future priority is to further strengthen initiatives to establish an acupuncture evidence ecosystem, to translate evidence into policy and practice, and to sustain the capacity for continuous technical support for health system policy development and implementation.
In summary, this protocol provides a plan for developing a guideline. The future guideline will help to establish evidence ecosystem of acupuncture, which will facilitate the application of acupuncture in clinical practice.
Author Contributions
YZ and DK: design the study. NL, YH, DZ, XL, JF, and RJ: data collection and analysis. QW and YZ: draft the protocol. YZ and JL: revise the protocol. All authors contributed to the article and approved the submitted version.
Funding
This protocol was supported by the National Natural Science Foundation of China (Grant No. 82004213) and the Project of Sichuan Provincial Department of Science and Technology (Grant No. 2021YFH0191).
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher's Note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
1. Litscher G. Integrative laser medicine and high-tech acupuncture at the medical University of Graz, Austria, Europe. Evid Based Compt Alternat Med. (2012) 2012:103109. doi: 10.1155/2012/103109
2. Zhao Y, Fang Y, Zhang Z, Liu J, Qin S, Li W, et al. Arousal effect and potential mechanism of dopamine-mediated acupuncture on traumatic brain injury. Acupunct Herbal Med. (2021) 1:22–30. doi: 10.1097/HM9.0000000000000005
3. He Y, Guo X, May BH, Zhang AL, Liu Y, Lu C, et al. Clinical evidence for association of acupuncture and acupressure with improved cancer pain: a systematic review and meta-analysis. JAMA Oncol. (2020) 6:271–78. doi: 10.1001/jamaoncol.2019.5233
4. Vickers AJ, Vertosick EA, Lewith G, MacPherson H, Foster NE, Sherman KJ, et al. Acupuncture for chronic pain: update of an individual patient data meta-analysis. J Pain. (2018) 19:455–74. doi: 10.1016/j.jpain.2017.11.005
5. Hrobjartsson A, Emanuelsson F, Thomsen ASS, Hilden J, Brorson S. Bias due to lack of patient blinding in clinical trials. A systematic review of trials randomizing patients to blind and nonblind sub-studies. Int J Epidemiol. (2014) 43:1272–83. doi: 10.1093/ije/dyu115
6. Lu T, Lu C, Li H, Xing X, Deng X, Li X, et al. The reporting quality and risk of bias of randomized controlled trials of acupuncture for migraine: methodological study based on STRICTA and RoB 2.0. Complement Ther Med. (2020) 52:102433. doi: 10.1016/j.ctim.2020.102433
7. Boutron I, Crequit P, Williams H, Meerpohl J, Craig JC, Ravaud P. Future of evidence ecosystem series: 1. Introduction Evidence synthesis ecosystem needs dramatic change. J Clin Epidemiol. (2020) 123:135–42. doi: 10.1016/j.jclinepi.2020.01.024
8. Guo L-H, Ma Y, Wu X-D. AGREE-based evaluation and content analysis of evidence-based clinical practice guidelines for acupuncture-moxibustion. Zhongguo Zhen Jiu. (2019) 39:1223–8. doi: 10.13703/j.0255-2930.2019.11.023
9. Guo Y, Zhao H, Wang F, Li SN, Sun YX, Han MJ, et al. Recommendations for acupuncture in clinical practice guidelines of the national guideline clearinghouse. Chin J Integr Med. (2017) 23:864–70. doi: 10.1007/s11655-016-2750-4
10. Akl EA, Haddaway NR, Rada G, Lotfi T. Future of Evidence Ecosystem Series: Evidence synthesis 2.0: when systematic, scoping, rapid, living, and overviews of reviews come together. J Clin Epidemiol. (2020) 123:162–5. doi: 10.1016/j.jclinepi.2020.01.025
11. Zhao N-Q, Yuan J-Y, Guo L-H, Dong G-F, Wang X, Wu X-D. Consideration on existing problems in development of clinical practice guidelines for acupuncture and moxibustion in China. Zhongguo Zhen Jiu. (2021) 41:445–8. doi: 10.13703/j.0255-2930.20200905-k0003
12. Créquit P, Boutron I, Meerpohl J, Williams HC, Craig J, Ravaud P. Future of evidence ecosystem series: 2. Current opportunities and need for better tools and methods. J Clin Epidemiol. (2020) 123:143–52. doi: 10.1016/j.jclinepi.2020.01.023
13. Vandvik PO, Brandt L. Future of evidence ecosystem series: Evidence ecosystems and learning health systems: why bother? J Clin Epidemiol. (2020) 123:166–70. doi: 10.1016/j.jclinepi.2020.02.008
14. Ravaud P, Crequit P, Williams HC, Meerpohl J, Craig JC, Boutron I. Future of evidence ecosystem series: 3. From an evidence synthesis ecosystem to an evidence ecosystem. J Clin Epidemiol. (2020) 123:153–61. doi: 10.1016/j.jclinepi.2020.01.027
15. Guyatt GH, Oxman AD, Schuenemann HJ, Tugwell P, Knottnerus A. GRADE. guidelines: a new series of articles in the Journal of Clinical Epidemiology. J Clin Epidemiol. (2011) 64:380–82. doi: 10.1016/j.jclinepi.2010.09.011
16. Guyatt G, Oxman AD, Akl EA, Kunz R, Vist G, Brozek J, et al. GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol. (2011) 64:383–94. doi: 10.1016/j.jclinepi.2010.04.026
17. Balshem H, Helfand M, Schuenemann HJ, Oxman AD, Kunz R, Brozek J, et al. GRADE guidelines: 3. Rating the quality of evidence. J Clin Epidemiol. (2011) 64:401–06. doi: 10.1016/j.jclinepi.2010.07.015
18. Wan K, Liang H, Yan G, Zou B, Huang C, Jiang M. A quality assessment of evidence-based guidelines for the prevention and management of ventilator-associated pneumonia: a systematic review. J Thorac Dis. (2019) 11:2795. doi: 10.21037/jtd.2019.06.56
19. Simancas-Racines D, Montero-Oleas N, Vernooij RWM, Arevalo-Rodriguez I, Fuentes P, Gich I, et al. Quality of clinical practice guidelines about red blood cell transfusion. J Evid Based Med. (2019) 12:113–24. doi: 10.1111/jebm.12330
20. Wu D, Jiang W, Yu L, Wang Y, Tao Y, Tang H, et al. Quality assessment of clinical practice guidelines for infectious diseases in China. J Evid Based Med. (2018) 11:95–100. doi: 10.1111/jebm.12293
21. Bhaumik S, Jagadesh S, Ellatar M, Kohli N, Riedha M, Moi M. Clinical practice guidelines in India: Quality appraisal and the use of evidence in their development. J Evid Based Med. (2018) 11:26–39. doi: 10.1111/jebm.12285
22. Chen Y, Yang K, Marusic A, Qaseem A, Meerpohl JJ, Flottorp S, et al. A reporting tool for practice guidelines in health care: the RIGHT statement. Ann Intern Med. (2017) 166:128. doi: 10.7326/M16-1565
23. Yang C, Hao Z, Zhang LL, Guo Q. Efficacy and safety of acupuncture in children: an overview of systematic reviews. Pediatr Res. (2015) 78:112–9. doi: 10.1038/pr.2015.91
24. Errington-Evans N. Acupuncture for anxiety. CNS Neurosci Ther. (2012) 18:277–84. doi: 10.1111/j.1755-5949.2011.00254.x
Keywords: acupuncture, guideline, evidence ecosystem, living systematic reviews, clinical trial
Citation: Wang Q, Li N, Li J, He Y, Li Y, Zhong D, Liu X, Fan J, Jin R, Kang D and Zhang Y (2022) A Protocol of a Guideline to Establish the Evidence Ecosystem of Acupuncture. Front. Med. 8:711197. doi: 10.3389/fmed.2021.711197
Received: 18 May 2021; Accepted: 19 November 2021;
Published: 15 February 2022.
Edited by:
Rocio de la Vega, University of Malaga, SpainReviewed by:
Hideki Maeda, Meiji Pharmaceutical University, JapanSegundo Mariz, European Medicines Agency, Netherlands
Copyright © 2022 Wang, Li, Li, He, Li, Zhong, Liu, Fan, Jin, Kang and Zhang. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Deying Kang, deyingkang@126.com; Yonggang Zhang, jebm_zhang@yahoo.com
†These authors have contributed equally to this work