Achieving global surgical excellence: an evidence-based framework to guide surgical quality improvement programs in low and middle income countries

Objectives: There is a lack of evidence-based guidelines for enhancing global surgical care delivery. We propose a set of recommendations to serve as a framework to guide surgical quality improvement and scale-up initiatives in low and middle income countries (LMICs).

Methods: From January-December 2019, we reviewed the available literature and their application toward LMIC settings. The first initiative was the establishment of Best Practices Recommendations intended to summarize best-level evidence around quality improvement processes that have shown to decrease morbidity and mortality in LMICs. The GRADE level of evidence and strength of the recommendation were assigned in accordance with the WHO handbook for guidelines development. The second initiative was the scale-up of principles and practices by establishing international expert consensus on the optimal organization of surgical services in LMICs using a modified Delphi methodology.

Results: Recommendations for three topic areas were established: reducing surgical site infections, improving quality of trauma systems, and interventions to reduce maternal and perinatal mortality. 27 studies were included in a quantitative synthesis and meta-analysis for interventions reducing surgical site infections, 27 studies for interventions improving the quality of trauma systems, and 14 studies for interventions reducing maternal and perinatal mortality. Using Delphi methodology, an international expert panel established consensus that district hospitals should place the highest priority on developing surgical services for low complexity, high volume conditions. At the national level, emergency and essential surgical care should be integrated within national Universal Health Coverage frameworks.

Conclusions: This project fills a critical cap in the rapidly developing field of global surgery: gathering evidence-based, practical, and cost-effective solutions that will serve as a guide for the efficient planning and allocation of resources necessary to promote quality and safe essential surgical services in LMICs.

1. Background

Since 2015, increasing surgical capacity in low- and middle-income countries (LMICs) has received increased attention in the international community with the inception of several landmark initiatives including the Disease Control Priorities, third edition (DCP3), on Essential Surgery and the Lancet Commission on Global Surgery (1, 2). In response, many countries have undertaken the creation of national policies and initiatives to strengthen infrastructure for surgical systems (3, 4). Similarly, private organizations, non-governmental organization (NGOs), universities, academic societies, surgical colleges and other on-the-ground providers have worked to increase capacity for surgical, obstetric, trauma, and anesthesia care in LMICs (5–8). While strides have been made in the assessment of surgical capacity and the establishment of national priorities for enhancing surgical care delivery, there is a lack of evidence-based guidelines to guide the scale up of high quality and safe surgical, obstetric, trauma, and anesthesia care.

Quality improvement is essential to improving morbidity and mortality in surgical systems (9, 10). Kruk et al. published an impactful work assessing the quality of health systems in the Sustainable Goals Era and found that healthcare in LMICs is often inadequate, of variable quality, and exacerbated in vulnerable populations (11). Confidence in their healthcare system is a valuable judge of a nation's overall performance, and this paper reports that one in three people in LMICs indicated negative experiences with their respective healthcare systems with regard to accessibility, respect, and communication (11, 12). There is also significant variation in surgical outcomes, with adults up to three times, and children seven times more likely to die after emergency abdominal surgery in LMICs compared to high-income countries (HICs) (13). It is estimated that 23 million disability-adjusted life-years are lost each year due to in-hospital adverse events alone and that two-thirds of these occur in LMICs (10). One challenge to establishing international best practices is that the surgical landscape looks vastly different across settings based on existing health infrastructure and the number and distribution of health care providers (8, 10–12). Some countries are more mature in their surgical systems, with well-established trauma systems, outcomes surveillance, and effective referral systems (11, 12). Other countries may rely on a patchwork of organizations, each covering different domains in surgical service delivery (5, 7, 12). One of the keys to developing high quality healthcare systems is the ability to scale-up at a respectable rate without losing impact. White et al. performed a thorough systematic review and found that from 1960 to 2020, there were only 31 studies that reported scale-up interventions as quality improvement measures, with the implementation of the WHO surgical safety checklist as the most common intervention (14). Similarly in a systematic review by Brima et al, of the 49 articles that reported hospital-based quality improvement studies in Africa, use of the surgical safety checklist comprised 29% and reduction of surgical site infections comprised 25% (15). Other interventions well-known to high income settings such as antimicrobial stewardship programs and streamlined postoperative care protocols are lacking in LMICs (14, 15). As all countries move toward self-sufficient surgical systems, practical and adaptable guidelines are needed to inform quality improvement initiatives across unique national, regional, and local settings.

In December of 2018, the G4 Alliance and the International Society of Surgery (ISS/SIC) established the International Standards and Guidelines for Quality Safe Surgery and Anesthesia (ISG-QSSA) Working Group. The group consisted of clinicians, epidemiologists, Ministers of Health, and research methodologists. The group was tasked to gather and compile existing evidence-based guidelines and recommendations for quality improvement in LMICs. From January to December 2019, the ISG-QSSA held a series of meetings to review the available literature and their application toward LMIC settings. Ultimately, two research initiatives were established. The specific objectives of the Best Practice Recommendations initiative were to evaluate the literature on quality improvement interventions, processes, and structures which reduce mortality and morbidity in LMICs. A secondary objective was to evaluate the balance between harms and benefits of interventions, patient and provider preferences and concerns, and the feasibility of introducing the interventions into LMIC settings. Subsequently, the objective of the scale-up principles and practices initiative was to establish international expert consensus on a set of statements describing the optimal distribution and prioritization of surgical services in LMICs based on prior published evidence on the efficacy of decentralizing or regionalizing surgical services (16–19). The statements covered three major areas: (1) the optimal distribution of surgical services, (2) the optimal prioritization of surgical services, and (3) policies and practices for enhancing surgical scale-up. This paper describes the research initiatives and findings of the G4 Alliance and the International Society of Surgery (ISS/SIC) International Standards and Guidelines for Quality Safe Surgery and Anesthesia (ISG-QSSA) Working Group (20, 21). We propose a set of recommendations to serve as a framework to guide surgical quality improvement and scale-up initiatives in LMICs.

2. Methods

2.1. Best practice recommendations

A systematic review was undertaken by the ISG-QSSA Working Group using the PubMed, Embase, Cochrane, WHO regional databases, Google Scholar, and Grey literature databases to summarize the interventional data from LMICs that have shown to improve morbidity and mortality. Three systematic reviews were ultimately completed and recommendations were created based on the available evidence. To comply with current standards for evidence assessment in the formulation of policy recommendations, methodology adapted from the WHO Handbook for Guidelines Development was used (22). A Guideline Development Group, consisting of 8 experts in the field, were tasked to collate the most relevant and significant clinical questions and assign expert review to externally validate or reject the proposed questions. Review questions were developed within a framework population (restricted to LMICs), presence or absence of interventions under investigation, and mortality and morbidity outcomes. A consensus meeting was held in 2020 to confirm the findings of the systematic reviews and associated recommendation and assess the quality of evidence using GRADE methodology using independent reviewers (23–26). Of the eight criterion for rating the quality of evidence as described by the GRADE methodology, we focused on the three most dependable criteria as suggested by Malmivara et al. (24) We used risk of bias, inconsistency of findings, and publication bias to up or downgrade the quality of studies included (24, 25). Final recommendations were approved by the ISG-QSSA and formally ratified by the G4 Alliance Permanent Council on November 5th, 2021.

2.2. Scale-up principles and practices

33 international surgical experts convened in Suva, Fiji at a meeting hosted by the Fiji Ministry of Health in March 2020 where a roundtable discussion was held to refine modified Delphi statements around the topics of complexity, volume, and acuity of surgical care in LMICs. By providing input on their needs and the potential utility of the findings, 27 statements were collaboratively created which covered the global definitions of organization, distribution, and prioritization of surgical services in LMICs. Next, an open call was made for experienced surgeons and public health experts in the LMIC setting, and nominees by the G4 Alliance and Ministries of Health were sent a survey to confirm their credentials and experience. Using specific criteria such as: geographic scope, relevant expertise, work setting, and recognized impact, the final 53 participants were chosen representing 27 different LMICs. The experts were chosen for their recognized authority, clinical expertise in a range of surgical services, diverse geographical scope, and work in both public and private sectors. Half of the participants represented general surgery while the other half represented other surgical specialties. The participants were distributed across Africa, Asia, Europe, the Americas, and Oceania. A two-round Delphi process was used to establish consensus among this international panel of surgeon experts from LMICs. The first round involved independent ratings of the agreed upon statements without any interaction with the other participants. Participants chose the following options for each of the statements provided: strongly agree, agree, neutral, disagree, and strongly disagree. They were also instructed to make comments or edits to the proposed statements. The second round enabled participants to see de-identified comments from other involved members, but again, they provided ratings independently without engaging in discussion. The process was completed after two rounds because of the high rate of agreement between the 53 independent reviewers. Data was collected on an online platform, REDCap, which ensured anonymity during the two round process.

3. Results

3.1. Best practice recommendations

Recommendations covering three topic areas were established: (1) reducing surgical site infections, (2) improving quality of trauma systems, and (3) interventions to reduce maternal and perinatal mortality. For interventions reducing surgical site infections, 27 studies were included in a quantitative synthesis and meta-analysis. For interventions improving the quality of trauma systems, 27 studies were included. For interventions reducing maternal and perinatal mortality, 14 studies were included. The following heat maps demonstrate the countries in which research studies were conducted in each topic area (Figures 1–3).

FIGURE 1

Figure 1. Heat map of studies assessing quality improvement processes, interventions, and structure for reducing morbidity and mortality from surgical site infections in LMICs.

FIGURE 2

Figure 2. Heat map of studies assessing quality improvement processes, interventions, and structure of trauma systems in LMICs.

FIGURE 3

Figure 3. Heat map of studies assessing the impact of quality improvement processes, interventions, and structure on maternal and perinatal mortality in LMICs.

Table 1 describes the final eleven Best Practice Recommendations. The GRADE level of evidence and strength of the recommendation are documented in accordance with the WHO handbook for guidelines development. A “strong” recommendation was made when it was clear that the net desirable consequences of the strategy outweighed those of no intervention. A “conditional” recommendation was made when it was less clear whether the net desirable consequences of the specified strategy outweighed those of no intervention.

TABLE 1

Table 1. The G4 alliance best practice recommendations for quality improvement processes in LMICs.

3.2. Scale-up principles and practices

Recommendations covering three topic areas were established: (1) the optimal distribution of surgical services, (2) the optimal prioritization of surgical services, and (3) policies and practices for enhancing surgical scale-up. Table 2 recommends the organization and prioritization of surgical services based on complexity, volume, and acuity of procedures. Organization was categorized as regionalized vs. decentralized and prioritization was categorized as very high, high, or low.

TABLE 2

Table 2. Recommendation matrix for surgical services based on complexity, volume, and acuity.

3.2.1. Recommendations for the optimal distribution of surgical services

The expert panel established consensus that low complexity surgical conditions should be decentralized, or managed by district centers close to communities. High complexity conditions should be regionalized or managed by specialized regional centers. In the case of trauma and emergency surgery, district centers should have the capacity to adequately stabilize patients and facilitate safe transfer of patients to regional centers for complex management.

3.2.2. Recommendations for the optimal prioritization of surgical services

The expert panel established consensus that district hospitals should place the highest priority on developing surgical services for low complexity, high volume conditions. There was general agreement that managing these conditions at district centers would relieve tertiary centers of these demands. Respondents also recommended that district centers place high priority on developing services for low complexity, low volume conditions such as non-trauma orthopedic surgery.

Recommendations were also made for the role of district centers in triaging high complexity conditions. District centers should prioritize establishing systems for screening and referral of high complexity surgical cases to specialized centers. In the case of complex trauma, district centers should place priority on developing capacity to stabilize patients and facilitate transport to regional centers.

3.2.3. Policies and practices for enhancing surgical scale-up

A set of principles for governments and organizations implementing surgical scale-up were developed. At the national level, emergency and essential surgical care should be integrated within national Universal Health Coverage (UHC) frameworks. National referral policies should be established to decrease delays in care, lower the cost of care, and improve outcomes. For surgical societies and university surgical programs in LMICs, in-country outreach is encouraged to reduce the backlog of neglected surgical diseases in underserved areas. Surgical societies or governments should establish registries and databases to better assess disease burden and specific facility performance, forming the backbone of performance assessment and monitoring.

3.3. Summary of recommendations

In summary, the ISG-QSSA proposes the following evidence-based recommendations to guide surgical quality improvement and scale-up in LMICs based on national, regional, district, and facility-level recommendations. On the national level, we propose integration of emergency and essential surgical care into national Universal Health Coverage (UHC) frameworks and the establishment of national referral policies that decrease delays in care, reduce costs, and improve outcomes. On the regional level, we propose the establishment of prehospital trauma systems, training of first responders, and training of trauma providers; regional center development of surgical services for high complexity cases; and the establishment of registries and databases to assess disease burden and facility performance, forming the backbone of quality improvement assessment and monitoring. On the district level, we propose district center development of surgical services for low complexity, high volume cases, procedures for stabilizing patients and facilitating safe and timely transfer in the case of complex trauma and emergency surgery, and the development and implementation of antibiotic prophylaxis guidelines. Lastly on the facility level, we propose the implementation of the WHO Surgical Safety Checklist, establishment of a hand hygiene program, creation of antimicrobial stewardship programs, upgrading of health care facilities to meet standards for providing comprehensive emergency and newborn obstetric care, implementation of maternal health quality improvement programs such as maternal death reviews, and institution of trauma audits.

4. Discussion

The recommendations put forth by this study are designed to provide a flexible framework that can be utilized by members of the global surgical community. They encompass surgery, anesthesia, trauma, and obstetrics recommendations, and utilize prioritization and distribution as a way to suggest both top down and bottom-up change. While all recommendations reflect evidence-based strategies to improve quality of surgical care in LMICs, individual recommendations pertain more specifically to unique tiers of a health care system. The designations of “national”, “region”, “district” and “facility-level” as mentioned above are flexible and can be defined more specifically according to the unique health system in which these recommendations will be applied. Additionally, individual recommendations may be relevant across multiple tiers and can be implemented in a parallel fashion, for example at both facility- and district-levels. Although the WHO highlighted the need to focus on scale-up over 15 years ago, there has been a lag between concept and reality (11, 12, 14). Implementation science education and experience may have been less common in decades past, but current research clearly indicates the need for these established frameworks to be at the center of any study design that aims to achieve sustainable implementation efforts (14, 27–29). With the matrix of recommendations reported in our study, we hope to add stepping stones that enables researchers, policymakers, and healthcare providers to create plans that synergize from each domain.

One of the strengths of this manuscript is the development of a set of building blocks that can help inform changes on both institutional and bureaucratic levels. The acknowledgement that there is no one size fits all technique to developed safe surgical systems is the first step to creating sustainable change. The WHO handbooks on diverse surgical topics have begun to set a foundation for strengthening surgical systems but there is more specificity needed to turn theory into action, which we hope to bridge with studies such as this (2, 10, 22, 29, 30). We aimed to contribute to existing literature by isolating publications from LMICs and following up with current knowledge, attitudes, and perceptions from LMIC experts who have the historical and current landscape of surgical care in their respective countries through a modified Delphi process. We envision the creation of a global implementation strategy for these recommendations that will complement the vast network of existing surgical communities. The G4 Alliance, as an organization, is working towards creating toolkits that will help individual hospitals, as well as national healthcare systems, enhance the already existing infrastructure to be able to practice and deliver healthcare at the highest of their capabilities (20). With the focus on surgery as an essential component of primary care, this solution-driven initiative will ultimately reduce the backlog of surgical neglected diseases by 2030 (27–30).

This study has several limitations including the fact the literature on quality improvement from LMICs is relatively sparse and does not include all the areas of perioperative care. Although the literature represented every region in the world, the geographic representations were skewed as resource limitation for research exists in certain areas of the world more than others. In addition, the members of the expert committee were predominantly from Africa and the situation may not be generalizable to distinct regions with unique baseline needs. Also, none of the panelists were trained specifically in obstetrics, while many do perform routine Cesarean sections. Nevertheless, the best practice recommendations covering maternal and perinatal care should be further vetted by LMIC obstetric experts. Most importantly, the majority of authors on this manuscript are from high-income countries. Although we received global participation during the multi-year process, the organization and drive for this manuscript ultimately stems from stakeholders in high-income countries, which highlights other issues in global surgery and collaborative research which we do not want to perpetuate. The Fiji Ministry of Health was actively involved in seeing the project through and we do aim to acknowledge the valuable time and effort involved for the coordination of the in-person portion of this study.

In this project, we started with a systematic review of the literature with the understanding that it may not be an accurate representation of healthcare systems in LMICs today. Research has shown that even with currently published data, we know very little about the actual patient experience in LMICs and the efficiency and competence of the overall healthcare system (11, 12, 14). Nevertheless, we do believe that having a starting point is essential to create targeted goals and inform new and higher quality research studies. We hope that this manuscript can serve as one tool in addressing the wide range of inequities and deficiencies across continents, not as a punitive measure, but as a call to action for both public and private sectors to work together in creating an achievable roadmap toward making a healthcare system that is for the people it serves. In conclusion, this project fills a critical cap in the rapidly developing field of global surgery: gathering evidence-based, practical, and cost-effective solutions that will serve as a guide for the efficient planning and allocation of resources necessary to promote quality and safe essential surgical services in LMICs.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

Ethical review and approval was not required for this study in accordance with the national legislation and the institutional requirements.

Author contributions

JH: concept design, data analysis, manuscript editing, final approval. L-YW: concept design, data analysis, manuscript writing, final approval. AR: concept design, data analysis, manuscript writing, final approval. JJ: concept design, data analysis, manuscript editing, final approval. MF: concept design, manuscript editing, final approval. CY: concept design, manuscript editing, final approval. AH: concept design, manuscript editing, final approval. All authors contributed to the article and approved the submitted version.

Funding

This work was supported by the G4 Alliance, the University of Chicago, and the International Society of Surgery.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

1. Debas HT, Donkor P, Gawande A, Jamison DT, Kruk ME, Mock CN eds. Essential surgery: Disease control priorities, third edition (volume 1). Washington (DC): The International Bank for Reconstruction and Development/The World Bank (2015). doi: 10.1596/978-1-4648-0346-8

2. Meara JG, Leather AJM, Hagander L, Alkire BC, Alonso N, Ameh EA, et al. Global surgery 2030: evidence and solutions for achieving health, welfare, and economic development. Surgery. (2015) 158(1):3–6. doi: 10.1016/j.surg.2015.04.011

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Mukhopadhyay S, Lin Y, Mwaba P, Kachimba J, Makasa E, Lishimpi K, et al. Implementing world health assembly resolution 68.15: national surgical, obstetric, and anesthesia strategic plan development–the Zambian experience. Bull Am Coll Surg. (2017) 102(6):28–35.28885807

PubMed Abstract | Google Scholar

4. Burssa D, Teshome A, Iverson K, Ahearn O, Ashengo T, Barash D, et al. Safe surgery for all: early lessons from implementing a national government-driven surgical plan in Ethiopia. World J Surg. (2017) 41(12):3038–45. doi: 10.1007/s00268-017-4271-5

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Ng-Kamstra JS, Riesel JN, Arya S, Weston B, Kreutzer T, Meara JG, et al. Surgical non-governmental organizations: global surgery's unknown nonprofit sector. World J Surg. (2016) 40(8):1823–41. doi: 10.1007/s00268-016-3486-1

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Vervoort D, Guetter CR, Munyaneza F, Trager LE, Argaw ST, Abraham PJ, et al. Non-governmental organizations delivering global cardiac surgical care: a quantitative impact assessment. Semin Thorac Cardiovasc Surg. (2022) 34(4):1160–5. doi: 10.1053/j.semtcvs.2021.08.010

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Merson MH. University engagement in global health. N Engl J Med. (2014) 370(18):1676–8. doi: 10.1056/NEJMp1401124

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Kagawa RC, Anglemyer A, Montagu D. The scale of faith based organization participation in health service delivery in developing countries: systematic [corrected] review and meta-analysis. PLoS One. (2012) 7(11):e48457. doi: 10.1371/journal.pone.0048457

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Ayanian JZ, Markel H. Donabedian's lasting framework for health care quality. N Engl J Med. (2016) 375(3):205–7. doi: 10.1056/NEJMp1605101

PubMed Abstract | CrossRef Full Text | Google Scholar

10. National Academies of Sciences, Engineering, and Medicine, Health and Medicine Division, Board on Health Care Services, Board on Global Health, Committee on Improving the Quality of Health Care Globally. Crossing the global quality chasm: improving health care worldwide. Washington (DC): National Academies Press (US) (2018). Available at: http://www.ncbi.nlm.nih.gov/books/NBK535653/ (Accessed April 11, 2022).

11. Kruk ME, Gage AD, Arsenault C, Jordan K, Leslie HH, Roder-DeWan S, et al. High-quality health systems in the sustainable development goals era: time for a revolution. Lancet Glob Health. (2018) 6(11):e1196–252. doi: 10.1016/S2214-109X(18)30386-3

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Institute of Medicine (US) Committee on Quality of Health Care in America. Crossing the quality chasm: a new health system for the 21st century. Washington (DC): National Academies Press (US) (2001).

13. Delisle M, Pradarelli JC, Panda N, Koritsanszky L, Sonnay Y, Lipsitz S, et al. Variation in global uptake of the surgical safety checklist. Br J Surg. (2020) 107(2):e151–60. doi: 10.1002/bjs.11321

PubMed Abstract | CrossRef Full Text | Google Scholar

14. White MC, Ahuja S, Peven K, McLean SR, Hadi D, Okonkwo I, et al. Scaling up of safety and quality improvement interventions in perioperative care: a systematic scoping review of implementation strategies and effectiveness. BMJ Glob Health. (2022) 7(10):e010649. doi: 10.1136/bmjgh-2022-010649

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Brima N, Morhason-Bello IO, Charles V, Davies J, Leather AJ. Improving quality of surgical and anaesthesia care in sub-saharan Africa: a systematic review of hospital-based quality improvement interventions. BMJ Open. (2022) 12(10):e062616. doi: 10.1136/bmjopen-2022-062616

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Iverson KR, Svensson E, Sonderman K, Barthélemy EJ, Citron I, Vaughan KA, et al. Decentralization and regionalization of surgical care: a review of evidence for the optimal distribution of surgical services in low- and middle-income countries. Int J Health Policy Manag. (2019) 8(9):521–37. doi: 10.15171/ijhpm.2019.43

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Henry JA. Decentralization and regionalization of surgical care as a critical scale-up strategy in low- and middle-income countries comment on “decentralization and regionalization of surgical care: a review of evidence for the optimal distribution of surgical services in low- and middle-income countries.”. Int J Health Policy Manag. (2021) 10(4):211–4. doi: 10.34172/ijhpm.2020.26

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Kreindler SA. Conceptualizing the organization of surgical services comment on “decentralization and regionalization of surgical care: a review of evidence for the optimal distribution of surgical services in low- and middle-income countries.”. Int J Health Policy Manag. (2021) 10(4):218–20. doi: 10.34172/ijhpm.2020.60

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Roder-DeWan S. Decentralization and regionalization: redesigning health systems for high quality maternity care comment on “decentralization and regionalization of surgical care: a review of evidence for the optimal distribution of surgical services in low- and middle-income countries”. Int J Health Policy Manag. (2021) 10(4):215–7. doi: 10.34172/ijhpm.2020.30

PubMed Abstract | CrossRef Full Text | Google Scholar

20. The G4 Alliance. Available at: https://www.theg4alliance.org/ (Accessed April 11, 2022).

21. ISS SIC—Home. Available at: http://iss-sic.com/ (Accessed April 11, 2022).

22. WHO Handbook for guideline development. 2nd edn. Geneva, Switzerland: WHO Press (2014). https://www.who.int/publications-detail-redirect/9789241548960 (Accessed April 11, 2022).

23. Goldet G, Howick J. Understanding GRADE: an introduction. J Evid-Based Med. (2013) 6(1):50–4. doi: 10.1111/jebm.12018

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Malmivaara A. Methodological considerations of the GRADE method. Ann Med. (2015) 47(1):1–5. doi: 10.3109/07853890.2014.969766

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Niederberger M, Spranger J. Delphi technique in health sciences: a map. Front Public Health. (2020) 8:457. doi: 10.3389/fpubh.2020.00457

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Spranger J, Homberg A, Sonnberger M, Niederberger M. Reporting guidelines for Delphi techniques in health sciences: a methodological review. Z Evid Fortbild Qual Gesundhwes. (2022) 172:1–11. doi: 10.1016/j.zefq.2022.04.025

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Xu K, Soucat A, Kutzin J, Brindley C, Dale E, Van de Maele N, et al. New perspectives on global health spending for universal health coverage. Geneva: World Health Organization (2017). [report].

28. Valentine N, Darby C, Bonsel GJ. Which aspects of non-clinical quality of care are most important? Results from WHO's General population surveys of “health systems responsiveness” in 41 countries. Soc Sci Med. (2008) 66:1939–50. doi: 10.1016/j.socscimed.2007.12.002

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Marmot M, The Commission on Social Determinants of Health. Achieving health equity: from root causes to fair outcomes. Lancet. (2007) 370:1153–63. doi: 10.1016/S0140-6736(07)61385-3

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Jamison DT, Alwan A, Mock CN, Nugent R, Watkins D, Adeyi O, et al. Universal health coverage and intersectoral action for health: key messages from disease control priorities, 3rd edn. Lancet. (2017) 391:1108–20. doi: 10.1016/S0140-6736(17)32906-9

PubMed Abstract | CrossRef Full Text | Google Scholar