Clinical practice guidelines for the treatment of squamous cell carcinoma of the head and neck: an assessment based on the AGREE II, AGREE-REX tools and the RIGHT checklist

Objective: The aim of this study was to obtain several published clinical practice guidelines (CPGs) for Head and neck squamous cell carcinoma (HNSCC) and critically assess and compare their quality by using three guideline quality evaluation tools, namely, AGREE II, AGREE-REX, and RIGHT, to support the development of future CPGs for HNSCC.

Methods: Clinical practice guidelines related to the diagnosis and treatment of HNSCC were screened through a comprehensive systematic literature search. Data were extracted from the guidelines which met the inclusion criteria, and two experienced head and neck oncology surgeons were trained to act as independent reviewers. The quality of the retrieved guidelines that met the inclusion and exclusion criteria was evaluated by using the AGREE II, AGREE-REX, and RIGHT tools. Then, the quality of the guidelines that met the criteria was assessed. Finally, conclusions and recommendations were drawn based on the scoring results.

Results: A total of eight guidelines met the inclusion criteria. Four guidelines(written by ASCO, NCCN, CCO and KCE) scored > 60% in five or more AGREE II quality domains, two guidelines(written by ASCO and KCE) scored > 60% in all AGREE-REX quality domains, and two guidelines(written by ASCO and KCE) scored > 60% in all quality domains on the RIGHT checklist and were considered “recommendable”.

Conclusions: The authors recommend consulting the American Society of Clinical Oncology guidelines for HNSCC and suggest that future guideline development groups refer to the guideline evaluation framework for guideline writing to enhance the applicability and effectiveness of clinical practice guidelines.

1 Introduction

Squamous cell carcinoma of the head and neck is a malignant tumor that occurs on the mucosal surfaces of the upper respiratory and digestive tracts (paranasal sinuses, nasopharynx, oropharynx, larynx, oral cavity, and nasal cavity). It accounts for approximately 90% of malignant tumors of the oral cavity and is the sixth most common malignant tumor in the world, with an annual incidence of approximately 0.7 million in 2020 and estimated annual mortality of approximately 350,000 (1). Because nasopharyngeal carcinoma is another uniquely characterized disease, with its epidemiology, clinical presentations, pathological manifestations, and treatments different from those of other squamous cell carcinomas of the head and neck, its clinical practice guidelines were excluded from this evaluation.

A clinical practice guideline (CPG) is defined as a guideline or recommendation that is systematically developed to assist clinicians and patients in the appropriate management of a specific clinical situation (2). Given the increasing research and understanding of head and neck squamous cell carcinoma (HNSCC) in recent years, several CPGs for HNSCC have been issued by organizations and committees in different countries, but their reliability has been questioned because of their varying quality (3). Clear and easy-to-understand CPGs are essential for the implementation of medical interventions in clinical practice (4). Therefore, it is essential to assess the quality of relevant guidelines to help improve the prevention of HNSCC and guide the clinical practice of treatment. Although guidelines for HNSCC have been evaluated globally (5, 6), they have all been evaluated at the methodological level based on the AGREE II tool, and no article has yet been published that independently assesses head and neck squamous cell carcinoma clinical practice guidelines using the AGREE-REX and RIGHT tools in terms of reporting quality and clinical applicability of guidelines.

The quality of clinical practice guidelines consists of two main aspects, namely, methodological quality and reporting quality (7, 8), and tools for assessing the quality of guideline reporting should be distinguished from those for assessing their methodology, as they differ in the purpose, structure, and content (9, 10). Among the various tools for evaluating the methodological quality of guidelines, the AGREE II tool has been shown to be valid and has been widely used in different areas of clinical practice. For the quality of guideline reporting, the RIGHT checklist has now been applied in a wide range of applications. It has been argued that these two evaluation tools should be combined to obtain a more comprehensive evaluation of guidelines (11). In this study, a rigorous methodological and reporting quality evaluation of guidelines that were selected by certain criteria was performed by using these two tools to identify areas for improvement in the guidelines. In this context, the main objectives of this study were to i) assess the quality of eligible clinical practice guidelines; ii) analyze the strengths and weaknesses of each guideline; and iii) synthesize the quality of the evidence and the strength of the associated recommendations to arrive at recommended clinical practice guidelines.

2 Materials and methods

2.1 Literature search and guideline selection

The following keywords were used from October 2023 to January 2024: “head and neck squamous cell carcinoma”, “head and neck neoplasms”, “guidelines”, and “clinical practice guidelines”, and their extensions were subjected to a literature search in PubMed, EMBASE, Web of Science, and Google Scholar to identify clinical practice guidelines for HNSCC. The reference sections of the retrieved papers were also examined for additional included articles. In addition, the websites of various scientific societies and international associations, such as the American Society of Clinical Oncology (ASCO), the European Society of Medical Oncology (ESMO), the Guidelines International Network (GIN), and the National Institute for Health and Care Excellence (NICE), were searched. The gray literature was also searched through ProQuest and Turning Research Into Practice. Throughout the search, guidelines were available in full text in English, with no restrictions on the year of publication, but for guidelines published by the same organization or institution, the most recent version that could be accessed at the time of writing was used.

2.2 Inclusion and exclusion criteria

2.2.1 Inclusion criteria

A. Clinical practice guidelines containing “statements” or “guidelines” or providing “recommendations” for HNSCC;

B. Treatment (conceptual) recommendations for patients diagnosed with head and neck squamous cell carcinoma in the oral cavity, oropharynx, and larynx (population) based on the literature and expert opinion;

C. Authored by world-renowned medical association and other bodies;

D. Guidelines with English language only.

2.2.2 Exclusion criteria

A. Clinical practice guidelines (CPGs) without treatment recommendations (for diagnosis, care, referral, etc.);

B. Nonguidelines;

C. Guidelines for squamous cell carcinoma at sites other than the head and neck;

D. Guidelines for other malignant tumors of the head and neck;

E. Other guideline types that are not clinical practice guidelines (e.g., service guidelines);

F. Full text in English is not available;

G. Previous versions of guidelines published by the same organization or association;

H. Guidelines not developed by a widely recognized medical association and other bodies.

2.3 Data screening

In the first stage, the retrieved documents were initially screened by reading the titles and abstracts; in the second stage, the full texts of the documents screened in the first step were read and rechecked to determine whether they met the above inclusion and exclusion criteria. Two independent reviewers (YZ, JL) carried out the above tasks to assess the eligibility of the guidelines. If there was a disagreement between the reviewers during the screening process, a joint discussion was held with a third expert reviewer(SL), and the guidelines were rescreened after a consensus was reached.

2.4 Data extraction

Two reviewers were independently responsible for extracting and documenting the following information from the included literature: basic information: title, journal name, authoring group/organization, year of publication, country or region of origin, purpose of the study, target population, and source of funding; (2) recommendations: strengths and limitations of the evidence, criteria for eligibility of the evidence, search strategy, content of the guideline, and the key recommendations described; and (3) quality control: whether the content is influenced by funding, conflicts of interest among writing team members and the external review process, guideline updating process, etc., and the design of information extraction forms based on AGREE II, AGREE-REX, and RIGHT entries.

2.5 Evaluation tools and scoring criteria

A. AGREE II and AGREE-REX tools: AGREE II was developed by the international Appraisal of Guidelines for Research and Evaluation (AGREE) team as a methodological tool for research and evaluation of guidelines and has been widely used worldwide (12). Six domains, namely, (1) scope and purpose; (2) stakeholder involvement; (3) rigor of development; (4) clarity of presentation; (5) applicability; and (6) editorial independence, comprising 23 items, were evaluated by two independent reviewers. As the AGREE II tool is not sufficient to ensure that guideline recommendations are credible or implementable when applied to the clinic, the 2019 International Guidelines Research Team further developed the AGREE-REX tool. This tool complements rather than replaces AGREE II (13). AGREE-REX aims to ensure guideline credibility, reliability, and implementability in clinical settings (14) and is organized into the following 3 domains: (1) clinical applicability; (2) values and preferences; and (3) implementability, with 9 items under each of the 3 domains (The specifics of the two tools were illustrated in Figures 1, 2). In this study, after evaluating the selected guidelines using the AGREE II tool, the recommended guidelines were evaluated again using the AGREE-REX tool. Each item was rated by two independent reviewers using a 7-point scale (1-strongly disagree to 7-strongly agree). The score for each domain was calculated as follows: (actual score - minimum possible score)/(maximum possible score - minimum possible score) × 100%, according to the guidance in the AGREE tool manual (15), and 60% was used as the domain pass threshold (16–18). In the use of the AGREE II tool, the guidelines were categorized and rated for quality according to the number of domains with a score of ≥ 60% as follows: ≥ 5 domains were considered “high quality”, 3-4 domains were considered “average quality”, and ≤ 2 domains with a score of ≥ 60% were considered “low quality” (19). Similarly, in the use of the AGREE-REX tool, this study employed a calculation of the percentage of “qualified” domains (i.e., mean domain score ≥ 60%) out of the total number of domains. This calculation yielded four possible values: 0%, 33.3%, 66.7%, and 100%. This is due to the fact that the AGREE-REX tool has only three domains. In this study, 100% of domains with an average score of ≥60% are considered to represent the standard for “high quality”, 66.7% is considered to represent “average quality”, and 0% and 33.3% are considered to represent “low quality”.

Figure 1

Figure 1. Description of AGREE II domains and items.

Figure 2

Figure 2. Description of AGREE-REX domains and items.

B. The RIGHT (Reporting Items for practice Guidelines in Healthcare Checklists): High-quality guidelines need to not only meet rigorous methodological standards but also standardize the reporting format (20). The RIGHT checklist, published by the International RIGHT Working Group in 2016, is a tool for evaluating guidelines from the perspective of guideline reporting standardization (9) and is now widely used internationally. The RIGHT checklist is divided into 7 domains and 22 items as follows: (1) basic information; (2) background; (3) evidence; (4) recommendations; (5) review and quality assurance; (6) funding and declaration and management of interests, and (7) other information (The specifics of the RIGHT tools was illustrated in Figure 3). The reporting rate for each guideline item was calculated as follows: items evaluated as “fully reported” were given 2 points, items evaluated as “partially reported” were given 1 point, and items evaluated as “not reported” and “not applicable” were given 0 point. The domain score was calculated as follows: (Interrater score for the domain/highest possible score for the domain) x 100%. This study assessed the quality of the guidelines by calculating the total average score of the average scores across the seven domains. A total average score of ≤ 60% is indicative of “low quality”, 60%-80% is indicative of “average quality”, while > 80% is indicative of “high quality”.

Figure 3

Figure 3. Description of RIGHT domains and items.

Two independent reviewers calculated the scores for each domain using each of the three tools and subsequently tabulated the results using Microsoft Office Excel 2021 software, as illustrated in Tables 1–3. The color red represents low scores, green represents high scores, and amber represents moderate scores. Additionally, the darker the red color, the smaller the value, and the darker the green color, the larger the value. In addition, the reviewers were consulted for their recommendations for this study in the context of the corresponding knowledge of this paper or in the context of the reviews provided.

Table 1

Table 1. AGREE II domain scores for the 8 identified HNSCC CPGs.

Table 2

Table 2. AGREE-REX domain scores for the 8 identified HNSCC CPGs.

Table 3

Table 3. RIGHT domain scores for the 8 identified HNSCC CPGs.

2.6 Quality control

The intraclass correlation coefficient (ICC) is one of the indicators for measuring the reliability of the observer and the reliability of the retest. In this study, we used the SPSS 25.0 software to calculate the ICC for the evaluation results of the two independent reviewers to test the consistency of the evaluations provided by the evaluators. When the ICC is <0.4, the consistency is not good; when the ICC is between 0.4 and 0.59, the consistency is average; when the ICC is between 0.6 and 0.74, the consistency is good; and when the ICC is greater than 0.75, the consistency is high. Prior to the formal start of the evaluation, two reviewers with methodological and professional expertise were systematically trained to ensure consistency in their understanding of the entries. Preevaluations of the guidelines were conducted to calculate the ICCs of the evaluation results. When the ICC was <0.75, the two reviewers discussed the entries with large differences in ratings and reevaluated the entries after reaching a consensus, until the ICC was ≥0.75, at which time the evaluation consistency was considered to be high, and the formal evaluation could begin. After the formal evaluation, the correlation coefficient within the group was recalculated to ensure that the ICC was ≥0.75, and then the data were recorded and analyzed.

3 Results

3.1 Essential features for inclusion in the guidelines

A total of 881 documents were retrieved through the searches of the databases and websites of scientific societies and international associations, and since some guidelines are not be cataloged in traditional databases or published as scientific papers, we manually searched the websites of internationally recognized guideline publishers and obtained seven guidelines. The resulting literature was manually deduplicated to remove a total of 30 duplicates from multiple databases. Initial screening was performed by reading the titles and abstracts of the literature to assess whether the literature could be included in this study, and 845 documents were excluded based on the exclusion criteria. We obtained full-text of 13 guidelines, and eight guidelines that met the screening criteria were ultimately included (21–28). The screening process is shown in Figure 4. The eight CPGs included were jointly authored by multidisciplinary teams and belonged to the clinical practice guidelines proposed for squamous cell carcinoma of the head and neck; each of them provided clear recommendations, and the basic information of the 8 guidelines is shown in Table 4.

Figure 4

Figure 4. Flow diagram of the identification of guidelines.

Table 4

Table 4. Characteristics of included clinical practice guidelines.

3.2 Results of the consistency tests

The intragroup correlation coefficients for the final evaluation results by the two evaluators are shown in Tables 5–7. The intragroup agreement between the two reviewers was good in all 16 domains of the three scales, which suggests that the opinions of the two reviewers were consistent, to a large extent, during the review process.

Table 5

Table 5. Intraclass correlation coefficients for inter-rater reliability across the 6 AGREE II domains.

Table 6

Table 6. Intraclass correlation coefficients for inter-rater reliability across the 3 AGREE-REX domains.

Table 7

Table 7. Intraclass correlation coefficients for inter-rater reliability across the 7 RIGHT domains.

3.3 Quality of methodology

The two independent reviewers calculated the scores for each of the six AGREE II domains after scoring the eight guidelines, and the specific results are shown in Table 1. The eight selected guidelines generally performed better in domain 1, “Scope and purpose”, and domain 4, “Clarity of presentation”, and worse in domain 5, “Applicability”, and domain 3, “Rigor of development”. The highest score was for domain 4, “Clarity of presentation”, at 92.4% ± 11.4%, followed by domain 1, “Scope and purpose”, at 81.3% ± 13.9%. The lowest score was for domain 5, “Application”, at 61.5% ± 25.7%, followed by domain 3, “Rigor of development”, at 70.6% ± 24.5%. The guidelines authored by the ASCO, CCO, NCCN, and KCE scored ≥60% in five or more domains and were considered to be high quality, while the rest of the guidelines were of average quality. At the same time, there was a small difference in the scores for domains 1 and 4 and a large difference in the scores for domains 5 and 6.

From the perspective of a single guideline, the highest scores among the eight guidelines selected were assigned to the ASCO guideline, with an average score of 99.5%. The lowest scoring guideline was developed by the Korean Thyroid-Head and Neck Surgery Working Group, with an average score of 57.3%, with lower scores for domain 5, “Applicability”; domain 6, “Editorial independence”; and domain 2, “Stakeholder involvement”. Lower scores were the main reason for the overall low scores of some guidelines. The lowest variabilities in scores were observed for the guidelines generated by the ASCO and KCE, while the highest variabilities in the NCCN and ESOM scores were related to large differences in their scores for different domains.

3.4 Quality of recommendations

Table 2 summarizes the ratings of the eight guidelines in the three AGREE-REX domains. Domain 1, “Clinical applicability,” had the highest score of 77.4% ± 15.7%, whereas domain 2, “Values and preferences,” had the lowest score of 37.8% ± 29.7%. The AGREE-REX scale scores were generally relatively low compared with those of the AGREE II scale. From a single-guideline perspective, the guidelines developed by the ASCO continued to receive the highest scores, while the guidelines developed by the ESOM received the lowest scores. All of the guidelines produced by the ASCO and KCE had domain scores ≥60%, whereas no guidelines produced by the ESOM and SEOM had domain scores ≥60%.

3.5 Quality of reporting

The scores for each domain of the RIGHT checklist are shown in Table 3, and among the eight selected guidelines, the overall performance was better for domains 1 and 2 and worse for domains 6 and 5. The highest scoring domain was domain 1, “Basic Information,” with 95.3% ± 4.1%, followed by domain 2, “Background,” with 76.6% ± 13.7%; the lowest scoring domain was domain 6, “Funding and declaration and management of interests,” with a score of 43.8% ± 34.5%, followed by domain 5, “Review and quality assurance”, with a score of 57.8% ± 35.3%. At the individual guideline level, the highest average scores were assigned to the guidelines authored by the ASCO, while the lowest scores were assigned to the guidelines authored by the NCCN. Of the eight guidelines, four had a mean score ≤60%.

4 Discussion

4.1 Innovativeness

Clinical practice guidelines have been increasingly used in the clinical care process, influencing patient diagnosis, treatment, care and outcomes; at the same time, there is a high demand from guideline users for trusted health advice (29). Although most guidelines are developed based on expert consensus and systematic literature searches, the lack of uniform standards for their preparation may affect their applicability and dissemination to the extent that it may be difficult to achieve the objectives of the developers. In previous studies, researchers have mostly used a single AGREE II tool to evaluate the methodological quality of guidelines, ignoring their reporting quality and clinical applicability. In this study, we used three tools, AGREE II, AGREE-REX, and RIGHT, to evaluate the methodology, quality of recommendations, and quality of reporting for eight clinical practice guidelines in three dimensions and analyzed the reasons while deriving more standardized guidelines through comparison. The results of this study provide methodological and reporting references, which could effectively improve future guideline development.

4.2 Methodological quality of the CPGs for HNSCC

Of the eight guidelines that were systematically evaluated, those produced by the ASCO, CCO, NCCN, and KCE were rated as “high quality” on the AGREE II scale, suggesting that these four guidelines may be of high methodological quality. The ASCO guidelines performed well in each domain, with no apparent weaknesses, while the CCO and KCE guidelines scored relatively low in domain 5(66.7% and 89.6%, respectively), “Applicability,” indicating that they lacked descriptions of the facilitators and barriers to guideline application, were not comprehensive enough to provide advice and/or tools for applying their recommendations, failed to consider clinical resources that may be needed to apply recommendations, and did not provide high-quality surveillance and monitoring tools. The lack of clinical resources required and the lack of clarity about the criteria for surveillance/audits imply that the guidelines do not adequately account for the different contexts of clinical application, which should clearly articulate how facilitators and impediments influence the guideline development process and the development of recommendations. There is a clear gap between the “independence” of the NCCN guidelines in domain 6 (scored 12.5%) and its performance in other domains, suggesting that the formulation of guideline recommendations fails to reflect a research process that is free of interference from other domains and interests, such as politics and economics. The potential for conflicts of interest and bias among members of guideline development groups has been demonstrated (30, 31), which may affect the impartiality of the guideline content.

Overall, the included guidelines performed well in domain 1, “Scope and purpose”, and domain 4, “Clarity of presentation”. High scores in the “Clarity of presentation” domain indicate that the recommendations communicated by the guidelines to the target group are clear and easy to recognize and understand, while high scores in “Scope and purpose” indicate that the guideline developers have a clear plan and vision for the purpose of the guidelines.

At the same time, these guidelines scored low in domain 5, “Applicability”, and domain 3, “Rigor of development”, for example, The KOREAN guideline scored 22.9% in domain 5 and the EHNS/ESMO/ESTRO guideline scored 32.2% in domain 3. The applicability of any CPG depends on several factors, such as rigorous development, clear presentation, editorial independence, adequate dissemination, and adequate implementation strategies (32). “Applicability” mainly affects the application and dissemination of the guidelines, whose main goal is to guide clinical practice in HNSCC treatment and to ensure clinical efficacy. which requires not only the preparation of well-established guidelines based on rigorous scientific evidence but also their practical use and dissemination by physicians at the clinical level. Increased “applicability” has been shown to be effective in improving adherence to the guideline in clinical practice (33, 34). “Rigor” is the most important and comprehensive area of clinical practice guideline development, encompassing 8 of the 23 items, and how recommendations are formulated based on the evidence will have a direct impact on the clinical applicability of the guideline, a process that largely determines the quality of CPGs (35)(e.g. The EHNS/ESMO/ESTRO guideline scored 32.2% and the SEOM guideline scored 50.0% in domain 3). In this study, we found that the lack of rigor was mainly due to the lack of a clear description of the criteria, strengths, and limitations of evidence selection in some guidelines, as well as to the lack of external review before publication or to the failure to clearly reflect the review process in the guidelines. We observed that only some of the selected guidelines provided complete and detailed descriptions of the search strategy and the grading of the level of evidence through appropriate tools such as GRADE, which ensured the transparency of the evidence search process while avoiding potential bias (36–38). Well-established and rigorous guidelines should clearly articulate the process of recommendation formulation and the sources of evidence (39, 40), as it has been shown (41–43) that an increase in the level of evidence is associated with an increase in the specificity of guideline recommendations and that specific guidelines provide clinicians with more feasible recommendations.

4.3 Quality of recommendations in the CPGs for HNSCC

Recommendations in high-quality guidelines should be evidence based, applicable, implementable, and take into account the values of all stakeholders. Among the included guidelines, the ASCO and KCE guidelines scored high and had no significant areas of weakness, with each guideline scoring more than 60% in each area, which we considered to be a high-quality guideline. Meanwhile, the guidelines prepared by the ESOM and SEOM did not score more than 60% in any area; therefore, we considered these two guidelines to have a low quality of recommendations at the quality of the recommendations level.

In domain 1, “Clinical applicability”, six guidelines received high overall scores, indicating that evidence, target user applicability, and patient/population applicability were adequately considered in the guideline development process, while the two guidelines written by the ESOM and SEOM scored lower. The problems of the ESOM guidelines centered on Item 2, “Target User Applicability”, where recommendations may be less likely to be adhered to due to a lack of adequate consideration of the applicability to the practice setting of the target user in formulating the recommendation, while the guideline written by SEOM scored lower in Item 1, “Evidence”, which is related to a lack of consideration of the practice setting of the target user during the guideline writing process. This is related to the lack of a thorough review of the quality of evidence during the guideline development process, which leads to a lack of evidence transparency and may hinder discussion and communication among guideline developers (44).

In general, domain 2, “Values and preferences”, was poorly represented, with all but two guidelines, namely, the ASCO and KCE guidelines, scoring less than 30% in this domain; in particular, the values and preferences of policy-makers and guideline developers were not adequately described, suggesting that during the guideline development process, guideline developers disregarded the values and preferences of multilevel stakeholders, which may result in guideline recommendations that are biased toward health professionals to the extent that they are not meaningful to patients (45–47). On the other hand, writing guidelines that do not take into account the values of the target populations and policy-makers may result in guideline adherence and outcomes that fall short of expectations, with high-quality studies not being used as they should be and a waste of resources (48, 49).

In domain 3, “Implementability”, most of the guidelines, with the exception of the ASCO and KCE guidelines, performed poorly. We noted that most of the guidelines lacked recommendations that were specific to different settings, such as a region and resource allocation, and that guideline developers did not tailor their recommendations to target populations from different settings, especially resource-poor settings, which may reduce the adherence to the guideline recommendations. For clinical practice guidelines to have an impact on the course of treatment and, ultimately, on outcomes, it is necessary to ensure that they are actually implemented and used by the final clinician in accordance with the guideline recommendations (50), and their “implementability” will be a determining factor in the ultimate adoption of guideline recommendations in the clinic.

The above analysis shows that quality control of recommendations is still a common problem in the development of CPGs for HNSCC, which requires attention of guideline developers.

4.4 Reporting quality of the CPGs for HNSCC

Of the eight guidelines included, only the ASCO and KCE guidelines received high reporting quality scores, with the average reporting rate for each domain being greater than 70%. The reporting rates to each domain of the RIGHT checklist see Figure 5.

Figure 5

Figure 5. Reporting rates to each domain of the RIGHT checklist.

Generally, with the exception of domain 1, “Basic Information”, the quality of reporting in all domains was less than optimal. The main reason for the poor quality of guideline reporting is the lack of awareness of standardization during the development process, which may lead to the loss of clinical guidance (51). In domain 5, “Review and quality assurance”, most of the guidelines provided partial information, i.e., only stated that the guideline had been externally reviewed but did not describe the specific review process, consideration of review comments, results, and quality control procedures, which should be strengthened by reporting on the review process and improving the quality of reporting in this domain. This finding is also consistent with the AGREE II requirements for quality of evidence.

Domain 6, “Funding and declaration and management of interests,” was underreported, particularly in Item 19a, “Describe the types of conflicts of interest relevant to the development of the guidance,” and Item 19b, “Describe methods for evaluating and managing conflicts of interest”. This may be due to the fact that most guidelines simply state that no conflict of interest exists without analyzing the type of conflict and response in detail. This result suggests that the current CPGs for HNSCC lack sufficient standardization and transparency in terms of external review and financial conflicts of interest, which may be related to factors such as space limitations by journals, the lack of awareness among guideline developers about the standardization of guideline reporting, the lack of explicit reporting of the corresponding content at the time of writing, and conflicts of interest that may influence the interpretation and recommendations for the treatment of disease (52, 53). Moreover, the lack of awareness of the guidelines and the shortcomings of guidelines in this area are reflected in other guidelines on different topics (54–56). By analyzing the content of specific entries, we found that “Accreditation and Quality Assurance” and “Declaration and Management of Funding and Conflicts of Interest” were the most important changes to the AGREE II areas of “rigor” and “independence”. The “independence” domain (11), and thus the quality of reporting in the development of normative guidelines, is another layer of complementing the quality of methodology.

4.5 Summary

In the present study, we observed that although most of the guidelines (except the CCO guideline) adopted the GRADE system to grade the quality of evidence, there still were more recommendations based on expert consensus in general, and multicenter randomized controlled trials were not sufficiently represented as a source of recommendations, which may lead to a greater lack of specificity, concreteness, and persuasiveness of recommendations. At the same time, the failure of some guidelines to clearly describe the process of evidence retrieval, selection criteria, and other processes of recommendation formation may lead to some controversy about the rationality and rigor of the process of recommendation formation.

Furthermore, we found that some guidelines that are recognized in the field of HNSCC treatment as important guides for clinical treatment approaches (e.g., the NCCN guideline and the ESMO guideline) did not have a significant advantage in scoring after evaluation with the three guideline evaluation tools. This may be related to the fact that the current guideline evaluation tools do not assess the clinical guidance content of the guidelines themselves and do not judge the validity of the recommendations but only evaluate their methodological and reporting quality. This result indicates, on the one hand, the lack of rigor in the process of evidence retrieval, selection criteria, and the formulation of recommendations for current guidelines with important clinical guidance, which may lead to disagreement among guideline users in the process of clinical application; on the other hand, the scores of the evaluation tool do not represent the true clinical value of the guidelines, to a certain extent, which may lead to the selection of guideline recommendations based on the evaluation results of the evaluation tool. Therefore, in the process of guideline development, clinical experts should fully cooperate with the methodology research group and emphasize the methodological and reporting quality of the guidelines.

Finally, guidelines are also of great importance in guiding the development of health policies, not only by providing clinicians with recommendations to help them base their practice on scientific evidence but also by improving the use of existing health-care resources. Therefore, guidelines should have a standardized development process and be able to provide sufficiently effective recommendations to guide decision-makers in the development and implementation of health policies. It is worth noting that most of guidelines worldwide were developed in high-income countries and regions because the compilation of CPGs requires a large investment of time and human, financial, and other resources that may be difficult for low- and middle-income countries to afford; therefore, low- and middle-income countries may need to consider the applicability of international guidelines and whether it is feasible to implement them in the context of the local economic level and cultural background.

4.6 Limitations

This study has several limitations as follows: (1) the number of guidelines included in this study was small, possibly because of the lack of comprehensiveness in the databases searched and the fact that some guidelines were not included in the databases, which led to the exclusion of some other available CPGs; furthermore, there is a risk of individual bias when manually searching for guidelines in the guideline repositories of associations and organizations; (2) CPGs that could not be published in full text in English were excluded from the study, and the language barrier was another limitation of the lack of comprehensiveness of the guidelines included in this study. (3) the evaluation with the three guideline evaluation tools used qualitative evaluation scales, which may have been influenced by reviewer subjectivity; (4) although scoring the quality of guidelines by counting the number of domains with scores ≥60% is a well-recognized and widely used methodology, it has not yet been formally validated; and (5) our study only assessed the methodology of the guidelines, the quality of the recommendations, and the quality of the reports and did not evaluate their impacts on clinical practice or patient outcomes.

5 Conclusions

The quality of most current clinical practice guidelines on the diagnosis and treatment of squamous cell carcinoma of the head and neck is relatively average, and there is still a need for guideline developers to further improve the quality of their guidelines according to the guideline development specifications, preferably with the involvement of methodology professionals, to enhance the applicability and implementability of the guidelines. According to the results of the comprehensive evaluation using the AGREE II tool, four of the eight group-written guidelines demonstrated high-quality content; according to the results of the AGREE-REX evaluation, only two guidelines met the specifications; according to the results of the RIGHT inventory evaluation, the quality of reporting was good for four guidelines, of which the ASCO guidelines performed excellent in the evaluation across the three tools. Therefore, it is recommended that the squamous cell carcinoma of the head and neck-related guidelines written by the ASCO be used for reference by health care professionals and patients. In conclusion, the methodological quality, recommendations and reporting quality of the current CPGs on head and neck squamous cell carcinoma still need to be improved, and in the process of guideline development in the future, it is recommended that guideline developers consider basing their work on the three evaluation tools as a framework to improve the quality of the current guidelines on head and neck squamous cell carcinoma to better guide clinical practice. Ultimately, the objective of guideline development is to provide more effective guidance for clinical practice.

Data availability statement

The original contributions presented in the study are included in the article/supplementary material. Further inquiries can be directed to the corresponding authors.

Author contributions

YZZ: Data curation, Investigation, Software, Writing – original draft, Writing – review & editing. JL: Data curation, Investigation, Writing – review & editing. SL: Data curation, Investigation, Writing – review & editing. YZ: Investigation, Writing – review & editing. XS: Investigation, Writing – review & editing. SH: Conceptualization, Project administration, Writing – review & editing. WD: Conceptualization, Project administration, Writing – review & editing.

Funding

The author(s) declare that financial support was received for the research, authorship, and/or publication of this article. This work was supported by the Department of Oral Maxillofacial-Head and Neck Surgery, School and Hospital of Stomatology, China Medical University.

Acknowledgments

We thank all authors of this study for their hard work.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

1. Romano A, Di Stasio D, Petruzzi M, Fiori F, Lajolo C, Santarelli A, et al. Noninvasive imaging methods to improve the diagnosis of oral carcinoma and its precursors: state of the art and proposal of a three-step diagnostic process. Cancers (Basel). (2021) 13:1–22. doi: 10.3390/cancers13122864

PubMed Abstract | Crossref Full Text | Google Scholar

2. G. Institute of Medicine Committee. Standards for developing trustworthy clinical practice. In: Graham R, Mancher M, Wolman DM, Greenfield S, Steinberg E, editors. Clinical practice guidelines we can trust. National Academies Press (US) Copyright 2011 by the National Academy of Sciences, Washington (DC (2011).

Google Scholar

3. Wang C, Luo X, Li M, Cui L, Li X, Han L, et al. Reporting quality of clinical practice guidelines regarding gout and hyperuricemia according to the RIGHT checklist: systematic review. Systematic Rev. (2021) 10:99. doi: 10.1186/s13643-021-01645-1

PubMed Abstract | Crossref Full Text | Google Scholar

4. Selva A, Sanabria AJ, Pequeño S, Zhang Y, Solà I, Pardo-Hernandez H, et al. Incorporating patients’ views in guideline development: a systematic review of guidance documents. J Clin Epidemiol. (2017) 88:102–12. doi: 10.1016/j.jclinepi.2017.05.018

PubMed Abstract | Crossref Full Text | Google Scholar

5. Kohler HF, Mehanna H, Shah JP, Sanabria A, Fagan J, Kuriakose MA, et al. Comparison of different guidelines for oral cancer. Eur Arch Otorhinolaryngol. (2021) 278:2961–73. doi: 10.1007/s00405-020-06423-8

PubMed Abstract | Crossref Full Text | Google Scholar

6. Madera Anaya MV, Franco JV, Merchan-Galvis AM, Gallardo CR, Bonfill Cosp X. Quality assessment of clinical practice guidelines on treatments for oral cancer. Cancer Treat Rev. (2018) 65:47–53. doi: 10.1016/j.ctrv.2018.03.001

PubMed Abstract | Crossref Full Text | Google Scholar

7. Cai M, Cotter RR, Wong SL, Mayo SW. Evaluating clinical practice guidelines for the management of rectal cancer: Did they get it RIGHT? J Surg Oncol. (2024) 129:1475–80. doi: 10.1002/jso.v129.8

PubMed Abstract | Crossref Full Text | Google Scholar

8. Vlayen J, Aertgeerts B, Hannes K, Sermeus W, Ramaekers D. A systematic review of appraisal tools for clinical practice guidelines: multiple similarities and one common deficit. Int J Qual Health care: J Int Soc Qual Health Care. (2005) 17:235–42. doi: 10.1093/intqhc/mzi027

PubMed Abstract | Crossref Full Text | Google Scholar

9. Chen Y, Yang K, Marušic A, Qaseem A, Meerpohl JJ, Flottorp S, et al. A reporting tool for practice guidelines in health care: the RIGHT statement. Ann Internal Med. (2017) 166:128–32. doi: 10.7326/M16-1565

PubMed Abstract | Crossref Full Text | Google Scholar

10. Huwiler-Müntener K, Jüni P, Junker C, Egger M. Quality of reporting of randomized trials as a measure of methodologic quality. Jama. (2002) 287:2801–4. doi: 10.1001/jama.287.21.2801

PubMed Abstract | Crossref Full Text | Google Scholar

11. Yao X, Ma J, Wang Q, Kanters D, Ali MU, Florez ID. A comparison of AGREE and RIGHT: which clinical practice guideline reporting checklist should be followed by guideline developers? J Gen Internal Med. (2020) 35:894–8. doi: 10.1007/s11606-019-05508-3

PubMed Abstract | Crossref Full Text | Google Scholar

12. Makarski J, Brouwers MC. The AGREE Enterprise: a decade of advancing clinical practice guidelines. Implementation science: IS. (2014) 9:103. doi: 10.1186/s13012-014-0103-2

PubMed Abstract | Crossref Full Text | Google Scholar

13. Brouwers MC, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G, et al. advancing guideline development, reporting, and evaluation in health care. Prev Med. (2010) 51:421–4. doi: 10.1016/j.ypmed.2010.08.005

PubMed Abstract | Crossref Full Text | Google Scholar

14. Florez ID, Brouwers MC, Kerkvliet K, Spithoff K, Alonso-Coello P, Burgers J, et al. Assessment of the quality of recommendations from 161 clinical practice guidelines using the Appraisal of Guidelines for Research and Evaluation-Recommendations Excellence (AGREE-REX) instrument shows there is room for improvement. Implementation science: IS. (2020) 15:79. doi: 10.1186/s13012-020-01036-5

PubMed Abstract | Crossref Full Text | Google Scholar

15. Brouwers MC, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G, et al. advancing guideline development, reporting and evaluation in health care. CMAJ: Can Med Assoc J = J l’Association medicale Can. (2010) 182:E839–42. doi: 10.1503/cmaj.090449

PubMed Abstract | Crossref Full Text | Google Scholar

16. Shin M, Prasad A, Arguelles GR, Wakim JJ, Chorath K, Moreira A, et al. Appraisal of clinical practice guidelines for the evaluation and management of neck masses in children. J paediatrics Child Health. (2021) 57:803–9. doi: 10.1111/jpc.15516

PubMed Abstract | Crossref Full Text | Google Scholar

17. Romeo V, Stanzione A, Cocozza S, Ugga L, Cuocolo R, Brunetti A, et al. A critical appraisal of the quality of head and neck cancer imaging guidelines using the AGREE II tool: A EuroAIM initiative. Cancer Med. (2019) 8:209–15. doi: 10.1002/cam4.2019.8.issue-1

PubMed Abstract | Crossref Full Text | Google Scholar

18. Chorath K, Garza L, Tarriela A, Luu N, Rajasekaran K, Moreira A. Clinical practice guidelines on newborn hearing screening: A systematic quality appraisal using the AGREE II instrument. Int J Pediatr otorhinolaryngology. (2021) 141:110504. doi: 10.1016/j.ijporl.2020.110504

PubMed Abstract | Crossref Full Text | Google Scholar

19. Hoffmann-Eßer W, Siering U, Neugebauer EA, Brockhaus AC, Lampert U, Eikermann M. Guideline appraisal with AGREE II: Systematic review of the current evidence on how users handle the 2 overall assessments. PloS One. (2017) 12:e0174831. doi: 10.1371/journal.pone.0174831

PubMed Abstract | Crossref Full Text | Google Scholar

20. Yang Y, Lu J, Ma Y, Xi C, Kang J, Zhang Q, et al. Evaluation of the reporting quality of clinical practice guidelines on lung cancer using the RIGHT checklist. Trans Lung Cancer Res. (2021) 10:2588–602. doi: 10.21037/tlcr-21-405

PubMed Abstract | Crossref Full Text | Google Scholar

21. Machiels JP, Rene Leemans C, Golusinski W, Grau C, Licitra L, Gregoire V, et al. Ehns Executive Board, Reprint of “Squamous cell carcinoma of the oral cavity, larynx, oropharynx and hypopharynx: EHNS-ESMO-ESTRO Clinical Practice Guidelines for diagnosis, treatment and follow-up. Oral Oncol. (2021) 113:105042. doi: 10.1016/j.oraloncology.2020.105042

PubMed Abstract | Crossref Full Text | Google Scholar

22. Mesia R, Iglesias L, Lambea J, Martinez-Trufero J, Soria A, Taberna M, et al. SEOM clinical guidelines for the treatment of head and neck cancer (2020). Clin Transl Oncol. (2021) 23:913–21. doi: 10.1007/s12094-020-02533-1

PubMed Abstract | Crossref Full Text | Google Scholar

23. Pfister. DG, Haddad. RI, Shah. JP, Spencer. S, Hicks. WL, Sher. D, et al. NCCN clinical practice guidelines in oncology (NCCN guidelines^®) head and neck cancers. (2024) NCCN Guidelines. Available online at: www.nccn.org/

Google Scholar

24. Homma A, Ando M, Hanai N, Harada H, Honma Y, Kanda T, et al. Summary of Japanese clinical practice guidelines for head and neck cancer - 2022 update edited by the Japan society for head and neck cancer. Auris Nasus Larynx. (2024) 51:174–88. doi: 10.1016/j.anl.2023.07.003

PubMed Abstract | Crossref Full Text | Google Scholar

25. Winquist E AC, Meyers B, Yoo J, Chan K. Systemic therapy in the curative treatment of head and neck squamous cell cancer. In: Winquist E, Arinze C, editors. Cancer care ontario; 2016 august 10 [Endorsed 2022 january]. Program in evidence-based care guideline no.: 5-11 version 2 ENDORSED. Toronto (ON: Belgian Health Care Knowledge Centre (KCE)) (2016).

Google Scholar

26. Grégoire V LR, Heus P, van de Wetering F, Hooft L, Scholten R, Verleye L, et al. ORAL CAVITY CANCER: DIAGNOSIS, TREATMENT AND FOLLOW-UP. Good Clinical Practice (GCP) Brussels: Belgian Health Care Knowledge Centre (KCE) (2014). Available online at: https://www.cancercareontario.ca/en/guidelines-advice/types-of-cancer/31711

Google Scholar

27. Joo YH, Cho JK, Koo BS, Kwon M, Kwon SK, Kwon SY, et al. Guidelines for the surgical management of oral cancer: korean society of thyroid-head and neck surgery. Clin Exp Otorhinolaryngol. (2019) 12:107–44. doi: 10.21053/ceo.2018.01816

PubMed Abstract | Crossref Full Text | Google Scholar

28. Koyfman SA, Ismaila N, Crook D, D’Cruz A, Rodriguez CP, Sher DJ, et al. Management of the neck in squamous cell carcinoma of the oral cavity and oropharynx: ASCO clinical practice guideline. J Clin oncology: Off J Am Soc Clin Oncol. (2019) 37:1753–74. doi: 10.1200/JCO.18.01921

PubMed Abstract | Crossref Full Text | Google Scholar

29. Elwyn G, Quinlan C, Mulley A, Agoritsas T, Vandvik PO, Guyatt G. Trustworthy guidelines - excellent; customized care tools - even better. BMC Med. (2015) 13:199. doi: 10.1186/s12916-015-0436-y

PubMed Abstract | Crossref Full Text | Google Scholar

30. Neuman J, Korenstein D, Ross JS, Keyhani S. Prevalence of financial conflicts of interest among panel members producing clinical practice guidelines in Canada and United States: cross sectional study. BMJ. (2011) 343:d5621. doi: 10.1136/bmj.d5621

PubMed Abstract | Crossref Full Text | Google Scholar

31. The L. Managing conflicts of interests in clinical guidelines. Lancet (London England). (2019) 394:710. doi: 10.1016/S0140-6736(19)32000-8

PubMed Abstract | Crossref Full Text | Google Scholar

32. Grimshaw J, Freemantle N, Wallace S, Russell I, Hurwitz B, Watt I, et al. Developing and implementing clinical practice guidelines. Qual Health care: QHC. (1995) 4:55–64. doi: 10.1136/qshc.4.1.55

PubMed Abstract | Crossref Full Text | Google Scholar

33. Lumba-Brown A, Prager EM, Harmon N, McCrea MA, Bell MJ, Ghajar J, et al. A review of implementation concepts and strategies surrounding traumatic brain injury clinical care guidelines. J neurotrauma. (2021) 38:3195–203. doi: 10.1089/neu.2021.0067

PubMed Abstract | Crossref Full Text | Google Scholar

34. Yeates KO, Barlow KM, Wright B, Tang K, Barrett O, Berdusco E, et al. Health care impact of implementing a clinical pathway for acute care of pediatric concussion: a stepped wedge, cluster randomised trial. Cjem. (2023) 25:627–36. doi: 10.1007/s43678-023-00530-1

PubMed Abstract | Crossref Full Text | Google Scholar

35. Hoffmann-Eßer W, Siering U, Neugebauer EAM, Brockhaus AC, McGauran N, Eikermann M. Guideline appraisal with AGREE II: online survey of the potential influence of AGREE II items on overall assessment of guideline quality and recommendation for use. BMC Health Serv Res. (2018) 18:143. doi: 10.1186/s12913-018-2954-8

PubMed Abstract | Crossref Full Text | Google Scholar

36. Manchikanti L. Evidence-based medicine, systematic reviews, and guidelines in interventional pain management, part I: introduction and general considerations. Pain physician. (2008) 11:161–86. doi: 10.36076/ppj

Crossref Full Text | Google Scholar

37. Guyatt G, Akl EA, Oxman A, Wilson K, Puhan MA, Wilt T, et al. Synthesis, grading, and presentation of evidence in guidelines: article 7 in Integrating and coordinating efforts in COPD guideline development. An official ATS/ERS workshop report. Proc Am Thorac Soc. (2012) 9:256–61. doi: 10.1513/pats.201208-060ST

PubMed Abstract | Crossref Full Text | Google Scholar

38. Goldet G, Howick J. Understanding GRADE: an introduction. J Evidence-Based Med. (2013) 6:50–4. doi: 10.1111/jebm.2013.6.issue-1

PubMed Abstract | Crossref Full Text | Google Scholar

39. Neumann I, Alonso-Coello P, Vandvik PO, Agoritsas T, Mas G, Akl EA, et al. Do clinicians want recommendations? A multicenter study comparing evidence summaries with and without GRADE recommendations. J Clin Epidemiol. (2018) 99:33–40. doi: 10.1016/j.jclinepi.2018.02.026

PubMed Abstract | Crossref Full Text | Google Scholar

40. Brandt L, Vandvik PO, Alonso-Coello P, Akl EA, Thornton J, Rigau D, et al. Multilayered and digitally structured presentation formats of trustworthy recommendations: a combined survey and randomised trial. BMJ Open. (2017) 7:e011569. doi: 10.1136/bmjopen-2016-011569

PubMed Abstract | Crossref Full Text | Google Scholar

41. Thomas ET, Thomas ST, Perera R, Gill PJ, Moloney S, Heneghan C. The quality of diagnostic guidelines for children in primary care: A meta-epidemiological study. J paediatrics Child Health. (2023) 59:1053–60. doi: 10.1111/jpc.16454

PubMed Abstract | Crossref Full Text | Google Scholar

42. Tunnicliffe DJ, Singh-Grewal D, Kim S, Craig JC, Tong A. Diagnosis, monitoring, and treatment of systemic lupus erythematosus: A systematic review of clinical practice guidelines. Arthritis Care Res. (2015) 67:1440–52. doi: 10.1002/acr.v67.10

PubMed Abstract | Crossref Full Text | Google Scholar

43. Collins KK, Smith CF, Ford T, Roberts N, Nicholson BD, Oke JL. Adequacy of clinical guideline recommendations for patients with low-risk cancer managed with monitoring: systematic review. J Clin Epidemiol. (2024) 169:111280. doi: 10.1016/j.jclinepi.2024.111280

PubMed Abstract | Crossref Full Text | Google Scholar

44. Shen WQ, Yao L, Wang XQ, Hu Y, Bian ZX. Quality assessment of cancer cachexia clinical practice guidelines. Cancer Treat Rev. (2018) 70:9–15. doi: 10.1016/j.ctrv.2018.07.008

PubMed Abstract | Crossref Full Text | Google Scholar

45. Frank L, Basch E, Selby JV. The PCORI perspective on patient-centered outcomes research. Jama. (2014) 312:1513–4. doi: 10.1001/jama.2014.11100

PubMed Abstract | Crossref Full Text | Google Scholar

46. Armstrong MJ, Bloom JA. Patient involvement in guidelines is poor five years after institute of medicine standards: review of guideline methodologies. Res involvement engagement. (2017) 3:19. doi: 10.1186/s40900-017-0070-2

PubMed Abstract | Crossref Full Text | Google Scholar

47. Chong CA, Chen IJ, Naglie G, Krahn MD. How well do guidelines incorporate evidence on patient preferences? J Gen Internal Med. (2009) 24:977–82. doi: 10.1007/s11606-009-0987-8

PubMed Abstract | Crossref Full Text | Google Scholar

48. Tricco AC, Zarin W, Rios P, Nincic V, Khan PA, Ghassemi M, et al. Engaging policy-makers, health system managers, and policy analysts in the knowledge synthesis process: a scoping review. Implementation science: IS. (2018) 13:31. doi: 10.1186/s13012-018-0717-x

PubMed Abstract | Crossref Full Text | Google Scholar

49. Langlois EV, Becerril Montekio V, Young T, Song K, Alcalde-Rabanal J, Tran N. Enhancing evidence informed policymaking in complex health systems: lessons from multi-site collaborative approaches. Health Res Policy Syst. (2016) 14:20. doi: 10.1186/s12961-016-0089-0

PubMed Abstract | Crossref Full Text | Google Scholar

50. Sarkies MN, Jones LK, Gidding SS, Watts GF. Improving clinical practice guidelines with implementation science. Nature reviews. Cardiology. (2022) 19:3–4. doi: 10.1038/s41569-021-00645-x

PubMed Abstract | Crossref Full Text | Google Scholar

51. Wang Q, Duan Y, Liang J, Chen Z, Chen J, Zheng Y, et al. Reporting quality of 2014-2018 clinical practice guidelines on diabetes according to the RIGHT checklist. Endocrine. (2019) 65:531–41. doi: 10.1007/s12020-019-02005-9

PubMed Abstract | Crossref Full Text | Google Scholar

52. Wang AT, McCoy CP, Murad MH, Montori VM. Association between industry affiliation and position on cardiovascular risk with rosiglitazone: cross sectional systematic review. BMJ. (2010) 340:c1344. doi: 10.1136/bmj.c1344

PubMed Abstract | Crossref Full Text | Google Scholar

53. Moynihan RN, Cooke GP, Doust JA, Bero L, Hill S, Glasziou PP. Expanding disease definitions in guidelines and expert panel ties to industry: a cross-sectional study of common conditions in the United States. PloS Med. (2013) 10:e1001500. doi: 10.1371/journal.pmed.1001500

PubMed Abstract | Crossref Full Text | Google Scholar

54. Xiao Y, Jiang L, Tong Y, Luo X, He J, Liu L, et al. Evaluation of the quality of guidelines for assisted reproductive technology using the RIGHT checklist: A cross-sectional study. Eur J obstetrics gynecology Reprod Biol. (2019) 241:42–8. doi: 10.1016/j.ejogrb.2019.07.039

PubMed Abstract | Crossref Full Text | Google Scholar

55. Cheng C, Wu X, Song W, Li D, Hao L, Li X, et al. A reporting quality evaluation of the clinical practice guidelines for bladder cancer based on the RIGHT checklist. Trans andrology Urol. (2022) 11:1586–97. doi: 10.21037/tau-22-712

PubMed Abstract | Crossref Full Text | Google Scholar

56. Zhao Y, Li Y, Li J, Song W, Zhao J, Xu Y, et al. Reporting quality of chronic kidney disease practice guidelines according to the RIGHT statement: a systematic analysis. Ther Adv chronic Dis. (2020) 11:1–10. doi: 10.1177/2040622320922017

PubMed Abstract | Crossref Full Text | Google Scholar