How to improve adherence of guidelines for localized testicular cancer surveillance: A Delphi consensus study

Stage-I testicular germ-cell tumor (TGCT) has excellent cure rates. Surveillance is fully included in patient’s management, particularly during the first years of follow-up. Surveillance guidelines differ between the academic societies, mainly concerning imaging frequency and long-term follow-up. We evaluated surveillance practice and schedules followed by French specialists and set up a DELPHI method to obtain a consensual surveillance program with an optimal schedule for patients with localized TGCT. First, an online survey on surveillance practice of stage-I TGCT based on clinical-cases was conducted among urologists, radiation-oncologists and medical-oncologists. These results were compared to ESMO/EAU and AFU guidelines. Then a panel of experts assessed surveillance proposals following a Delphi-CM. Statements were drafted after analysis of the previous survey and systematic literature review, with 2 successive rounds to reach a consensus. The study was conducted between July 2018 and May 2019. Concerning the first step: 61 participated to the survey (69% medical-oncologists, 15% urologists, 16% radiation-oncologists). About 65% of practitioners followed clinico-biological guidelines concerning 1 to 5 years of follow-up, but only 25% stopped surveillance after the 5th-year. No physician followed the EAU/ESMO guidelines of de-escalation chest imaging. Concerning the second step: 32 experts (78% medical-oncologists, 16% urologists, 6% radiation-oncologists) participated to the Delphi-CM. Thanks to Delphi-CM, a consensus was reached for 26 of the 38 statements. Experts agreed on clinico-biological surveillance modalities and end of surveillance after the 5th-year of follow-up. For seminoma, abdominal ultrasound was proposed as an option to the abdominopelvic (AP) scan for the 4th-year of follow-up. No consensus was reached regarding de-escalation of chest imaging. To conclude, the survey proved that French TGCT-specialists do not follow current guidelines. With Delphi-CM, a consensus was obtained for frequency of clinico-biological surveillance, discontinuation of surveillance after the 5th-year, stop of AP scan on the 4th-year of follow-up for seminoma. Questions remains concerning type and frequency of chest imaging.

1 Introduction

Testicular germ cell tumor (TGCT), including seminoma and non-seminoma, is a good prognosis cancer. It mainly affects young men with a peak of incidence around 30 years (1). Localized stage [stage IA - IB as defined in AJCC 2017 (2)] is more frequent and represent almost 70% of cases (3). After surgery, management of localized TGCT will be either surveillance alone, or adjuvant chemotherapy followed by surveillance, depending on histology, patient’s comorbidities and risk factors for recurrence.

The different French and European academic societies [European Association of Urology (EAU), French Association of Urology (AFU), European Society for Medical Oncology (ESMO)] have published recommendations concerning follow-up schemes for TGCT. Recent guidelines, suggest a de-escalation in surveillance, particularly for radiological imaging and for the surveillance rhythm after the first 2 years of follow-up (4–7). Follow-up schemes are based on expert’s opinions because of lack of clinical trial data. Guidelines differ between academic societies, as EAU and AFU recommendations for imaging surveillance (Supplementary Table 1). Given these differences, it induces more complexity for the practitioner to choose the follow-up scheme to apply.

Initially, we conducted this study to evaluate adherence to guidelines and to study which follow-up schemes were applied by French practitioners. Then, we applied a Delphi-consensus method (Delphi-CM) in order to obtain a consensus concerning surveillance of localized TGCT among French practitioners involved in French Urology Association (AFU) and Genito-urinary group (GETUG).

2 Material and methods

2.1 Step 1: Survey

An electronic survey was developed by the steering committee to ascertain current clinical follow-up practices. Participants were identified through the AFU and GETUG email lists consisting of physicians caring for TGCT patients. The survey was sent by email to 500 members. It was open from July to October 2018; data were collected through an online survey system.

The survey contained 23 questions divided into two sections: Part I related to demographic data of the responders and Part II asked responders to evaluate patient follow-up in four different cases. Two cases involved surveillance for seminoma with or without adjuvant chemotherapy, and two cases involved surveillance for non-seminoma in the same situations (Supplementary Data 1). Questionnaires were closed multiple-choice-questions, based on national (AFU) and international (EAU, ESMO) guidelines. Participants had to answer at least one question of the Part II to be included. Responses were quantitative variables. We calculated medians of each response and compared them to AFU and EAU/ESMO’s guidelines. Survey responses consistent with guidelines were considered as adherence of the recommendation. Conversely, in case of disagreement, it was considered as non-adherence. All non-adherence statements were discussed with the Delphi-CM in the step 2.

2.2 Step 2: Delphi consensus method

2.2.1 Steering committee

The steering committee was composed of five oncologists (FJ, KF, SC, AF, AD), two urologist (AM, TM), one radiation-oncologists (DP) and a biostatistician (FP). The following steps were carried out by the steering committee including: (1) selection of expert panel; (2) generation of clinical statements based on the controversial results from the survey and on the current medical literature; (3) definition of the consensus levels and agreement according to Delphi-CM; (4) interpretation of the results; (5) final wording in the consensus document.

2.2.2 Expert panel

The expert panel involved in the rating process were composed of experts in uro-oncology who were solicited within the AFU’s and the GETUG’s group. They received questionnaires by email. The nominative lists of experts who accepted to participate are provided in the acknowledgments section. Responses were collected from March to June 2019.

2.2.3 Generation of statements

Based on the survey results, only follow-up practices that differed from AFU and EAU/ESMO guidelines were discussed. After literature review, 38 statements belonging to 4 major clinical situations (seminoma and non-seminoma in surveillance only or after adjuvant chemotherapy, respectively) were written by the steering committee.

2.2.4 Rating and analysis of the questionnaire

For the first round: experts were asked to indicate on a scale ranging from one (totally disagree) to nine (totally agree) how the statement was relevant. For each rating lower than 7, participants were invited to justify their votes, in order to understand the reason of disagreement and to adjust the statement. A statement was defined as appropriate (approved its acceptance) if the median of all scores was ≥ 7 and there was agreement among all experts of the rating committee (range of rating 7–9 for strong consensus and 5–9 for relative consensus); or inappropriate (did not satisfied to consider) if the median of all scores was ≤ 3; or uncertain if the median of all scores was comprised between 4 and 6. After the first round, statements with strong consensus were accepted and those inappropriate were rejected. Others items were reassessed by the steering committee. For the second round: questionnaire and results of the first round were sent to all expert of the rating group. Each expert was asked to rate the questionnaire with the same scale. The same rating methodology as for the first round was applied, however appropriate statement with relative consensus were accepted.

2.3 Ethics approval

This study consisted of a survey of expert opinions and no patient data were collected, so no ethical approval was required to perform this study.

3 Results

3.1 Step 1: Survey

Sixty-one participants completed the survey resulting in a 12.2% response rate. Characteristics concerning the demographic data are presented in Table 1. Majority of them are in charge of more than 5 patients with TGCT per year. According to clinical situations, we observed a heterogeneity of follow-up during the first five years of follow-up with 30 to 50% adherence to AFU’s guidelines, 20 to 36% adherence to EAU/ESMO’s guidelines and 6 to 45% of practices not corresponding to any guidelines. Tables 2, 3 report the current practice of respondents compared to guidelines. After the 5th year of surveillance, only 21 to 34% of respondents declared stopping follow-up as recommended by guidelines. More than 50% of respondents affirmed to continue follow-up until the 10th year of surveillance (Table 4).

TABLE 1

Table 1 Baseline characteristics of survey respondents and Delphi consensus method experts.

TABLE 2

Table 2 Results of survey for seminoma comparing to academic societies guidelines.

TABLE 3

Table 3 Results of survey for non-seminoma comparing to academic societies guidelines.

TABLE 4

Table 4 Current practice of respondents after the 5^th year of follow-up.

3.2 Step 2: Delphi consensus method

3.2.1 Expert panel

Thirty-two experts participated to the Delphi-CM. Characteristics of the expert panel are reported on Table 1.

3.2.2 First round

The first questionnaire consisted of 38 ranking items. After round 1, a strong consensus was obtained on 1 item (2.6%), relative consensus for 1 item (2.6%), 36 items were judged uncertain (94.8%) and none were judged inappropriate (Supplementary Table 2). Hence, a total of 37 items were proposed for new rating (Figure 1).

FIGURE 1

Figure 1 Flow chart.

3.2.3 Second round

After the second round of rating, 4 items were judged appropriate and 33 items were uncertain. According to the Delphi-CM, we cancelled two extreme values of vote for each uncertain item. Hence, 20 additional items were accepted: 1 with strong consensus and 19 with relative consensus (Table 5).

TABLE 5

Table 5 Results of the rating process following the second round of Delphi consensus method.

3.2.4 Item with consensus

In total, strong consensus was obtained for 2 items (5.2%) concerning clinical and biological examination for seminoma. Relative consensus was reached for 24 items (63.2%) (Table 5). Eleven items concerned clinical and biological examination for seminoma (n=3) and non-seminoma (n=8) and 4 relating to discontinuation of surveillance after the 5th-year. Proposals on decrease of imaging have been accepted for seminoma [stop of abdomino-pelvic (AP) scan on 4th-year for seminoma (n=2)] and non-seminoma [no systematic imaging on the 4th-year after adjuvant chemotherapy (n=1)]. Concerning chest imaging, for non-seminoma, experts agreed to realize exams two times per year on the first two years (n=2), with CT-scan rather than chest X-ray in case of surveillance only (n=1) and to continue chest-imaging one time per year from the 3rd-year (n=1). Concerning AP-scan, for non-seminoma, it could be done 1-2 times (1st-year) and one time (2nd-year), in the absence of lymphovascular invasion (LVI) (n=1). If LVI, AP-scan would be done 2 times a year on the first two years (n=1).

3.2.5 Item lacking consensus

Twelve items (31.6%) lacked consensus (Table 5). Five items concerning frequency of clinical and biological exams. Physicians disagreed with discontinuing chest imaging surveillance in seminomas (n=2). They did not accept to decrease chest imaging surveillance in non-seminoma (n=3), to reduce AP-scan on the 2nd-year of follow-up for LVI negative non-seminoma (n=1) and to stop doing scanner on the 4th-year of follow-up for non-seminoma on surveillance alone (n=1).

4 Discussion

Our study highlights the low-adherence of French practitioners to academic societies’ guidelines explaining a heterogeneity of practices. These differences mainly concerned frequencies of physical/biological examination and imaging, particularly methods for chest surveillance. In our study we described less than 50% of adherence to guidelines and up to 45% of practices did not corresponding to any of the guidelines. In a second time, from these conflicting situations between practices and recommendations, we performed a Delphi-CM to obtain a consensus of follow-up for TGCT. After two rounds, we reached consensus for 68.4% of statements.

Our results stress that physician adherence is critical to successful application of the recommendations. Non–guideline adherence for TGCT patients is common, most frequently in the form of inappropriate imaging (8). There are many barriers to guideline adherence including lack of awareness, lack of familiarity and lack of agreement (9, 10).

Non–guideline adherence for TGCT patients was associated with inferior global quality-of-life (11). As TGCT survival improves, quality of care such as reducing treatment and monitoring exams toxicity is one of the main objectives of the future years (12).

Due to the lack of strong scientific data, some items issued from the international guidelines have been elaborated issued from experts’ opinions, particularly for the imaging surveillance. In our survey, on the whole, the clinico-biological surveillance was in accordance to guidelines. The main disagreements concerned imaging exams. In retrospective study of TGCT patients on surveillance alone, adherence to imaging was shown to be inconsistent with guidelines (13). Thanks to the Delphi-CM, experts agreed on several proposals on imaging, as stop of imaging on the 4th-year. Experts did not agree on the decrease in chest imaging as suggest by last guidelines, whether for stopping the chest CT-scan in the follow-up of seminoma or performing a chest X-ray in non-seminoma. This lack of consensus explains the lack of adhesion on the guidelines as experts do not seem to be agree with them. This discrepancy can be explained by the limited evidence regarding pulmonary recurrence. In retrospective studies, it was showed that all relapses were detected by abdominopelvic imaging or markers elevation (14, 15). Thoracic recurrence was always associated with another abnormality (15). Moreover, survival of TGCT patients is good with a 5-year relapse-free survival of around 90% (16). These results were recently confirmed by the analysis of the SWENOTECA cohort. Tandstad et al. reported for clinical stage I TGCT 7.9% of relapse of which 1.9% occurred within the first 5 years. Only 1.5% of relapse occurred beyond the first 5 years (17).

Although there is a scientific rationale for decrease in chest imaging, it seems that practitioners are not ready to implement it in their practices. Performing repeated imaging in these young patients can have clinical impact such as an increase of the risk of radiation secondary cancers and kidney failure induced by contrast products (18–21). One of the options suggested by experts in our study, to limit radiation, was to perform a low-dose thoracic scanner for thoracic surveillance. Although the dose delivered during a low-dose scanner is lower than conventional scanner, it is still 70 to 250 times higher than dose received during a chest X-ray (22), but it could be an acceptable option.

Regarding the cessation of follow-up at 5 years, the majority of survey participants reported a continuation to 10 years. Majority of them reported that they pursued surveillance beyond 5 years because all recommendations were not clear on it. After implementation of the Delphi-CM, the experts agreed on a cessation at 5 years, in accordance with literature and EAU/ESMO guidelines (5, 23, 24).

To our knowledge this is the first report of the use of the Delphi-CM for the application in routine of different national and international recommendations for TGCT follow-up adapted to the opinion of TGCT practitioners.

Nevertheless, this study has some limitations. As purposive sampling was used (and participants were, therefore, not randomly selected), representativeness cannot be assured. However, using AFU’s and GETUG’s mailing list, we targeted different French practitioners who follow TGCT patients. Moreover, the whole premise behind the Delphi theory is that the panel members are in fact experts in their field, therefore, yielding results of increased accuracy, instead of selecting a representative sample of the population. Only 12.2% of participants responded to the survey. This could have affected the potential for guidelines’ adherence as well as the amount of data analyzed. However this participation rate corresponds to that usually found using an online questionnaire (25). With 32 experts, the size of our panels for the Delphi-CM is in accordance with the published recommendations relative to the required size of a panel used for being representative (26, 27).

5 Conclusion

French and European guidelines concerning the follow-up of the TGCT patients are different. Thanks to the Delphi-CM, an expert consensus was obtained for frequency of clinico-biological surveillance, stop of AP scan on the 4th year of follow-up for seminoma and discontinuation of surveillance after the 5th year. The continuation of this project is to spread these results in order to inform as many practitioners as possible to increase the adherence to the guidelines.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Author contributions

FJ, AF and ADS conceived of the presented idea. ADS and FP verified the analytical methods. ADS performed the calculations. EC and FJ helped supervise the project. ADS, EC and FJ wrote the paper with input from all authors. All authors contributed to the article and approved the submitted version.

Funding

The study was supported by Centre François Baclesse (Caen Normandie, France).

Acknowledgments

The authors are grateful to the panelists for their participation to the study: Damien Pouessel, Constance Thibault, Jérôme Rigaud, François Audenet, Jean-Pierre Lotz, Pierre Clavere, Antoine Angelergues, Nadine Houede, Catherine Becht, Carolina Saldana, Cédric Lebacle, Olivier Huillard, Diego Tosi, Camille Serrate, Yohann Loriot, Géraldine Pignot, Johann Barkatz, Giulia Baciarello, Ali Hasbini, Christine Chevreau, Emmanuelle Bompas, Mouna Ayadi, Sylvestre Le Moullec, Antoine Thiery-Vuillemin, Jonathan Olivier, Franck Priou, Philippe Barthelemy, Sylvain Ladoire, Elouen Boughalem, Lionnel Geoffrois, Mathieu Laramas, Eric Voog, Loic Mourey, Friederike Schlürmann.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fonc.2022.1036190/full#supplementary-material

Abbreviations

AFU, French Association of Urology; EAU, European Association of Urology; ESMO, European Society for Medical Oncology; TGCT, testicular germ cell tumor.

References

1. Binder-Foucard F, Belot A, Delafosse P, Remontet L, Woronoff A-S BN. Estimation nationale de l’incidence et de la mortalité par cancer en France entre 1980 et 2012. In: Partie 1 – tumeurs solides. Saint-Maurice (Fra: Inst Veill Sanit (2013). p. 122.

Google Scholar

2. Amin MB, Greene FL, Edge SB, Compton CC, Gershenwald JE, Brookland RK, et al. The Eighth Edition AJCC Cancer Staging Manual: Continuing to build a bridge from a population-based to a more "personalized" approach to cancer staging. CA Cancer J Clin (2017) 67(2):93–9. doi: 10.3322/caac.21388. Epub 2017 Jan 17

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Miki T, Kamoi K, Fujimoto H, Kanayama HO, Ohyama C, Suzuki K, et al. Clinical characteristics and oncological outcomes of testicular cancer patients registered in 2005 and 2008: The first large-scale study from the cancer registration committee of the japanese urological association. Int J Urol (2014) 21:4–9. doi: 10.1111/iju.12441

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Schmoll HJ, Jordan K, Huddart R, Pes ML, Horwich A, Fizazi K, et al. Testicular seminoma: ESMO clinical practice guidelines for diagnosis, treatment and follow-up. Ann Oncol (2010) 21:v140–6. doi: 10.1093/annonc/mdq176

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Honecker F, Aparicio J, Berney D, Beyer J, Bokemeyer C, Cathomas R, et al. ESMO consensus conference on testicular germ cell cancer : Diagnosis, treatment and follow-up. Ann Oncol (2018), 29(8):1658–86. doi: 10.1093/annonc/mdy217

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Durand X, Fléchon A, Murez T, Rocher L, Camparo P, Morel-Journel N, et al. Recommandations en onco-urologie 2016-2018 du CCAFU : Tumeurs germinales testiculaires. Progrès en Urol (2016) 27:S147–65. doi: 10.1016/S1166-7087(16)30706-0

CrossRef Full Text | Google Scholar

7. Albers P, Albrecht W, Algaba F, Bokemeyer C, Cohn-Cedermark G, Fizazi K, et al. Guidelines on testicular cancer: 2015 update. Eur Urol (2015) 68:1054–68. doi: 10.1016/j.eururo.2015.07.044

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Wymer KM, Pearce SM, Harris KT, Pierorazio PM, Eggener SE. Adherence to national comprehensive cancer network guidelines for testicular cancer. J Urol (2016), 197(3 Pt 1):684–9. doi: 10.1016/j.juro.2016.09.073

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Cabana M, Rand C, Powe N, Wu A, Wilson M, Abboud P, et al. Why don’t physicians follow clinical practice guidelines? a framework for improvement. JAMA (1999) 282:1458–65. doi: 10.1001/jama.282.15.1458

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Keikes L, Van Oijen M, Lemmens V, Koopman M, Punt C. Evaluation of guideline adherence in colorectal cancer treatment in the Netherlands: A survey among medical oncologists by the Dutch colorectal cancer group. Clin Color Cancer (2018) 17:58–64. doi: 10.1016/j.clcc.2017.10.007

CrossRef Full Text | Google Scholar

11. Saoud R, Andolfi C, Aizen J, Wymer K, Burg M, Barashi N, et al. Impact of non-guideline-directed care on quality of life in testicular cancer survivors. Eur Urol Focus (2021) 7:1137–42. doi: 10.1016/j.euf.2020.10.005

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Huang M, Cheaib J, Su Z, Biles M, Sharma R, Zhang A, et al. Assessing quality of care in the diagnosis and treatment of early-stage testicular cancer: A critical review and summary. Urol Oncol (2021) 39:400–8. doi: 10.1016/j.urolonc.2021.02.001

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Gyawali B, Griffiths R, Robinson A, McInnes M, Bedard P, Booth C. Use of imaging for active surveillance in testicular cancer: Is real-world practice concordant with guidelines? Can Urol Assoc J (2022) 16:26–33. doi: 10.5489/cuaj.7246

PubMed Abstract | CrossRef Full Text | Google Scholar

14. De La Pena H, Sharma A, Glicksman C, Joseph J, Subesinghe M, Traill Z, et al. No longer any role for routine follow-up chest x-rays in men with stage I germ cell cancer. Eur J Cancer (2017) 84:354–9. doi: 10.1016/j.ejca.2017.07.005

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Harvey ML, Geldart TR, Duell R, Mead GM, Tung K. Routine computerised tomographic scans of the thorax in surveillance of stage I testicular non-seminomatous germ-cell cancer - a necessary risk? Ann Oncol (2002) 13:237–42. doi: 10.1093/annonc/mdf032

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Brandt M, Ruf C, Dieckmann K, Syring I, Ruckes C, Nestler T, et al. Clinical characteristics, treatment patterns and relapse in patients with clinical stage IS testicular cancer. World J Urol (2022) 40:327–34. doi: 10.1007/s00345-021-03889-x

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Tandstad T, Hellesnes R, Haugnes H, Karlsdottir A, Langberg C, Negaard H, et al. Late relapses in testicular cancer: Results from a national cohort. J Clin Oncol (2022) 40. doi: 10.1200/JCO.2022.40.16_suppl.5008

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Brenner DJ, Hall EJ. Computed tomography — an increasing source of radiation exposure. N Engl J Med (2007) 357:2277–84. doi: 10.1056/NEJMra072149

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Baysson H, Journy N, Roué T, Ducou-lepointe H, Etard C, Bernier M. Exposition à la scanographie dans l’enfance et risque de cancer à long terme. une synthèse des études épidémiologiques récentes. Bull Cancer (2016) 103:190–8. doi: 10.1016/j.bulcan.2015.11.003

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Berrington de Gonzalez A, MM MS, Kim K-P, Bhargavan M, Lewis R, Mettler F, et al. Projected cancer risks from computed tomographic scans performed in the united states in 2007. Arch Intern Med (2009) 169:2071–7. doi: 10.1001/archinternmed.2009.440

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Nash K, Hafeez A, Hou S. Hospital-acquired renal insufficiency. Am J Kidney Dis (2002) 39:930–6. doi: 10.1053/ajkd.2002.32766

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Ferretti G, Jankowski A. Doses d’exposition des examens radiologiques thoraciques. Rev Mal Respir (2010) 27:644–50. doi: 10.1016/j.rmr.2010.03.024

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Mortensen MS, Lauritsen J, Kier MGG, Bandak M, Appelt AL, Agerbæk M, et al. Late relapses in stage I testicular cancer patients on surveillance. Eur Urol (2016) 70:365–71. doi: 10.1016/j.eururo.2016.03.016

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Tandstad T, Ståhl O, Håkansson U, Dahl O, Haugnes HS, Klepp OH, et al. One course of adjuvant BEP in clinical stage I nonseminoma mature and expanded results from the SWENOTECA group. Ann Oncol (2014) 25:2167–72. doi: 10.1093/annonc/mdu375

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Sebo P, Maisonneuve H, Cerutti B, Fournier J, Senn N, Haller D. Rates, delays, and completeness of general practitioners’ responses to a postal versus web-based survey: A randomized trial. J Med Internet Res (2017) 19:83. doi: 10.2196/jmir.6308

CrossRef Full Text | Google Scholar

26. Haute Autorité de Santé. Élaboration de recommandations de bonne pratique _ méthode « Recommandations par consensus formalisé ». (2010).

Google Scholar

27. Fitch K, Bernstein SJ, Aguilar MD, Burnand B, LaCalle JR. (2001) The RAND/UCLA appropriateness method user’s manual (No. RAND/MR-1269-DG-XII/RE). Santa Monica CA: Rand Corp.

Google Scholar