Patient decision aids for patients with differentiated thyroid carcinoma: development process and alpha and beta testing

Background: Patient decision aids (PtDAs) are structured clinical tools that facilitate shared decision-making. Two important treatment decisions for patients with differentiated thyroid cancer (DTC), which could benefit from PtDAs, are as follows (1): the extent of surgery decision in patients with low-risk DTC and (2) the decision to start or delay starting the treatment with tyrosine kinase inhibitors (TKIs) in patients with advanced tumors.

Material and methods: PtDAs for these two decisions were developed using the International Patient Decision Aids Standards (IPDAS) quality criteria in an iterative process of prototype development via alpha and beta testing by patients and physicians. The information content of the PtDAs was based on the available literature, current guidelines, and patient’s needs, preferences, and values.

Results: The web-based PtDAs underwent two rounds of alpha testing, revisions, and beta testing. The PtDAs have the same structure, consisting of six steps: a general introduction, information about the treatment options, comparing the treatment options, knowledge questions, a values clarification exercise, and saving the information. The alpha testing (n = 8 patients, n = 10 physicians) showed that the PtDAs were highly acceptable and usable for decision-making. Results of the beta testing in 20 patients showed that two patients did not use the PtDA; the other 18 patients found that the PtDAs were readable (n = 17) and helpful (n = 14) for decision-making. All patients recommend using the PtDAs.

Conclusions: Evidence-based PtDAs were created for patients with DTC for two different treatment decisions. Our final version was judged to be clear, balanced, and helpful in decision-making.

Introduction

Differentiated thyroid cancer (DTC) is rapidly increasing in incidence throughout the world, mostly as a result of the increased use of diagnostic imaging and surveillance (1–3). Primary treatment for most patients with DTC consists of surgical removal of the thyroid (total thyroidectomy), followed by treatment with radioactive iodide (I¹³¹, RAI) to ablate the remaining thyroid remnants or destroy (microscopic) DTC remnants. This is followed by life-long thyroid hormone therapy (4, 5). In patients with low-risk tumors smaller than 1 cm, the removal of only the affected lobe (lobectomy) is currently the standard of care. These patients do not require RAI treatment, and most of them maintain normal thyroid function. Several studies, including a recent meta-analysis (6), suggest that this more conservative approach results in similar long-term outcomes and therefore could be applied also in selected patients with low-risk tumors larger than 1 cm. However, because there are no prospective randomized controlled clinical trials (RCTs), comparing different surgical approaches for patients with low-risk DTC larger than 1 cm, management recommendations are currently based on retrospective data (7). After primary treatment, the majority of patients with DTC have an excellent long-term prognosis (1, 8–11).

Nonetheless, after primary treatment, up to 30% of the DTC patients develop recurrent disease and/or distant metastases. These patients have a less favorable prognosis amounting to an average life expectancy of 3–5 years for those with tumors that are nonresponsive to RAI (12, 13). For patients with metastatic disease, both local and systemic treatments are available, but overall complete remission is only seen in one-third of cases (14) and, while these treatments can improve DTC-related symptoms and disease-free survival, there is no evidence that these treatments result in a clear improvement of the overall survival (OS).

Therefore, for both patients with low-risk DTC and with RAI- refractory advanced DTC, there is a discussion about the optimal treatment strategy that better balances the risks and benefits for the individual patients and their personal preferences. As such, some patients with low-risk DTC might currently undergo overtreatment, which could negatively affect their quality of life (QOL) (6).

The current guidelines of the American Thyroid Association (ATA) state that less aggressive therapy, for example, a thyroid lobectomy and a tailored follow-up, can be equally acceptable and explicitly mention room for patients’ preferences (15). Similarly, for asymptomatic or mildly symptomatic RAI- refractory DTC patients, an important unanswered question regards the optimal timing of starting tyrosine kinase inhibitors (TKIs). For those who are asymptomatic or only mildly symptomatic, starting the treatment too early may expose them to side effects and worsen their QOL without evidence of a survival benefit (16). The recent European Thyroid Association (ETA) guidelines state that the decision to start TKIs should include patient-related medical factors and patients’ preferences with respect to treatment goals and patient values, and acceptance of adverse effects (17).

Given the different benefits and harms of surgery and TKI treatment, the care for patients with DTC should be better personalized. Patients could be informed using plain-language, evidence-based information about these decisions. Patient decision aids (PtDAs) are suitable instruments to support decision-making. Studies have shown that PtDAs generally improve patient knowledge, result in more realistic patient treatment expectations, increase active patient participation in decision-making, and reduce indecisiveness (4, 18). Currently, no PtDAs for DTC patients for the above-mentioned treatment decisions are available. This study presents the development process and alpha and beta testing of different PtDAs in order to provide decision support for two treatment decisions in patients with DTC (1): the extent of surgery in patients with low-risk DTC larger than 1 cm, and (2) the decision to wait or start with TKIs in patients with asymptomatic or mildly symptomatic advanced, RAI-refractory DTC.

Materials and methods

Development process

The development of the PtDAs was part of the Communication Booster (COMBO) study (NCT03905369), which aimed to develop, evaluate, and implement decision-support tools for DTC patients. Thirteen hospitals (six academic and seven non-academic) in the Netherlands participated, as well as the Dutch patient association “Schildklier Organisatie Nederland (SON).” In the study, we randomized patients into an intervention or control group. Patients in both groups received general information about DTC from their physicians. In addition, the intervention group received also the PtDA. The Medical Ethical Committee (CMO) of the region Arnhem–Nijmegen approved the study protocol (MEC-2018-4521).

The International Patient Decision Aids Standards (IPDAS) were used to guide the development of the PtDAs and were based on behavioral and decision-making theories underlying the Ottawa Decision Support Framework (19–21). To guide the development process, recommendations by Coulter et al. (21) and the Dutch Guideline were consulted (22). The development process is shown in Figure 1 and was performed by a project group consisting of a PhD student who is also a medical doctor (AK), a decision-making scientist (PS), an endocrinologist (RN-M), an implementation scientist (RH), and an oncologist (PO), assisted by a patient and physician expert panel. None of these patients or physicians had any conflicts of interest.

FIGURE 1

Figure 1 Development process of the patient’s decision aids.

Scope and purpose of PtDAs

The scope and purpose of the PtDAs were defined by the project group. Meetings were held with the project group to reach a consensus on the scope, purpose, and target audience. The main scope was to improve informed choice regarding two different treatment decisions in patients with DTC. Therefore, the project group agreed to develop a PtDA for each treatment decision (Figure 2). The target audience for the first PtDA was defined as patients with low-risk DTC (> 1 cm) according to the ATA criteria (15), considering the extent of thyroid surgery. For the second PtDA, the target group consisted of patients with advanced RAI- refractory asymptomatic or mildly symptomatic DTC considering whether to wait or start with TKI treatment.

FIGURE 2

Figure 2 Overview of the different decision aids in the Netherlands.

Content and format

The first part of the COMBO study consisted of a literature review for clinical parameters, resulting in a meta-analysis of the extent of surgery decision (6), followed by focus groups with patients with DTC and physicians (endocrinologists, surgeons, oncologists, nuclear medicine physicians) treating patients with DTC to explore patients’ needs, preferences, and values regarding the two treatment decisions, which resulted in a focus group paper (23). Based on the identified needs, guidelines, literature review, and the expertise in the project group, domains were determined that were important for decision-making in this setting: (1) risks and (2) benefits of both treatment decisions; (3) oncological outcomes in both treatment decisions; and (4) patients’ personal values in decision-making. For the first three domains, we collected available literature (6, 24–26); for the fourth domain, values mentioned by patients and physicians were extracted from the focus group interviews (23). The development was an iterative process (Figure 1), with the content and format informed by the IPDAS guidelines. An online, web-based format was chosen by the project group to provide a PtDA that is tailored to a treatment decision. This resulted in the prototype draft.

Methods of alpha & beta testing and revision

The first drafts of the two PtDAs (“thyroid lobectomy or total thyroidectomy” and “to wait or start with TKIs”) were finished after an iterative process of revising and reviewing the prototype content with the project group. In order to ensure the PtDAs were accessible to a broad audience, these drafts were evaluated on readability by communication experts experienced in increasing readability (“Stichting Makkelijk Lezen”) (27). The reading level aimed to enable 95% of the Dutch population to understand the text, level A2-B1, according to the Common European Framework of for Languages (28). This resulted in the second draft (29) (Figure 1).

The second draft was sent by email to two different panels, consisting of eight patients and nine physicians (Figure 1). The eight patients made a decision regarding DTC in the past (four with low-risk DTC and four with advanced disease). The nine physicians consisted of two endocrinologists, two surgeons, and five oncologists treating patients with DTC. Patients and physicians were asked to evaluate this draft of the PtDA on clear language, content, layout, and comprehensibility. They were recruited from participating Dutch academic hospitals. Feedback on the PtDAs was also given by two patient advocates recruited from the patient association SON (Figure 1). This resulted in a third draft with a web-based format.

This third web-based draft was developed and alpha-tested by patients who made a decision regarding DTC in the past (Figure 1). The patients were recruited by endocrinologists from the participating Dutch academic hospitals, who asked one or two of their patients to participate in a face-to-face interview about the PtDA. After verbal consent, patients were approached by the investigator (AK) by telephone. Eight patients (four with low-risk DTC and four with advanced disease) were willing to participate (Table 1). All interviews were conducted by AK, and further interviews were determined by data saturation. The first part of the interview was unstructured using a think-aloud method (30). The second part was semi-structured, and participants were asked to fill out a questionnaire on content, structure, length, readability, balance, comprehensibility, relevance, reliability, completeness, and usability (Figure 1; Table 2). This resulted in the fourth draft.

TABLE 1

Table 1 Demographic characteristics of patients and physicians who participated in the alpha testing.

TABLE 2

Table 2 Alpha testing among DTC patients and physicians treating patients with DTC.

As suggested by Coulter (21), physicians were invited to participate in alpha testing as well. Therefore, the fourth draft was sent to an expert panel of 10 physicians (two endocrinologists, six surgeons, and two oncologists) from academic and nonacademic Dutch hospitals, all specializing in treating patients with DTC. None of them were involved in the initial development of the PtDA. They were asked to assess the PtDA for usability, acceptability of the content and format, practicality of use in the clinical pathways, and their perceived efficacy. They were further asked whether they would be willing to hand out this PtDA to their patients and at what moment in time. Their suggestions were incorporated to create the final fifth version (Figure 1), which was deemed acceptable for clinical use. The number of interviews was, again, determined by data saturation (Figure 1; Table 1). In addition to the quantitative evaluation, the quality of the fourth draft was tested against the IPDAS criteria.

Beta testing

The beta testing, or “ real-world testing, “ was organized during the ongoing randomized controlled COMBO trial (RCT, NCT03905369). The first 20 participants (Figure 1; Table 3) in the intervention group who received the PtDA and who were not involved in the design phase were asked to evaluate the feasibility of the PtDA with a questionnaire containing questions on usefulness, length, amount of information, comprehensibility, and reliability (Table 4). The quality of the PtDAs was tested using the IPDAS criteria.

TABLE 3

Table 3 Demographic characteristics of patients who participated in the beta testing.

TABLE 4

Table 4 Beta testing during the RCT among DTC patients.

Results

Development process

Content and format

Both PtDAs were divided into a general introduction and six steps: general information about DTC; treatment options; comparison of the treatment options; important items; values clarification exercise; and saving the information. The general introduction indicated for whom the PtDA was applicable and contained an explanation of how to use the PtDA. The first step gave general information about DTC. In step 2, the treatment options with the main risks, benefits, and oncological outcomes are presented. In step 3, patients could compare the treatment options. In step 4, patients were asked to answer factual knowledge questions to check for comprehension. Step 5 contained a values clarification exercise with five statements. These were based on patients’ values extracted from the focus group interviews (Figure 3) (23). Patients could indicate the importance of each statement on a four-point Likert importance scale ranging from not important to very important. The next page contained two empty boxes. Box 1 asked patients to tell what matters in their lives in general, and box 2 asked for their concerns regarding treatment. They were also asked to indicate which option they preferred and the strength of this preference. In the last step, patients were stimulated to save their answers using email or printing options and to bring these answers along and discuss them with their physicians in the next consultation.

FIGURE 3

Figure 3 Example of the values clarification exercise of the extent of surgery PtDA (translated from Dutch into English).

Patients and physicians’ expert panel

The second draft of the extent of surgery PtDA was revised by four patients and three physicians. One patient and all three physicians suggested developing two separate PtDAs for “thyroid lobectomy or total thyroidectomy” and “no additional surgery or totalizing thyroidectomy” because of the different treatment options (Figure 2). Percentages of complications and a note on life-long adjustment difficulties of substitution with thyroid hormones were added.

The second draft of the TKI PtDA was revised by four patients and five physicians. One patient and all five physicians suggested developing two separate PtDAs, one for sorafenib and one for lenvatinib, the two TKIs that are currently approved for use and reimbursed in the Netherlands, because of different outcome percentages and adverse events (Figure 2). More detailed information about the effect of TKIs, adverse events, and information on the palliative character of TKIs were added, as were progression-free survival rates. Both patients and physicians indicated that the general information about DTC was not appropriate for patients with metastatic disease because these patients were already familiar with thyroid cancer, so this information was deleted. In the end, four separate PtDAs for the extent of the surgery decision and the TKI decision were developed (Figure 2). Based on this information, the third draft was developed in the form of web-based PtDAs.

Results of alpha & beta testing and revision

Alpha testing 1: patients

The third draft was alpha-tested by four patients treated with surgery and four with advanced disease. Their baseline characteristics are presented in Table 1. All patients were satisfied with the content, format, and layout. Length and the amount of information were assessed “just right” by all participants. The information was predominantly judged as balanced and comprehensible, and participants preferred 100-person diagrams for risk communication (31). All participants found the PtDAs useful for decision-making if they had to choose between the treatment options. Participants also suggested some minor changes. To improve the relevance of the extent of surgery, they suggested including more general information about low-risk DTC. To increase the usability of the TKI PtDAs, it was suggested to clarify the navigation through the PtDAs. Lastly, to increase readability, one participant suggested changing the colors of the 100-person diagrams. Other results from the questionnaire are shown in Table 2.

Alpha testing 2: physicians

The fourth draft was evaluated by 10 physicians (two endocrinologists, six surgeons, and two oncologists) from academic and nonacademic hospitals, all of whom specialized in treating patients with DTC. Their baseline characteristics are presented in Table 1. Overall, they were satisfied with the content, format, and layout of the PtDAs. Length and the amount of information were assessed “just right” by all physicians.

For the extent of surgery PtDAs, physicians suggested adjusting the surgery time for thyroid lobectomy as compared to total thyroidectomy because the duration of surgery is not the same for both procedures. There was a discussion about the presented recurrence rates (RR). For the extent of surgery PtDA, physicians suggested reducing the RR for total thyroidectomy from 7% to 4%, based on the following literature regarding the addition of RAI: The mean percentage of RR after the addition of RAI is 4% (32–35). A recent review by Verburg et al. (26) showed that literature published in the last decade offers data that support adjuvant postoperative RAI in DTC patients. Recently, Leboulleux et al. (36) showed in an RCT that the RAI ablation did not result in a significant oncological benefit in patients with low-risk tumors smaller than 2 cm and therefore could possibly be omitted in these cases. The latter represents only a part of the patients targeted by this PtDA, and omitting the RAI ablation for low-risk patients who underwent total thyroidectomy is currently not routinely applied in the Netherlands. All the abovementioned suggestions were admitted in the PtDAs. For the TKI PtDAs, there were no specific suggestions.

Other results from the questionnaire are shown in Table 2. All physicians mentioned that they would recommend the use of the PtDAs. Figure 3 shows the values clarification exercise of the final version of the “thyroid lobectomy or total thyroidectomy” PtDA.

Beta testing

The final version was beta- tested during the RCT of the COMBO study. All participants in the intervention group were asked to fill out a questionnaire about the feasibility of the PtDA. A total of 20 participants were already included in this intervention group. Their baseline characteristics are presented in Table 3. Two participants did not use the PtDA. The other 18 participants were satisfied with the content, format, and layout of the PtDAs (Table 4). Length and the amount of information were assessed “just right” by 89% of participants; the median grade was 8 out of 10, and all participants recommended using the PtDA. Almost all suggested handing over the PtDA as early as possible after the diagnosis, ahead of the decision appointment at home, and also to include more details about their current daily life. They also suggested including the option of active surveillance.

IPDAS criteria

The IPDAS collaboration checklist was used to estimate the quality of the PtDAs (20). Of the 64 items on the checklist, 55 quality criteria were applicable to our PtDAs given the scope of the PtDAs. The final version of the PtDAs met 52 out of the 55 applicable IPDAS criteria (95%). Among the 23 criteria for “content,” 21 criteria were met. The two unmet criteria were on listing the option of doing nothing and viewing personalized probabilities based on their own situation. Among the 26 criteria in the “ development process,” 25 were met. The one unmet criterion was the provision of alternative methods to understand the information, such as audio or video options. Lastly, after the beta test, the six criteria for “effectiveness” were all met.

Discussion

This article describes the systematic development and pilot testing of web-based PtDAs for patients with low-risk (> 1 cm) DTC regarding the extent of primary surgery and for patients with advanced disease regarding starting or delaying the start of TKI treatment. We performed alpha testing with patients and physicians and beta testing with patients. To make the PtDA accessible to every eligible patient, it was written in the A2-B1 language level according to the Common European Framework of for Languages (28). The PtDAs were considered clear, balanced, and helpful for decision-making. The amount of information, length, presentation, and clarity of information received positive feedback. None of the participants indicated that the content was confusing. The criteria for evidence-based PtDA development that have been established by the IPDAS were followed (20). Our PtDAs met 52 of 55 quality criteria for content, development process, and effectiveness as formulated in the IPDAS checklist. Patients acting as reviewers who made a treatment decision in the past indicated they would have preferred to use the present PtDAs, if they had been available at the time of decision-making.

Most, but not all, of the IPDAS criteria were met. For example, listing the option of active surveillance was recommended in the IPDAS criteria. However, we did not include active surveillance in the PtDA for the extent of the surgery decision. At this moment, in national and international guidelines, active surveillance for low-risk DTC patients is not mentioned as a primary treatment option. In recent years, there has been emerging evidence on the safety of active surveillance as an option for the management of micro papillary thyroid carcinomas (mPTCs < 1 cm). More research is necessary before including this option in the PtDA for patients with low-risk tumors larger than 1 cm. When available, these data can be incorporated into a future version of the PtDA. Regarding the criteria to provide information on “viewing probabilities based on their own situation,” we assessed that based on the current knowledge, we could not provide additional information that might better individualize the prediction of the outcomes. For example, although we hypothesize that in general elderly patients have a higher risk of complications or adverse events, length of hospital stay, and mortality (37), we could not find whether and to what extent the risks are higher specifically in elderly DTC patients. A third criterion we did not meet was “the provision of alternative methods to understand the information, such as audio or video options.” Patient participants in our focus group interviews did not prefer audio and/or video options to understand information.

To the best of our knowledge, these are the first documented PtDAs aiming to support DTC patients in these two treatment decisions. Regarding PtDAs in low-risk DTC patients (1–4 cm), there is only one published PtDA focusing on RAI therapy in patients with low-risk papillary thyroid cancer (PTC) (4). This PtDA is limited to the decision to follow or omit RAI treatment after a total thyroidectomy. On the other hand, Brito et al. (38) and Pitt et al. (39, 40) recently developed a treatment choice tool (paper cards) for patients with DTC. These tools included the option of active surveillance, implying that these tools are also useful for informed patients with mPTC. Both tools need further testing before being implemented on a broad scale.

Our PtDAs were designed to facilitate conversations about treatment options for DTC patients in two different treatment decisions. In general, PtDAs have been shown to improve patient knowledge of the health care decision, decrease decisional conflict, and facilitate shared decision-making (18). Patients who use a PtDA are more often satisfied with the choice than those who receive standard counseling (18). However, PtDAs promote conversations between physicians and patients and do not replace the need for a patient–physician consultation. Treatment options still need to be explained to patients to help individualize the trade between harms and benefits according to the patient’s specific situation and clinical situation. PtDAs facilitate a preference-based decision in which patient values and preferences are incorporated (41). By clarifying patients’ values, PtDAs encourage the treatment option that best fits the patient. Therefore, a values clarification exercise was added asking patients to select arguments for and against a treatment option. Feldman-Stewart et al. showed in an RCT in patients with early-stage prostate cancer that values clarification exercises led to better preparation for decision-making and to less regret at the > 1-year follow-up (42).

Strengths and limitations

A strength of our study is the structured development process of the PtDAs, which systematically uses the input of patients, physicians, and patient advocates. Involving patients in all stages of development yielded important insights. Furthermore, our statements in the values clarification exercise were based on patients’ values extracted from the focus group interviews (23). Another strength is that we developed four very specific PtDAs for each of the decision steps to support decision-making for the full decision trajectory of DTC patients.

A limitation in the development process of the PtDAs is the reliability of evidence regarding the information presented about the clinical outcomes of the different options. This limitation is inherent to the lack of prospective trials on head-to-head comparisons of the treatment options, which can provide the highest level of evidence. Nonetheless, shared-decision making is appropriate for situations where there is insufficient evidence that one option is superior to another, which is also the case for the decisions for which the PtDA has been developed in the present study. Moreover, the information provided in the PtDAs is based on the best available evidence. First, regarding the extent of surgery, only retrospective trials are available, showing no differences in oncological outcomes in terms of RR and OS (6). Second, for the use of TKIs, only two RCTs are available (24, 25). The outcome in terms of OS has not been published yet. Additionally, in daily practice, the effect of TKIs can be different, as the treatment regimens used in practice may differ from regimens used in the RCTs, particularly because of ad hoc individualized dose adjustments in patients who are deemed more prone to toxicity or who develop AEs. Furthermore, information desired in rural locations or other countries may differ, potentially making the PtDAs less applicable in settings where less accurate diagnostic tools or less experienced physicians are available. As DTC research continues, new studies may require updating of outcome rates and treatment modalities. Moreover, a prospective randomized controlled trial is ongoing (NCT03905369) investigating the effect and implementation of the PtDAs.

Conclusion

Novel evidence-based PtDAs were created for patients with DTC. These PtDAs were positively evaluated to support patients and physicians in shared decision-making by patients having undergone the treatments, patient advocates, and physicians. The PtDAs address an important need for DTC patients and aim to increase patient knowledge and guide patients toward an informed decision (23). The PtDAs will be made publicly available after the large prospective trial has been completed.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

Ethical review and approval was not required for the study on human participants in accordance with the local legislation and institutional requirements. Written informed consent from the patients/participants or patients/participants’ legal guardian/next of kin was not required to participate in this study in accordance with the national legislation and the institutional requirements.

Author contributions

AK: conceptualization (equal), writing —original draft preparation (lead), and writing —review and editing (lead). RH: conceptualization (equal) and writing —review and editing (equal). PO: writing —review and editing (equal). RN-M: conceptualization (equal) and writing —review and editing (equal). PS: conceptualization (equal) and writing —review and editing (equal). COMBO study group: conceptualization (supporting). All authors have read and agreed to the published version of the manuscript.

Group members of COMBO study group

Marieke Snel; m.snel@lumc.nl, Noortje van der Kleij-Corssmit; e.p.m.vanderkleij-corssmit@lumc.nl, Johannes Bonenkamp; han.bonenkamp@radboudumc.nl, Koen Dreijerink; k.dreijerink@amsterdamumc.nl, Evelien van Dam, Grard Nieuwenhuijzen; grardnieuwenhuijzen@catharinaziekenhuis.nl, Mariel Keemers; m.keemers@cwz.nl, Lieke Welling; l.welling@lumc.nl, Iris van der Ploeg; i.vd.ploeg@nki.nl, Sanne Engelen; sanne.engelen@mumc.nl, Danielle Dercks; d.dercks@schildklier.nl, Frans Geenen; fgeenen@xs4all.nl.

Funding

This research was funded by the Dutch Cancer Society, 11463.

Acknowledgments

We thank all patients, patient advocates, and physicians from collaborating hospitals who reviewed several drafts of the DAs.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

1. Cabanillas ME, McFadden DG, Durante C. Thyroid cancer. Lancet (2016) 388(10061):2783–95. doi: 10.1016/S0140-6736(16)30172-6

PubMed Abstract | CrossRef Full Text | Google Scholar

2. McLeod DS, Sawka AM, Cooper DS. Controversies in primary treatment of low-risk papillary thyroid cancer. Lancet (2013) 381(9871):1046–57. doi: 10.1016/S0140-6736(12)62205-3

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Vaccarella S, Franceschi S, Bray F, Wild CP, Plummer M, Dal Maso L. Worldwide thyroid-cancer epidemic? the increasing impact of overdiagnosis. N Engl J Med (2016) 375(7):614–7. doi: 10.1056/NEJMp1604412

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Sawka AM, Straus S, Brierley JD, Tsang RW, Rotstein L, Rodin G, et al. Decision aid on radioactive iodine treatment for early stage papillary thyroid cancer–a randomized controlled trial. Trials (2010) 11:81. doi: 10.1186/1745-6215-11-81

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Oncoline. Schildkliercarcinoom. (2015). Available at: https://richtlijnendatabase.nl/richtlijn/schildkliercarcinoom/algemeen.html.

Google Scholar

6. Bojoga A, Koot A, Bonenkamp J, de Wilt J, Inthout J, Stalmeier P, et al. The impact of the extent of surgery on the long-term outcomes of patients with low-risk differentiated non-medullary thyroid cancer: a systematic meta-analysis. J Clin Med (2020) 9(7):2316. doi: 10.3390/jcm9072316

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Tuttle RM. Controversial issues in thyroid cancer management. J Nucl Med (2018) 59(8):1187–94. doi: 10.2967/jnumed.117.192559

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Durante C, Montesano T, Torlontano M, Attard M, Monzani F, Tumino S, et al. Papillary thyroid cancer: time course of recurrences during postsurgery surveillance. J Clin Endocrinol Metab (2013) 98(2):636–42. doi: 10.1210/jc.2012-3401

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Murthy SP, Balasubramanian D, Anand A, Limbachiya SV, Subramaniam N, Nair V, et al. Extent of thyroidectomy in differentiated thyroid cancers-review of evidence. Indian J Surg Oncol (2018) 9(1):90–6. doi: 10.1007/s13193-017-0661-2

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Kim SH, Roh JL, Gong G, Cho KJ, Choi SH, Nam SY, et al. Differences in the recurrence and survival of patients with symptomatic and asymptomatic papillary thyroid carcinoma: an observational study of 11,265 person-years of follow-up. Thyroid (2016) 26(10):1472–9. doi: 10.1089/thy.2016.0238

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Lamartina L, Grani G, Durante C, Borget I, Filetti S, Schlumberger M. Follow-up of differentiated thyroid cancer - what should (and what should not) be done. Nat Rev Endocrinol (2018) 14(9):538–51. doi: 10.1038/s41574-018-0068-3

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Ibrahim EY, Busaidy NL. Treatment and surveillance of advanced, metastatic iodine-resistant differentiated thyroid cancer. Curr Opin Oncol (2017) 29(2):151–8. doi: 10.1097/CCO.0000000000000349

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Babu G, Kainickal CT. Update on the systemic management of radioactive iodine refractory differentiated thyroid cancer (Review). Mol Clin Oncol (2021) 14(2):35. doi: 10.3892/mco.2020.2197

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Durante C, Haddy N, Baudin E, Leboulleux S, Hartl D, Travagli JP, et al. Long-term outcome of 444 patients with distant metastases from papillary and follicular thyroid carcinoma: benefits and limits of radioiodine therapy. J Clin Endocrinol Metab (2006) 91(8):2892–9. doi: 10.1210/jc.2005-2838

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Haugen BR, Alexander EK, Bible KC, Doherty GM, Mandel SJ, Nikiforov YE, et al. 2015 American Thyroid association management guidelines for adult patients with thyroid nodules and differentiated thyroid cancer: the American thyroid association guidelines task force on thyroid nodules and differentiated thyroid cancer. Thyroid (2016) 26(1):1–133. doi: 10.1089/thy.2015.0020

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Fleeman N, Houten R, Bagust A, Richardson M, Beale S, Boland A, et al. Lenvatinib and sorafenib for differentiated thyroid cancer after radioactive iodine: a systematic review and economic evaluation. Health Technol Assess. (2020) 24(2):1–180. doi: 10.3310/hta24020

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Fugazzola L, Elisei R, Fuhrer D, Jarzab B, Leboulleux S, Newbold K, et al. 2019 European Thyroid association guidelines for the treatment and follow-up of advanced radioiodine-refractory thyroid cancer. Eur Thyroid J (2019) 8(5):227–45. doi: 10.1159/000502229

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Stacey D, Légaré F, Col NF, Bennett CL, Barry MJ, Eden KB, et al. Decision aids for people facing health treatment or screening decisions. Cochrane Database Syst Rev (2014) 1):Cd001431. doi: 10.1002/14651858.CD001431.pub4

CrossRef Full Text | Google Scholar

19. Volk RJ, Llewellyn-Thomas H, Stacey D, Elwyn G. Ten years of the international patient decision aid standards collaboration: evolution of the core dimensions for assessing the quality of patient decision aids. BMC Med Inform Decis Mak. (2013) 13 Suppl 2(Suppl 2):S1. doi: 10.1186/1472-6947-13-S2-S1

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Elwyn G, O'Connor A, Stacey D, Volk R, Edwards A, Coulter A, et al. Developing a quality criteria framework for patient decision aids: online international Delphi consensus process. Bmj (2006) 333(7565):417. doi: 10.1136/bmj.38926.629329.AE

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Coulter A, Stilwell D, Kryworuchko J, Dolan Mullen P, Ng CJ, van der Weijden T. A systematic development process for patient decision aids. BMC Med Inform Decis Mak. (2013) 13 Suppl 2(Suppl 2):S2. doi: 10.1186/1472-6947-13-S2-S2

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Kvdg CBO. Handleiding keuzehulpontwikkeling. (2009). Available at: https://docplayer/nl/7361403-Handleiding-keuzehulpontwikkeling.html.

Google Scholar

23. Koot A, Netea-Maier R, Ottevanger P, Hermens R, Stalmeier P. Needs, preferences, and values during different treatment decisions of patients with differentiated thyroid cancer. J Personalized Med (2021) 11(7):682. doi: 10.3390/jpm11070682

CrossRef Full Text | Google Scholar

24. Schlumberger M, Tahara M, Wirth LJ, Robinson B, Brose MS, Elisei R, et al. Lenvatinib versus placebo in radioiodine-refractory thyroid cancer. N Engl J Med (2015) 372(7):621–30. doi: 10.1056/NEJMoa1406470

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Brose MS, Nutting CM, Jarzab B, Elisei R, Siena S, Bastholt L, et al. Sorafenib in radioactive iodine-refractory, locally advanced or metastatic differentiated thyroid cancer: a randomised, double-blind, phase 3 trial. Lancet (2014) 384(9940):319–28. doi: 10.1016/S0140-6736(14)60421-9

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Verburg FA, Flux G, Giovanella L, van Nostrand D, Muylle K, Luster M. Differentiated thyroid cancer patients potentially benefitting from postoperative I-131 therapy: a review of the literature of the past decade. Eur J Nucl Med Mol Imaging. (2020) 47(1):78–83. doi: 10.1007/s00259-019-04479-1

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Lezen SM, Hoefnagel M, Bakker H. Keurmerk gewone taal. (2003). Available at: www.stichtingmakkelijklezen.nl.

Google Scholar

28. Cambridge University Press & Assessment. (2023). Available at: https://www.cambridgeenglish.org/exams-and-tests/cefr.

Google Scholar

29. Heij K VW. Schrijven in eenvoudig nederlands. Den Haag: Sdu uitgevers (2006).

Google Scholar

30. Lundgrén-Laine H, Salanterä S. Think-aloud technique and protocol analysis in clinical decision-making research. Qual Health Res (2010) 20(4):565–75. doi: 10.1177/1049732309354278

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Hawley ST, Zikmund-Fisher B, Ubel P, Jankovic A, Lucas T, Fagerlin A. The impact of the format of graphical presentation on health-related knowledge and treatment choices. Patient Educ Couns. (2008) 73(3):448–55. doi: 10.1016/j.pec.2008.07.023

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Tuttle RM, Tala H, Shah J, Leboef R, Ghossein R, Gonen M, et al. Estimating risk of recurrence in differentiated thyroid cancer after total thyroidectomy and radioactive iodine remnant ablation: using response to therapy variables to modify the initial risk estimates predicted by the new American thyroid association staging system. Thyroid (2010) 20(12):1341–9. doi: 10.1089/thy.2010.0178

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Castagna MG, Maino F, Cipri C, Belardinni V, Theodoropoulou A, Cevnini G, et al. Delayed risk stratification, to include the response to initial treatment (surgery and radioiodine ablation), has better outcome predictivity in differentiated thyroid cancer patients. Eur J Endocrinol (2011) 165(3):441–6. doi: 10.1530/EJE-11-0466

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Molinaro E, Giani C, Agate L, Biagini A, Bianchi F, Brozzi F, et al. Patients with differentiated thyroid cancer who underwent radioiodine thyroid remnant ablation with low-activity ¹³¹I after either recombinant human TSH or thyroid hormone therapy withdrawal showed the same outcome after a 10-year follow-up. J Clin Endocrinol Metab (2013) 98(7):2693–700. doi: 10.1210/jc.2012-4137

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Brassard M, Borget I, Edet-Sanson A, Giraudet A-L, Mundler O, Toubeau M, et al. Long-term follow-up of patients with papillary and follicular thyroid cancer: a prospective study on 715 patients. J Clin Endocrinol Metab (2011) 96(5):1352–9. doi: 10.1210/jc.2010-2708

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Leboulleux S, Bournaud C, Chougnet CN, Zerdoud S, Al Ghuzlan A, Catargi B, et al. Thyroidectomy without radioiodine in patients with low-risk thyroid cancer. N Engl J Med (2022) 386(10):923–32. doi: 10.1056/NEJMoa2111953

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Turrentine FE, Wang H, Simpson VB, Jones RS. Surgical risk factors, morbidity, and mortality in elderly patients. J Am Coll Surg (2006) 203(6):865–77. doi: 10.1016/j.jamcollsurg.2006.08.026

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Brito JP, Moon JH, Zeuren R, Kong SH, Kim YG, Iniguez-Ariza NM, et al. Thyroid cancer treatment choice: a pilot study of a tool to facilitate conversations with patients with papillary microcarcinomas considering treatment options. Thyroid (2018) 28(10):1325–31. doi: 10.1089/thy.2018.0105

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Pitt SC, Saucke MC. Thyroid cancer treatment choice toolkit. University of Wisconsin - Madison School of Medicine and Public Health - Department of Surgery. (2017). Available at: https://www.hipxchange.org/ThyroidCancerTreatmentChoice.

Google Scholar

40. Pitt SC, Saucke MC. Novel decision support interventions for low-risk thyroid cancer. JAMA Otolaryngol Head Neck Surg (2020) 146(11):1079–81. doi: 10.1001/jamaoto.2020.2279

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Fagerlin A, Pignone M, Abhyankar P, Col N, Feldman-Stewart D, Gavaruzzi T, et al. Clarifying values: an updated review. BMC Med Inform Decis Mak. (2013) 13 Suppl 2(Suppl 2):S8. doi: 10.1186/1472-6947-13-S2-S8

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Feldman-Stewart D, Tong C, Siemens R, Alibhai S, Pickles T, Robinson J, et al. The impact of explicit values clarification exercises in a patient decision aid emerges after the decision is actually made: evidence from a randomized controlled trial. Med Decis Making. (2012) 32(4):616–26. doi: 10.1177/0272989X11434601

PubMed Abstract | CrossRef Full Text | Google Scholar