Implementation of thyroid-related patient-reported outcomes in routine clinical practice

Patient-reported outcomes (PROs) are increasingly used in clinical practice to improve clinical care. Multiple studies show that systematic use of PROs can enhance communication with patients and improve patient satisfaction, symptom management and quality of life. Further, such data can be aggregated to examine health levels for patient groups, improve quality of care, and compare patient outcomes at the institutional, regional or national level. However, there are barriers and challenges that should be handled appropriately to achieve successful implementation of PROs in routine clinical practice. This paper briefly overviews thyroid-related PROs, describes unsolved quality of life issues in benign thyroid diseases, provides examples of routine collection of PROs, and summarizes key points facilitating successful implementation of thyroid-related PROs in routine clinical practice. Finally, the paper touches upon future directions of PRO research.

Background

Quality of life (QoL) can be measured by patient-reported outcomes (PROs). According to the US Food and Drug Administration, “a PRO is a measurement of any aspect of a patient’s health status that comes directly from the patient (i.e., without the interpretation of the patient’s responses by a physician or anyone else)” (1). The prognosis of benign thyroid diseases, including hypothyroidism, is generally good with appropriate treatment, although many patients experience major QoL impairments at the time of disease presentation (2–6). Furthermore, despite good biochemical management and/or definitive treatment (e.g. goitre surgery), long-term follow-up studies have demonstrated persistent QoL deficits (3–5, 7, 8). Therefore, a major goal of clinical trials in thyroid diseases is to scrutinize new treatment strategies or novel therapies, with the aim of optimizing QoL. During the last two decades, PROs have been extensively and successfully applied in clinical studies of benign thyroid diseases. In the beginning, PROs were administered as paper surveys, but PROs are increasingly collected electronically via computers, tablets, or smart phones (9). The electronic methods of collecting PRO data allow real-time QoL measurements and presentation to clinicians in busy clinical practices, i.e. the electronic methods make routine PRO assessments feasible (10). However, no studies have described or systematically evaluated an implementation of PROs in routine thyroid clinical practice, despite accumulating evidence from other fields (e.g. oncology and inflammatory diseases) showing that routine use of PRO’s can improve clinical care (9, 11–14). The implementation of routine PROs is anticipated to improve several aspects of patient care by: identifying unrecognized and potentially treatable health problems, assessing the effectiveness of different treatments, detecting adverse effects, monitoring disease progress, improving patient-physician communication, and promoting shared decision making and patient empowerment (15–17). Eventually, these improvements may lead to better health outcomes in the form of reduced symptoms, better QoL and enhanced patient satisfaction (16), all known to be affected in individuals with thyroid disease in general, including hypothyroidism (18). Thyroid diseases are3interrelated both within the individual patient (e.g. co-existence3of hypothyroidism and goitre in classical Hashimoto’s disease) and across time (e.g. hypothyroidismresulting from treatment of non-toxic goitre or hypothyroidism leading to thyrotoxicosis in periods with overtreatment) and thus, implementation of PROs in an outpatient clinic should cover all benign thyroid conditions. Therefore, rather than focusing on hypothyroidism alone, this review considers such broader perspectives within a thyroid outpatient clinic.

Thyroid-related PROs

There are two broad categories of PROs, i.e. disease-specific and generic. Disease-specific tools typically assess symptoms, functioning and patient perceptions that relate to a well-defined disease or condition, whereas generic tools evaluate broader categories that can be affected by a multitude of conditions. Disease-specific tools tend to be more sensitive to differences among clinically relevant groups and more responsive to small changes in health, as compared to generic tools (see Table 1) (23). The 36-item Short-Form Health Survey version 2 (SF-36v2) (21, 24, 25) and the EuroQol Group EQ-5D Health Survey (EQ-5D) (19) are two of the most widely used generic PRO instruments. In this paper, we focus on PROs developed for benign thyroid diseases. A systematic review from 2016 evaluated the measurement properties of PROs targeted for benign thyroid diseases (26). Based on measurement properties, that review recommended the Thyroid-Related Patient-Reported Outcome (ThyPRO) (22, 27–29) to assess QoL in patients with benign thyroid disease, while measurement properties for the Graves’ Ophthalmopathy Quality Of Life survey (GO-QOL) (20, 30–32) and the Thyroid Treatment Satisfaction Questionnaire (ThyTSQ) (33, 34) were deemed satisfactory for measuring QoL in Graves’ orbitopathy and treatment satisfaction with hypothyroidism, respectively. The strengths and limitations of all PROs intended for use in patients with benign thyroid diseases are tabulated in the systematic review paper by Wong et al. (26).

TABLE 1

Table 1 There are two broad categories of PROs, generic and disease-specific.

The ThyPRO survey covers all benign diseases (hypothyroidism, hyperthyroidism, goitre, and orbitopathy), which is advantageous for the reasons stated above. The ThyPRO survey is the most extensively validated PRO for benign thyroid diseases (26), and it has proven more responsive to clinical changes than the most widely used generic survey, i.e. the SF-36v2 Health Survey (21, 28). The original ThyPRO questionnaire consisted of 85 items summarized into 13 scales, as well as a single item measuring overall impact of thyroid disease on QoL. Later, the shorter 39-item version (ThyPRO-39) was developed, and it has demonstrated good measurement properties (35).

The GO-QOL survey consists of 16 items summarized in two subscales, and it is the most widely used PRO to assess QoL in Graves’ orbitopathy. Validated condition-specific PROs that may be relevant in a thyroid surgery setting exist, e.g. the Voice Handicap Index-10 (VHI-10) (36) or the Swallowing Quality of Life questionnaire (SWAL-QOL) (37), both of which have been applied to assess quality or treatment effects of thyroid surgery (38, 39).

The choice of PRO depends on the rationale for assessment, and hence researchers and clinicians should first decide what they want to measure. This might be e.g. impact of disease or its treatment on symptoms, daily activities, emotional well-being or side effects. The PRO selected should be well validated and have appropriate measurement properties for the disease in question. Finally, the researchers and clinicians need to identify the most appropriate tool for that task.

Important QoL issues in benign thyroid diseases

Numerous studies of benign thyroid diseases have demonstrated deficits in multiple QoL domains (4–6, 40–43). In this section we discuss a few selected studies that have examined the impact of treatment on QoL. These studies also exemplify some of the many unsolved QoL issues in thyroid diseases. Stott et al. conducted a double-blind, randomized, placebo-controlled, parallel-group trial in 737 older adults with persistent subclinical hypothyroidism, where participants were randomized to levothyroxine or placebo (44). The results demonstrated that levothyroxine had no QoL benefits in older people with subclinical hypothyroidism, as measured by the ThyPRO survey. Shakir et al. conducted a randomized, double-blind, crossover study of 75 hypothyroid patients randomly allocated to 1 of 3 treatment arms, levothyroxine, levothyroxine + liothyronine or desiccated thyroid extract (45). Outcomes were similar among the three groups, as measured by four different validated PROs. However, patients most symptomatic on levothyroxine alone preferred and responded positively to therapy with levothyroxine + liothyronine or desiccated thyroid extract. Another recent study examined the effect of natural dessicated thyroid, as evaluated with ThyPRO-39 (35) and EQ-5D-L (19) at baseline and six months after initiation of treatment (46). Patients with levothyroxine unresponsive hypothyroidism experienced significant improvement in QoL. A recent consensus paper summarizes the evidence on levothyroxine/liothyronine combination therapy (47). However, whether to initiate substitution therapy in subclinical hypothyroidism and whether to choose combination therapy rather than monotherapy for QoL indications is currently unsettled which beyond QoL issues needs to take into consideration the time-dependent thyroid dysfunction related excess morbidity and mortality (48–50). For a comprehensive insight into various QoL issues in hypothyroidism we refer to a recent review (5).

Grove-Laugesen et al. randomized patients with first time diagnosis of Graves’ disease to vitamin D or placebo in addition to standard treatment with antithyroid drugs (42). Unexpectedly, the change of the ThyPRO Composite QoL scale was significantly worse in the Vitamin D group and the same tendency was seen for the Overall QoL-impact scale, Impaired Daily Life scale and Hyperthyroid Symptoms scale. Cramon et al. prospectively examined QoL before and 6 months after initiation of treatment in patients with toxic nodular goitre or Graves’ disease (51). At baseline, both patient groups had significantly poorer QoL on all ThyPRO and SF-36 domains, as compared with general population samples, and QoL improved markedly after six months of treatment for Graves’ disease while improving more modestly in patients with toxic nodular goitre. In both groups, QoL impairments persisted six months after treatment on multiple domains compared with general population samples. Törring et al. conducted a long-term follow-up study in patients with Graves’ disease treated with antithyroid drugs, radioiodine therapy, or surgery. The patients showed QoL impairments compared to general population norms 6-10 years after treatment (7), but patients treated with radioiodine therapy had worse long-term QoL than those treated with antithyroid drugs or surgery. Important limitations were lack of randomization and insufficient data on comorbidity.

As demonstrated by the above mentioned three studies, treatment may negatively and unexpectedly affect QoL, exemplified by QoL impairments both in the short- and long-term. Future clinical trials should be designed in such a way that both patients and clinicians can be advised to choose the most optimal treatment for effectively restoring the QoL.

Examples of routine use of PROs

Implementing PROs in routine clinical practice is another option to better comprehend the impact of disease and treatment on QoL, and thereby better identify novel treatment targets. While no studies have been published concerning the potential effects of routine PRO assessment among thyroid patients, positive effects have been demonstrated in other patient groups. In a sentinel study by Velikova et al. 286 cancer patients were randomized to either 1) completion of PROs at routine clinical visits and feedback of results to physicians, 2) PRO completion, without feedback, or 3) no PRO assessment as a control group (10). Routine PRO assessment had a beneficial impact on physician-patient communication and resulted in better QoL and emotional functioning in some patients. Basch et al. assessed the impact of PRO monitoring in 766 cancer patients initiating chemotherapy (52). Participants were randomized to usual care or the PRO group, in which frequent symptoms associated with cancer treatment were self-reported at and between visits. Severity or worsening of symptoms triggered an email alert to a clinical nurse, and the reported symptoms were accessible for the treating oncologist as well. Integration of PROs significantly increased overall survival. Potential mechanisms were early responsiveness to symptoms preventing adverse downstream consequences and ability to tolerate chemotherapy longer than the usual care group. Short et al. assessed the value and feasibility of web-based PRO assessments within routine HIV care (53). The study included 1630 HIV patients who completed PROs in the outpatient clinic before their routine care consultation. The PROs effectively identified issues to address, particularly anxiety and suicidal ideation. The majority of patients (82%) and health care providers (82%) indicated that the PROs added value to the visits. Mistry and colleagues applied the Rheumatoid Arthritis Impact of Disease (RAID) survey at routine visits in routine care (54). The PROs could identify patients in remission or low disease activity with a high positive predictive value, which provided an opportunity to optimize outpatient visits for these patients. Additionally, the routine use of PRO data revealed a high burden of unmet needs, in particular fatigue and sleep problems, to be addressed in future clinical care.

Routine PRO assessment thus has the potential to improve physician-patient communication, improve QoL, identify unmet needs, potentially save outpatient visits in patients with better QoL, and for some diseases even improve overall survival. However, there are also barriers to implementation of routine PRO collection. Limited resources, both regarding funding as well as the time-consuming nature of developing electronic infrastructure and implementing routine PROs are important barriers (15). Clinicians can be reluctant to use PROs routinely because they fear it will add to their already busy workload, and further, the clinicians may contend that they already understand their patients’ problems and do not need extra patient-reported information (11). For more thorough summaries on the effect of routine PRO collection, including challenges and barriers, we refer to the review papers by Chen et al., 2013, Howell et al., 2015, and Nguyen et al., 2021 (12–14). It is unresolved whether these potential benefits apply to routine PRO collection in benign thyroid diseases.

Implementation of thyroid-related PROs in routine clinical practice

There are several features of thyroid diseases, including hypothyroidism, that make routine PRO assessments particularly well suited for these patients: thyroid hormones affect all organ systems and thus cause a variety of different symptoms and QoL impairments including mental health issues; QoL impairments influence the choice of intervention (e.g. pressure symptoms and cosmetic concerns in non-toxic goitre); a substantial number of biochemically well-treated patients experience reduced QoL and may thus experience lack of congruence between the focus of the endocrinologist and themselves (55, 56); thyroid diseases are often chronic and occur at all ages, including during working life; thyroid diseases are rarely life-threatening, and focus on relevant QoL issues may thus have relatively large impact. The use of PROs in clinical studies of benign thyroid diseases is already well established. However, before the use of PROs can be extended to routine clinical practice, it is necessary to test the effect of routine PROs in feasibility studies and trials. Such studies and trials can be costly and require funding, which is often challenging for thyroid research.

Based on guidelines and studies of routine PRO collection, this section summarizes important topics to consider prior to implementation of routine thyroid-related PROs in feasibility studies and trials.

Goals

There is a range of possible goals: investigating the impact of thyroid disease on QoL, evaluating treatment, identifying symptoms that may be alleviated, aiding in treatment decision-making, improving patient-physician communication, etc. Before implementing PROs in clinical practice it is important to clarify how the implementation is expected to have an effect, thereby guiding study design and selection of the most appropriate PROs (57).

Selection of thyroid-related PROs

A literature search can both identify studies evaluating QoL and the PROs used for these measurements. An important consideration for the candidate tools is whether they address the symptoms and QoL domains most relevant for the target patient population (i.e. content validity). Next, it is crucial that the chosen PRO is thoroughly validated. In this respect, core considerations include: construct validity, reliability, responsiveness, discrimination ability, meaningful change, and translation validity (58). A systematic review from 2016 identified QoL instruments developed for thyroid diseases and graded these according to methodological quality and overall levels of evidence (26). The length of the PRO should be considered as the measure must neither be overly burdensome for patients nor for clinicians (4).

Resources and education

Experience from previous studies shows that it is important not to underestimate the amount of time and human resources needed to establish and sustain PRO projects in clinical routine practice (15). Identifying local clinical key persons (e.g. physician or/and nurse) who can facilitate the administration of PROs and answer questions from clinicians and patients may be an important factor for successful implementation (9). The staff should be educated to administrate, understand, and evaluate the chosen PRO before the implementation starts. Moreover, technology and technical support should be in place to ensure efficient collection of electronic PRO data.

PRO administration

The PROs should preferably be administered electronically, as paper surveys require time consuming data entry by the staff with risk of data entry errors. Collection via web-based systems accessible from the patients’ home with email prompts is one option. Another is a tablet solution in the out-patient clinic, where surveys are completed immediately before the consultation. Ideally, data should immediately be scored, and the results presented graphically in the electronic health record. In this way, the PRO results are readily available to the clinician to help guide the subsequent dialogue with the patient.

Interpretability and feedback

The PRO results should be easily understood and interpretable, for example by highlighting relevant changes and providing reference scores (see Figures 1, 2). In case the PRO results are to be fed back to patients, they should also be interpretable to lay persons. However, the best way to convey PRO information to patients and clinicians has yet to be determined (59–62).

FIGURE 1

Figure 1 Radar plot showing ThyPRO scales scores for patients with Graves’ disease at baseline (i.e. before initiation of treatment, N=88) and 6 months after treatment (N=66), as well as scores from a general reference population (N=739). The scales range from 0 to 100, with higher scores indicating worse health status. *Items in these scales are asked with attribution to thyroid disease and cannot be answered by respondents from the general population. A radar plot gives a comprehensive overview of all quality of life domains and a few assessments can be shown in one plot. However, this type of graph is not well suited for multiple longitudinal assessments. The radar plot is based on data from Cramon et al. (51). GD: Graves’ disease.

FIGURE 2

Figure 2 This figure illustrates how multiple longitudinal quality of life assessments can be presented to the clinician in the electronic health record. In this made-up example, a patient with newly diagnosed Graves’ disease completes the ThyPRO survey before each consultation. An indicator of worse QoL is seen to the left of the vertical axis as some PROs are scored opposite than ThyPRO (i.e. higher scores indicate better QoL). Reference scores are shown with blue dotted lines to facilitate the interpretation of patient scores. Scales are marked with an asterisk sign (*) and the scale name highlighted in red if the scores have deteriorated more than the MIC (minimal important change) (29), enabling the clinician to get a fast overview of domains with worsening QoL. Note: Items in scales of the two bottom rows are asked with attribution to thyroid disease and cannot be answered by respondents from the general population.

Ability to act

Clinicians are less inclined to raise issues if they cannot do anything about them (17, 57, 63). This may explain why PRO implementation increases discussion of symptoms more often than of complex QoL issues (e.g. function or participation) (64). For example, the clinician may be more prone to initiate treatment for worsening of eye symptoms in Graves’ orbitopathy, as compared to handling daily function problems in a biochemically well-treated patient with hypothyroidism. However, recognizing that more complex QoL issues may be as important as more traditional measures might have great impact for the patient and can enhance treatment satisfaction. Additionally, poor or worsening QoL in euthyroid patients with Hashimoto’s thyroiditis can easily be attributed to thyroid disease despite it may be caused by other undiagnosed (autoimmune) diseases. Therefore, QoL deterioration may help the clinician consider non-thyroid disease or consider the possibility to change treatment strategy (e.g., switch to combination therapy). An automatic algorithm assessing PRO data collected between visits may identify patients in need of an extra consultation or vice versa, omission of a scheduled consultation in case of good QoL. Action guidance may be developed using a systematic and multidisciplinary approach (65–68).

Evaluation of PRO value

A feasibility study with a limited number of patients and clinicians is recommended before more widespread implementation of routine PRO assessments. Such a study can identify challenges and barriers to PRO implementation, and moreover, examine how patients and clinicians value the PRO data. Ideally, feasibility studies should be followed by randomized controlled trials evaluating the impact of routine PRO collection on patient-physician communication, satisfaction, empowerment, QoL, and other health outcomes.

Previous studies of routine use of PRO have suggested that clinicians and researchers often experience practical challenges, such as administrative burden, problems with PRO interpretation, and perhaps even skepticism of the clinical value (15). We therefore recommend the abovementioned topics to be considered in the planning process in order to successfully implement PROs in routine clinical practice.

Perspectives and future directions

The routine use of PROs in clinical practice has the potential to improve the clinical care of individual patients with thyroid diseases by improving physician-patient communication, improving QoL, identifying unmet needs, and perhaps saving unnecessary outpatient visits. Moreover, routine PRO assessments can provide data on the performance of different treatments in ‘real-world’ conditions (i.e. effectiveness), rather than under ideal or controlled circumstances (i.e. efficacy) (69). The use of routine PROs is steadily increasing in malignant and non-malignant diseases, both at individual centres and across entire health care systems (11). Routine PROs can be used for benchmarking and quality improvement purposes when implemented at multiple centres within a health care system. Importantly, results of PROs can guide patients in their choice of provider as well as the choice of treatment (11).

In general, PROs are retrospective surveys, often with a recall period of one to four weeks. By ecological momentary assessments (EMAs) - an advancing method in the science of PROs - the current QoL is measured with repeated real-time assessments (70, 71). Typically, patients are notified several times daily to complete a very brief survey on their cell phone (see Figure 3). Theoretically, this approach has a number of advantages over retrospective surveys: 1) The assessments are not influenced by recall bias, 2) repeated daily samplings enable investigation of symptoms and QoL over time (e.g. daily fluctuations), 3) the method provides ecological validity, i.e. the assessments are made while patients are living their daily lives, which allows for contextual investigations of QoL, and 4) the data can be correlated to or even triggered by other relevant data retrieved by the cell phone (e.g. GPS-tracker, heart rhythm, diurnal rhythm, etc.) (72).

FIGURE 3

Figure 3 This figure illustrates three different user interfaces of the original smart phone app developed for EMA ThyPRO assessments (72). Left: it is possible to adapt the EMA assessments to the daily rhythm of the participants, and to pause EMA assessments (e.g. due to a business meeting). Middle: EMA notification. It is possible to postpone or decline EMA assessments on occasions where participants are unable to complete assessments. Right: EMA item with five response categories (in this example ‘Some’ has been chosen). It is possible to see as well as change all previous answers until the EMA assessment is completed. App: application; EMA: ecological momentary assessment.

An EMA version of ThyPRO has recently been developed and applied in a few studies (71–74). The concept of EMA has the potential to increase our understanding of the impact of thyroid diseases and their treatment on QoL. EMA assessments in patients with hypothyroidism treated with liothyronine is an obvious example for application of this method, as the diurnal fluctuations of plasma thyroid hormones concentrations associated with this treatment may cause swift changes in symptoms. The EMA method could also be used for short periods to closer investigate symptoms of interest detected by routine retrospective PROs. When using PROs, there is a risk that the act of monitoring symptoms may affect symptom levels (so-called reactivity). The risk of reactivity may be higher with repeated real-time measurements for routine care. However, a previous study on momentary pain demonstrated only minimal reactivity (75). The conversion of retrospective PROs to momentary versions can be a difficult task as the interpretation of the momentary wording of the questions may be different from that of the retrospective wording. A study by Boesen et al. demonstrated that cognitive interviewing can be an efficient tool for developing and evaluating momentary questions (71).

There are important limitations in current QoL research in thyroid diseases. As already mentioned, it is well-documented that patients with benign thyroid diseases experience both short- and long-term QoL impairments. To what extent such deficits are due to thyroid disease per se, pre-existing morbidity or subsequent co-morbidity is currently unsettled. More research is needed to examine the relationship between QoL and the time-dependent thyroid dysfunction-related excess morbidity. Implementation of thyroid-related PROs in daily clinical practice could potentially contribute to expand our knowledge on this important topic. In general, PROs do not contain questions about patient preferences, although these preferences are very important in treatment decisions. In research studies, The PROs can be supplemented with questions about patient preferences when relevant for the research questions.

The movement towards patient-centered health care has resulted in a greater interest in symptoms and QoL issues reported directly by patients. The increasing use of PROs in clinical research has expanded our knowledge of the impact of thyroid diseases and their treatment on QoL. We suggest implementation of PROs in routine clinical practice to be established through meticulously designed studies and trials questing the optimal mode and evaluating its impact on key patient-centered outcomes as well as health service performance. Routine QoL assessments provide the opportunity to monitor the impact of treatment on the outcomes most meaningful for patients and to help shaping the delivery of patient-centered health care.

Author contributions

PC participated in the planning process, wrote the manuscript draft, and approved the final manuscript. JB, MG, VB, SB, LH, UF-R, ÅR and TW participated in the planning process, read, and commented the manuscript draft, and approved the final manuscript. All authors contributed to the article and approved the submitted version.

Funding

UF-R’s research salary was sponsored by a research grant from Kirsten and Freddy Johansen’s Foundation

Conflict of interest

Author JB was employed by QualityMetric Incorporated, LLC.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

1. U.S. Department of Health and Human Services FDA Center for Drug Evaluation and Research, U.S. Department of Health and Human Services FDA Center for Biologics Evaluation and Research USD of H and HSFC for D and RH. Guidance for industry: patient-reported outcome measures: use in medical product development to support labeling claims: draft guidance. Health Qual Life Outcomes (2006) 4(79):1–20. doi: 10.1186/1477-7525-4-79

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Bianchi GP, Zaccheroni V, Solaroli E, Vescini F, Cerutti R, Zoli M, et al. Health-related quality of life in patients with thyroid disorders. Qual Life Res (2004) 13(1):45–54. doi: 10.1023/B:QURE.0000015315.35184.66

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Watt T, Groenvold M, Rasmussen ÅK, Bonnema SJ, Hegedüs L, Bjorner JB, et al. Quality of life in patients with benign thyroid disorders. a review. Eur J Endocrinol (2006) 154(4):501–10. doi: 10.1530/eje.1.02124

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Watt T, Cramon P, Frendl DM, Ware JE. Assessing health-related quality of life in patients with benign non-toxic goitre. Best Pract Res Clin Endocrinol Metab (2014) 28(4):559–75. doi: 10.1016/j.beem.2014.01.009

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Hegedüs L, Bianco AC, Jonklaas J, Pearce SH, Weetman AP, Perros P. Primary hypothyroidism and quality of life. Nat Rev Endocrinol (2022) 18(4):230–42. doi: 10.1038/s41574-021-00625-8

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Woo T, Li CH, Ganesananthan S, Rajendram R, Uddin J, Lee RWJ, et al. The effect of ophthalmic surgery for graves’ orbitopathy on quality of life: A systematic review and meta-analysis. Thyroid (2022) 32(2):177–87. doi: 10.1089/thy.2021.0411

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Törring O, Watt T, Sjölin G, Byström K, Abraham-Nordling M, Calissendorff J, et al. Impaired quality of life after radioiodine therapy compared to antithyroid drugs or surgical treatment for graves’ hyperthyroidism: A long-term follow-up with the thyroid-related patient-reported outcome questionnaire and 36-item short form health status survey. Thyroid (2019) 29(3):322–31. doi: 10.1089/thy.2018.0315

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Fast S, Hegedüs L, Pacini F, Pinchera A, Leung AM, Vaisman M, et al. Long-term efficacy of modified-release recombinant human thyrotropin augmented radioiodine therapy for benign multinodular goiter: results from a multicenter, international, randomized, placebo-controlled, dose-selection study. Thyroid (2014) 24(4):727–35. doi: 10.1089/thy.2013.0370

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Basch E, Barbera L, Kerrigan CL, Velikova G. Implementation of patient-reported outcomes in routine medical care. Am Soc Clin Oncol Educ B (2018) 23(38):122–34. doi: 10.1200/EDBK_200383

CrossRef Full Text | Google Scholar

10. Velikova G, Booth L, Smith AB, Brown PM, Lynch P, Brown JM, et al. Measuring quality of life in routine oncology practice improves communication and patient well-being: a randomized controlled trial. J Clin Oncol (2004) 22(4):714–24. doi: 10.1200/JCO.2004.06.078

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Nelson EC, Eftimovska E, Lind C, Hager A, Wasson JH, Lindblad S. Patient reported outcome measures in practice. BMJ (2015) 350:g7818. doi: 10.1136/bmj.g7818

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Chen J, Ou L, Hollis SJ. A systematic review of the impact of routine collection of patient reported outcome measures on patients, providers and health organisations in an oncologic setting. BMC Health Serv Res (2013) 13(1):1–24. doi: 10.1186/1472-6963-13-211

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Howell D, Molloy S, Wilkinson K, Green E, Orchard K, Wang K, et al. Patient-reported outcomes in routine cancer clinical practice: a scoping review of use, impact on health outcomes, and implementation factors. Ann Oncol Off J Eur Soc Med Oncol (2015) 26(9):1846–58. doi: 10.1093/annonc/mdv181

CrossRef Full Text | Google Scholar

14. Nguyen H, Butow P, Dhillon H, Sundaresan P. A review of the barriers to using patient-reported outcomes (PROs) and patient-reported outcome measures (PROMs) in routine cancer care. J Med Radiat Sci (2021) 68(2):186–95. doi: 10.1002/jmrs.421

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Chan EKH, Edwards TC, Haywood K, Mikles SP, Newton L. Implementing patient-reported outcome measures in clinical practice: a companion guide to the ISOQOL user’s guide. Qual Life Res (2019) 28(3):621–7. doi: 10.1007/s11136-018-2048-4

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Luckett T, Butow PN, King MT. Improving patient outcomes through the routine use of patient-reported data in cancer clinics: future directions. Psychooncology (2009) 18(11):1129–38. doi: 10.1002/pon.1545

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Boyce MB, Browne JP, Greenhalgh J. The experiences of professionals with using information from patient-reported outcome measures to improve the quality of healthcare: a systematic review of qualitative research. BMJ Qual Saf (2014) 23(6):508–18. doi: 10.1136/bmjqs-2013-002524

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Mitchell AL, Hegedüs L, Žarković M, Hickey JL, Perros P. Patient satisfaction and quality of life in hypothyroidism: An online survey by the british thyroid foundation. Clin Endocrinol (Oxf) (2021) 94(3):513–20. doi: 10.1111/cen.14340

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Herdman M, Gudex C, Lloyd A, Janssen M, Kind P, Parkin D, et al. Development and preliminary testing of the new five-level version of EQ-5D (EQ-5D-5L). Qual Life Res (2011) 20(10):1727–36. doi: 10.1007/s11136-011-9903-x

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Terwee CB, Gerding MN, Dekker FW, Prummel MF, Wiersinga WM. Development of a disease specific quality of life questionnaire for patients with graves’ ophthalmopathy: the GO-QOL. Br J Ophthalmol (1998) 82(7):773–9. doi: 10.1136/bjo.82.7.773

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Ware JE, Sherbourne CD. The MOS 36-item short-form health survey (SF-36). i. conceptual framework and item selection. Med Care (1992) 30(6):473–83.

PubMed Abstract | Google Scholar

22. Watt T, Bjorner JB, Groenvold M, Rasmussen ÅK, Bonnema SJ, Hegedüs L, et al. Establishing construct validity for the thyroid-specific patient reported outcome measure (ThyPRO): an initial examination. Qual Life Res (2009) 18(4):483–96. doi: 10.1007/s11136-009-9460-8

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Patrick DL, Deyo RA. Generic and disease-specific measures in assessing health status and quality of life. Med Care (1989) 27(3 Suppl):S217–32. doi: 10.1097/00005650-198903001-00018

PubMed Abstract | CrossRef Full Text | Google Scholar

24. McHorney CA, Ware JE, Raczek AE. The MOS 36-item short-form health survey (SF-36): II. psychometric and clinical tests of validity in measuring physical and mental health constructs. Med Care (1993) 31(3):247–63. doi: 10.1097/00005650-199303000-00006

PubMed Abstract | CrossRef Full Text | Google Scholar

25. McHorney CA, Ware JE, Rachel Lu JF, Sherbourne CD. The MOS 36-item short-form health survey (SF-36): III. tests of data quality, scaling assumptions, and reliability across diverse patient groups. Med Care (1994) 32(1):40–66. doi: 10.1097/00005650-199401000-00004

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Wong CKH, Lang BHH, Lam CLK. A systematic review of quality of thyroid-specific health-related quality-of-life instruments recommends ThyPRO for patients with benign thyroid diseases. J Clin Epidemiol (2016) 78:63–72. doi: 10.1016/j.jclinepi.2016.03.006

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Watt T, Hegedüs L, Groenvold M, Bjorner JB, Rasmussen ÅK, Bonnema SJ, et al. Validity and reliability of the novel thyroid-specific quality of life questionnaire, ThyPRO. Eur J Endocrinol (2010) 162(1):161–7. doi: 10.1530/EJE-09-0521

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Watt T, Cramon P, Hegedüs L, Bjorner JB, Bonnema SJ, Rasmussen ÅK, et al. The thyroid-related quality of life measure ThyPRO has good responsiveness and ability to detect relevant treatment effects. J Clin Endocrinol Metab (2014) 99(10):3708–17. doi: 10.1210/jc.2014-1322

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Nordqvist SF, Boesen VB, Rasmussen ÅK, Feldt-Rasmussen U, Hegedüs L, Bonnema SJ, et al. Determining minimal important change for the thyroid-related quality of life questionnaire ThyPRO. Endocr Connect (2021) 10(3):316. doi: 10.1530/EC-21-0026

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Terwee CB, Gerding MN, Dekker FW, Prummel MF, van der Pol JP, Wiersinga WM. Test-retest reliability of the GO-QOL: a disease-specific quality of life questionnaire for patients with graves’ ophthalmopathy. J Clin Epidemiol (1999) 52(9):875–84. doi: 10.1016/s0895-4356(99)00069-4

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Terwee CB, Dekker FW, Mourits MP, Gerding MN, Baldeschi L, Kalmann R, et al. Interpretation and validity of changes in scores on the graves’ ophthalmopathy quality of life questionnaire (GO-QOL) after different treatments. Clin Endocrinol (Oxf) (2001) 54(3):391–8. doi: 10.1046/j.1365-2265.2001.01241.x

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Dietrich A, Taylor P, White P, Wilson V, Uddin J, Lee RWJ, et al. Establishing the usefulness of the GO-QOL in a UK hospital-treated population with thyroid eye disease in the CIRTED trial. Psychol Health Med (2018) 23(sup1):1341–55. doi: 10.1080/13548506.2018.1503693

PubMed Abstract | CrossRef Full Text | Google Scholar

33. McMillan CV, Bradley C, Woodcock A, Razvi S, Weaver JU. Design of new questionnaires to measure quality of life and treatment satisfaction in hypothyroidism. Thyroid (2004) 14(11):916–25. doi: 10.1089/thy.2004.14.916

PubMed Abstract | CrossRef Full Text | Google Scholar

34. McMillan C, Bradley C, Razvi S, Weaver J. Evaluation of new measures of the impact of hypothyroidism on quality of life and symptoms: the ThyDQoL and ThySRQ. Value Health (2008) 11(2):285–94. doi: 10.1111/j.1524-4733.2007.00232.x

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Watt T, Bjorner JB, Groenvold M, Cramon P, Winther KH, Hegedüs L, et al. Development of a short version of the thyroid-related patient-reported outcome ThyPRO. Thyroid (2015) 25(10):1069–79. doi: 10.1089/thy.2015.0209

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Rosen CA, Lee AS, Osborne J, Zullo T, Murry T. Development and validation of the voice handicap index-10. Laryngoscope (2004) 114(9):1549–56. doi: 10.1097/00005537-200409000-00009

PubMed Abstract | CrossRef Full Text | Google Scholar

37. McHorney CA, Bricker DE, Kramer AE, Rosenbek JC, Robbins JA, Chignell KA, et al. The SWAL-QOL outcomes tool for oropharyngeal dysphagia in adults: I. conceptual foundation and item development. Dysphagia (2000) 15(3):115–21. doi: 10.1007/s004550010012

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Sorensen JR, Printz T, Iwarsson J, Grøntved ÅM, Døssing H, Hegedüs L, et al. The impact of post-thyroidectomy paresis on quality of life in patients with nodular thyroid disease. Otolaryngol Head Neck Surg (2019) 161(4):589–97. doi: 10.1177/0194599819855379

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Sorensen JR, Watt T, Cramon P, Døssing H, Hegedüs L, Bonnema SJ, et al. Quality of life after thyroidectomy in patients with nontoxic nodular goiter: A prospective cohort study. Head Neck (2017) 39(11):2232–40. doi: 10.1002/hed.24886

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Ettleson MD, Raine A, Batistuzzo A, Batista SP, McAninch E, Teixeira MCTV, et al. Brain fog in hypothyroidism: Understanding the patient’s perspective. Endocr Pract (2022) 28(3):257–64. doi: 10.1016/j.eprac.2021.12.003

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Sorensen JR, Døssing H, Watt T, Cramon P, Hegedüs L, Bonnema SJ, et al. The effect of laser thermal ablation on quality of life: Improvements in patients with solid-cystic thyroid nodules. Thyroid (2022) 32(8):917–25. doi: 10.1089/thy.2021.0665

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Grove-Laugesen D, Cramon PK, Malmstroem S, Ebbehoj E, Watt T, Hansen KW, et al. Effects of supplemental vitamin d on muscle performance and quality of life in graves’ disease: A randomized clinical trial. Thyroid (2020) 30(5):661–71. doi: 10.1089/thy.2019.0634

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Winther KH, Cramon P, Watt T, Bjorner JB, Ekholm O, Feldt-Rasmussen U, et al. Disease-specific as well as generic quality of life is widely impacted in autoimmune hypothyroidism and improves during the first six months of levothyroxine therapy. PloS One (2016) 11(6):1–12. doi: 10.1371/journal.pone.0156925

CrossRef Full Text | Google Scholar

44. Stott DJ, Rodondi N, Kearney PM, Ford I, Westendorp RG, Mooijaart SP, et al. Thyroid hormone therapy for older adults with subclinical hypothyroidism. N Engl J Med (2017) 376(26):1024. doi: 10.1056/NEJMoa1603825.

CrossRef Full Text | Google Scholar

45. Shakir MKM, Brooks DI, Mcaninch EA, Fonseca TL, Mai VQ, Bianco AC, et al. Comparative effectiveness of levothyroxine, desiccated thyroid extract, and Levothyroxine+Liothyronine in hypothyroidism. J Clin Endocrinol Metab (2021) 106(11):E4400–13. doi: 10.1210/clinem/dgab478

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Heald AH, Premawardhana L, Taylor P, Okosieme O, Bangi T, Devine H, et al. Is there a role for natural desiccated thyroid in the treatment of levothyroxine unresponsive hypothyroidism? results from a consecutive case series. Int J Clin Pract (2021) 75(12):1–9. doi: 10.1111/ijcp.14967

CrossRef Full Text | Google Scholar

47. Jonklaas J, Bianco AC, Cappola AR, Celi FS, Fliers E, Heuer H, et al. Evidence-based use of Levothyroxine/Liothyronine combinations in treating hypothyroidism: A consensus document. Eur Thyroid J (2021) 10(1):10–38. doi: 10.1159/000512970

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Jonklaas J. Optimal thyroid hormone replacement. Endocr Rev (2022) 43(2):366–404. doi: 10.1210/endrev/bnab031

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Lillevang-Johansen M, Abrahamsen B, Jørgensen HL, Brix TH, Hegedüs L. Over- and under-treatment of hypothyroidism is associated with excess mortality: A register-based cohort study. Thyroid (2018) 28(5):566–74. doi: 10.1089/thy.2017.0517

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Lillevang-Johansen M, Abrahamsen B, Jørgensen HL, Brix TH, Hegedüs L. Excess mortality in treated and untreated hyperthyroidism is related to cumulative periods of low serum TSH. J Clin Endocrinol Metab (2017) 102(7):2301–9. doi: 10.1210/jc.2017-00166

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Cramon P, Winther KH, Watt T, Bonnema SJ, Bjorner JB, Ekholm O, et al. Quality-of-Life impairments persist six months after treatment of graves’ hyperthyroidism and toxic nodular goiter: A prospective cohort study. Thyroid (2016) 26(8):1010–8. doi: 10.1089/thy.2016.0044

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Basch E, Deal AM, Dueck AC, Scher HI, Kris MG, Hudis C, et al. Overall survival results of a trial assessing patient-reported outcomes for symptom monitoring during routine cancer treatment. JAMA - J Am Med Assoc (2017) 318(2):197–8. doi: 10.1001/jama.2017.7156

CrossRef Full Text | Google Scholar

53. Short D, Fredericksen RJ, Crane HM, Fitzsimmons E, Suri S, Bacon J, et al. Utility and impact of the implementation of same-day, self-administered electronic patient-reported outcomes assessments in routine HIV care in two north American clinics. AIDS Behav (2022) 26(7):2409–24. doi: 10.1007/s10461-022-03585-w

PubMed Abstract | CrossRef Full Text | Google Scholar

54. Mistry J, Sharif M, Prideaux A, Smith C, Sumbwanyambe M, Sibley M, et al. Use of rheumatoid arthritis impact of disease (RAID) in routine care; identification of DAS28 remission and unmet patient-reported outcomes. Rheumatol Adv Pract (2020) 4(2):1–6. doi: 10.1093/rap/rkaa013

CrossRef Full Text | Google Scholar

55. Peterson SJ, Cappola AR, Castro MR, Dayan CM, Farwell AP, Hennessey JV, et al. An online survey of hypothyroid patients demonstrates prominent dissatisfaction. Thyroid (2018) 28(6):707–21. doi: 10.1089/thy.2017.0681

PubMed Abstract | CrossRef Full Text | Google Scholar

56. Nexø MA, Watt T, Cleal B, Hegedüs L, Bonnema SJ, Rasmussen ÅK, et al. Exploring the experiences of people with hypo- and hyperthyroidism. Qual Health Res (2015) 25(7):945–53. doi: 10.1177/1049732314554093

PubMed Abstract | CrossRef Full Text | Google Scholar

57. Greenhalgh J, Pawson R, Wright J, Black N, Valderas JM, Meads D, et al. Functionality and feedback: a protocol for a realist synthesis of the collation, interpretation and utilisation of PROMs data to improve patient care. BMJ Open (2014) 4(7):1–7. doi: 10.1136/bmjopen-2014-005601

CrossRef Full Text | Google Scholar

58. Montgomery N, Howell D, Ismail Z, Bartlett SJ, Brundage M, Bryant-Lukosius D, et al. Selecting, implementing and evaluating patient-reported outcome measures for routine clinical use in cancer: the cancer care Ontario approach. J Patient-Reported outcomes (2020) 4(1):1–9. doi: 10.1186/s41687-020-00270-1

CrossRef Full Text | Google Scholar

59. Gaissmaier W, Wegwarth O, Skopec D, Müller AS, Broschinski S, Politi MC. Numbers can be worth a thousand pictures: individual differences in understanding graphical and numerical representations of health-related information. Health Psychol (2012) 31(3):286–96. doi: 10.1037/a0024850

PubMed Abstract | CrossRef Full Text | Google Scholar

60. Bantug ET, Coles T, Smith KC, Snyder CF, Rouette J, Brundage MD. Graphical displays of patient-reported outcomes (PRO) for use in clinical practice: What makes a pro picture worth a thousand words? Patient Educ Couns (2016) 99(4):483–90. doi: 10.1016/j.pec.2015.10.027

PubMed Abstract | CrossRef Full Text | Google Scholar

61. Jensen RE, Bjorner JB. Applying PRO reference values to communicate clinically relevant information at the point-of-care. Med Care (2019) 57:S24–30. doi: 10.1097/MLR.0000000000001113

PubMed Abstract | CrossRef Full Text | Google Scholar

62. Snyder C, Smith K, Holzner B, Rivera YM, Bantug E, Brundage M, et al. Making a picture worth a thousand numbers: recommendations for graphically displaying patient-reported outcomes data. Qual Life Res (2019) 28(2):345–56. doi: 10.1007/s11136-018-2020-3

PubMed Abstract | CrossRef Full Text | Google Scholar

63. Kroenke K, Monahan PO, Kean J. Pragmatic characteristics of patient-reported outcome measures are important for use in clinical practice. J Clin Epidemiol (2015) 68(9):1085–92. doi: 10.1016/j.jclinepi.2015.03.023

PubMed Abstract | CrossRef Full Text | Google Scholar

64. Takeuchi EE, Keding A, Awad N, Hofmann U, Campbell LJ, Selby PJ, et al. Impact of patient-reported outcomes in oncology: a longitudinal analysis of patient-physician communication. J Clin Oncol (2011) 29(21):2910–7. doi: 10.1200/JCO.2010.32.2453

PubMed Abstract | CrossRef Full Text | Google Scholar

65. Hughes EF, Wu AW, Carducci MA, Snyder CF. What can I do? recommendations for responding to issues identified by patient-reported outcomes assessments used in clinical practice. J Support Oncol (2012) 10(4):143–8. doi: 10.1016/j.suponc.2012.02.002

PubMed Abstract | CrossRef Full Text | Google Scholar

66. Schougaard LMV, Larsen LP, Jessen A, Sidenius P, Dorflinger L, de Thurah A, et al. AmbuFlex: tele-patient-reported outcomes (telePRO) as the basis for follow-up in chronic and malignant diseases. Qual Life Res (2016) 25(3):525–34. doi: 10.1007/s11136-015-1207-0

PubMed Abstract | CrossRef Full Text | Google Scholar

67. Grove BE, Ivarsen P, De Thurah A, Schougaard LM, Kyte D, Hjøllund NH. Remote follow-up using patient-reported outcome measures in patients with chronic kidney disease: the PROKID study - study protocol for a non-inferiority pragmatic randomised controlled trial. BMC Health Serv Res (2019) 19(1):1–12. doi: 10.1186/s12913-019-4461-y

PubMed Abstract | CrossRef Full Text | Google Scholar

68. Mejdahl CT, Schougaard LMV, Hjollund NH, Riiskjær E, Lomborg K. Patient-reported outcome measures in the interaction between patient and clinician - a multi-perspective qualitative study. J patient-reported outcomes (2020) 4(1):1–10. doi: 10.1186/s41687-019-0170-x

CrossRef Full Text | Google Scholar

69. Black N. Patient reported outcome measures could help transform healthcare. BMJ (2013) 346(7896):1–5. doi: 10.1136/bmj.f167

CrossRef Full Text | Google Scholar

70. Shiffman S, Stone AA, Hufford MR. Ecological momentary assessment. Annu Rev Clin Psychol (2008) 4:1–32. doi: 10.1146/annurev.clinpsy.3.022806.091415

PubMed Abstract | CrossRef Full Text | Google Scholar

71. Boesen VB, Nissen SB, Groenvold M, Bjorner JB, Hegedüs L, Bonnema SJ, et al. Conversion of standard retrospective patient-reported outcomes to momentary versions: cognitive interviewing reveals varying degrees of momentary compatibility. Qual Life Res (2018) 27(4):1065–76. doi: 10.1007/s11136-017-1762-7

PubMed Abstract | CrossRef Full Text | Google Scholar

72. Boesen VB, Feldt-Rasmussen U, Bjorner JB, Cramon P, Groenvold M, Nygaard B, et al. How should thyroid-related quality of life be assessed? recalled patient-reported outcomes compared to here-and-Now measures. Thyroid (2018) 28(12):1561–70. doi: 10.1089/thy.2018.0210

PubMed Abstract | CrossRef Full Text | Google Scholar

73. Boesen VB, Feldt-Rasmussen U, Bjorner JB, Cramon PK, Grøenvold M, Rasmussen ÅK, et al. Shorter recall period for the thyroid-related patient-reported outcome measure ThyPRO did not change the accuracy as evaluated by repeated momentary measurements. Thyroid (2020) 30(2):185–91. doi: 10.1089/thy.2019.0380

PubMed Abstract | CrossRef Full Text | Google Scholar

74. Boesen VB, Bjorner JB, Groenvold M, Cramon PK, Rasmussen AK, Watt T, et al. Ecological momentary assessments (EMAs) did not improve responsiveness of patient-reported outcomes on quality of life. J Clin Epidemiol (2020) 125:138–47. doi: 10.1016/j.jclinepi.2020.06.006

PubMed Abstract | CrossRef Full Text | Google Scholar

75. Stone AA, Broderick JE, Schwartz JE, Shiffman S, Litcher-Kelly L, Calvanese P. Intensive momentary reporting of pain with an electronic diary: Reactivity, compliance, and patient satisfaction. Pain (2003) 104(1–2):343–51. doi: 10.1016/S0304-3959(03)00040-X

PubMed Abstract | CrossRef Full Text | Google Scholar