Skip to main content

GENERAL COMMENTARY article

Front. Pharmacol., 02 June 2020
Sec. Respiratory Pharmacology

Commentary: “Healthcare Professionals’ Preferred Efficacy Endpoints and Minimal Clinically Important Differences in the Assessment of New Medicines for Chronic Obstructive Pulmonary Disease” by Dankers M et al. in Frontiers in Pharmacology 2020; 10: 1519

Harma Alma,*Harma Alma1,2*Corina de Jong,Corina de Jong1,2Janwillem Kocks,,Janwillem Kocks2,3,4Thys van der Molen,Thys van der Molen1,2
  • 1Department of General Practice and Elderly Care Medicine, University of Groningen, University Medical Center Groningen, Groningen, Netherlands
  • 2GRIAC Research Institute, University of Groningen, University Medical Center Groningen, Groningen, Netherlands
  • 3General Practitioners Research Institute, Groningen, Netherlands
  • 4Observational and Pragmatic Research Institute, Singapore, Singapore

A Commentary on
Healthcare Professionals' Preferred Efficacy Endpoints and Minimal Clinically Important Differences in the Assessment of New Medicines for Chronic Obstructive Pulmonary Disease

by Dankers M, Nelissen-Vrancken MHJMG, Surminski SMK, Lambooij AC, Schermer TR and van Dijk L. Front Pharmacol (2020) 10:1519. doi: 10.3389/fphar.2019.01519

Introduction

Pharmacological clinical trials require obligatory endpoints in evaluating the clinical relevance of their outcomes (Jones, 2001; US Department of Health and Human Services FDA Center for Drug Evaluation and Research, 2006). The minimal clinically important difference (MCID) is an important parameter that represents the threshold of an outcome measure at which the observed change can be considered clinically relevant for the patient, hence justifying the therapy (Jaeschke et al., 1989). Frequently applied endpoints in clinical trials for patients with chronic obstructive pulmonary disease (COPD) include spirometry with forced expiratory volume in one second (FEV1), exacerbations, and health status (US Department of Health and Human Services FDA Center for Drug Evaluation and Research, 2006; Jones et al., 2012; Cazzola et al., 2015).

The most frequently selected health status instrument in scientific research is the St. George’s Respiratory Questionnaire (SGRQ) with a proclaimed MCID of four units on a total scale from 0 to 100 (Jones et al., 1991; US Department of Health and Human Services FDA Center for Drug Evaluation and Research, 2006; Jones et al., 2012). Higher scores represent worse health status. MCIDs can be determined using anchor-, distribution-, or opinion-based methods (Copay et al., 2007). It is with great interest that Dankers et al. have investigated healthcare professionals’ opinion on among others the MCID threshold of the SGRQ. To our knowledge, this is the first study to investigate these expert-ratings on the MCID of the SGRQ. The authors’ conclusion was that healthcare professionals prefer a higher cutoff value for clinical relevance of 11 on the SGRQ instead of the currently used MCID of four by registration authorities.

Redefining the MCID of the SGRQ

We agree with Dankers et al. that the MCID of the SGRQ should in fact be higher than four points, and that current clinical trials hence may have overestimated the interpretation of treatment effects. In general, MCIDs of instruments are approximately 7–10% of the maximum scale score (Copay et al., 2007). This would represent a change of 7–10 points on the SGRQ. In a published systematic review in the European Respiratory Journal, Alma et al. (2018b) reviewed the available content on the MCID of COPD health status tools including the SGRQ (Alma et al., 2018b). A meta-analysis by means of triangulation procedures was performed and resulted in weighted MCID cutoff values of −7.43 (4 studies, range −10.19 to −2.40) for the SGRQ; −2.54 (6 studies, range −3.80 to −1.00) for the COPD Assessment Test (CAT); and −0.43 (5 studies, range −0.62 to −0.21) for the Clinical COPD Questionnaire (CCQ).

Moreover, multiple publications emerged over the past few years on the MCID of the SGRQ, proclaiming that this threshold should in fact be higher than four units (Welling et al., 2015; Alma et al., 2016; Alma et al., 2018a; Alma H.J. et al., 2019; Alma H. et al., 2019). Figure 1 summarizes the evidence from these publications. The majority of estimates is in the range of five to nine points depending on the study setting as well as the direction of the measured change (improvement versus deterioration). Only MCID estimates from routine clinical practice come somewhat close (range three to six points) to the proclaimed MCID of four units (Jones, 2005).

FIGURE 1
www.frontiersin.org

Figure 1 Plot of minimal clinically important difference (MCID) estimates for the St. George’s Respiratory Questionnaire (SGRQ) from recent publications and systematic review. Overview of MCID estimates for improvement anddeterioration (horizontal axis) from intervention/PR versus RCP (vertical axis). Estimates are presented as circles (anchor-based estimates), squares (distribution-based estimates), diamonds (weighted or triangulated MCIDs), or asterisks (lower and upper limit of the observed range or 95% CI). The vertical orange reference line represented the currently accepted MCID in the literature. The dashed vertical orange reference line mirrors the accepted MCID for improvement into deterioration. MCID estimates are color coded per publication: systematic review and triangulation (black) (Alma et al., 2018b); various MCID methods (red) (Alma et al., 2016); follow-up period and anchor question (green) (Alma et al., 2018a); improvement versus deterioration (blue) (Alma H.J. et al., 2019); MCID and dynamic factors (orange) (Alma H. et al., 2019).

We would like to highlight that experts’ opinions on the MCID of the SGRQ are of interest as they convey a message of clinical judgment; however there is clear evidence that ratings by physicians and patients on quality of life and MCIDs are not in line (Janse et al., 2004). What relevance does an expert-based MCID of 11 units have, if it does not match with the experienced change by the patient with COPD? Especially since the SGRQ is defined as a patient-reported outcome (PRO). It is also remarkable that experts in the Netherlands were asked about a questionnaire like the SGRQ, without being very familiar with it. Ratings were based upon a short description of the questionnaire and its total scoring range. On what was the respondent’s judgment based? Was it just a gut-feeling or was there actually rationale for? Since it is known that Dutch healthcare professionals have more experience with the CCQ (Van der Molen et al., 2003; Snoeck-Stroband et al., 2015), one would think ratings for this clinically applied tool, would be more reliable than for an instrument that is infrequently used. It would make more sense to formulate a threshold for something complicated as an MCID, once the professional has knowledge about its content and clinical application.

It is also interesting to have a closer look at the number of responders in this analysis, since this was less than 3% of the group of invited professionals (227 replied out of 7,731 invitations). Is this representative and was this in fact the group of professionals most frequently working with patients with COPD and the SGRQ? Especially since approximately half of this group was a pharmacist, rather than a physician or practice nurse. Although pharmacists play a key role in the support of patients with COPD, they do not work directly with health status instruments in their day-to-day practice. This inexperience may also be reflected in the large variation of responses by the healthcare professionals, because the standard deviation was 10.1 out of the rated overall MCID of the SGRQ of 11 units.

Last, but not least it is unknown if the professionals reviewed thresholds at the individual level or at the group level (Beaton et al., 2001; Kocks et al., 2010). Physicians and healthcare professionals meet the individual patient in clinical practice, possibly requiring more change after intervention. In scientific research, conclusions are drawn based on reaching the MCID threshold at the group level. Extreme high and low change scores are usually integrated in the regression to the mean phenomenon.

Discussion

Dankers et al. have an important share in continuing the discussion on the MCID of the SGRQ. Experts’ opinions on the threshold of the SGRQ could be valuable, yet should not be leading. It is very remarkable that despite new studies on the MCID of the SGRQ, the threshold of four points (Jones, 2005)—based upon unclear methodology—is still applied in scientific research. If we continue doing so, trial outcomes could be severely overestimated in its interpretation. Costly therapies may be offered to patients without a valid basis. Possibly with some good individual responses, but at the group level they may not meet sufficiently clinically relevant improvements. The newer health status instruments (like the CCQ and CAT) have shown better effect sizes and responsiveness, and their MCID is more rigorously determined. Integrating these measures in future studies may lead to better assessment of treatment response than continuing using the SGRQ.

Author Contributions

HA wrote the first draft. CJ, JK, and TM reviewed and finalized the manuscript.

Funding

Junior Scientific Masterclass of the University of Groningen.

Conflict of Interest

TM holds the copyright of the Clinical COPD Questionnaire.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Abbreviations

95%CI, 95% confidence interval; BLVR, bronchoscopic lung volume reduction; MCID, minimal clinically important difference; PR, pulmonary rehabilitation; RCP, routine clinical practice; SGRQ, St. George’s Respiratory Questionnaire.

References

Alma, H., de Jong, C., Jelusic, D., Wittmann, M., Schuler, M., Flokstra-de Blok, B., et al. (2016). T. Health status instruments for patients with COPD in pulmonary rehabilitation: defining a minimal clinically important difference. Prima. Care Respirat. Med. 26, 16041. doi: 10.1038/npjpcrm.2016.41

CrossRef Full Text | Google Scholar

Alma, H., de Jong, C., Tsiligianni, I., Sanderman, R., Kocks, J. W. H., van der Molen, T. (2018a). Assessing health status over time: impact of recall period and anchor question on the minimal clinically important difference of copd health status tools. Health Qual. Life Outcomes 16 (1), 130. doi: 10.1186/s12955-018-0950-7

PubMed Abstract | CrossRef Full Text | Google Scholar

Alma, H., de Jong, C., Jelusic, D., Wittmann, M., Schuler, M., Kollen, B., et al. (2018b). Clinically relevant differences in COPD health status: systematic review and triangulation. Eur. Respirat. J. 52 (3), 1800412. doi: 10.1183/13993003.00412-2018

CrossRef Full Text | Google Scholar

Alma, H. J., de Jong, C., Jelusic, D., Wittmann, M., Schuler, M., Sanderman, R., et al. (2019). Thresholds for clinically important deterioration versus improvement in COPD health status: results from a randomized controlled trial in pulmonary rehabilitation and an observational study during routine clinical practice. BMJ Open 9 (6), e025776. doi: 10.1136/bmjopen-2018-025776

PubMed Abstract | CrossRef Full Text | Google Scholar

Alma, H., de Jong, C., Jelusic, D., Wittmann, M., Schuler, M., Kollen, B., et al. (2019). Baseline health status and setting impacted minimal clinically important differences in COPD: an exploratory study. J. Clin. Epidemiol. 116, 49–61. doi: 10.1016/j.jclinepi.2019.07.015

PubMed Abstract | CrossRef Full Text | Google Scholar

Beaton, D. E., Bombardier, C., Katz JN and Wright, J. G. (2001). A taxonomy for responsiveness. J. Clin. Epidemiol. 54 (12), 1204–1217. doi: 10.1016/S0895-4356(01)00407-3

PubMed Abstract | CrossRef Full Text | Google Scholar

Cazzola, M., Hanania, N. A., MacNee, W., Rüdell, K., Hackford, C., Tamimi, N. (2015). A review of the most common patient-reported outcomes in COPD – revisiting current knowledge and estimating future challenges. Int. J. COPD 10, 725–738. doi: 10.2147/COPD.S77368

CrossRef Full Text | Google Scholar

Copay, A. G., Subach, B. R., Glassman, S. D., Polly, D. W., Jr., Schuler, T. C. (2007). Understanding the minimum clinically important difference: a review of concepts and methods. Spine J. 7 (5), 541–546. doi: 10.1016/j.spinee.2007.01.008

PubMed Abstract | CrossRef Full Text | Google Scholar

Jaeschke, R., Singer, J., Guyatt, G. H. (1989). Measurement of health status. Ascertaining the Minimal Clinically Important Difference. Controlled Clin. Trials 10 (4), 407–415. doi: 10.1016/0197-2456(89)90005-6

CrossRef Full Text | Google Scholar

Janse, A. J., Gemke, R. J. B. J., Uiterwaal, C. S. P. M., Van der Tweel, I., Kimpen, J. L. L., Sinnema, G. (2004). Quality of life: patients and doctors don’t always agree: a meta-analysis. J. Clin. Epidemiol. 57 (7), 653–661. doi: 10.1016/j.jclinepi.2003.11.013

PubMed Abstract | CrossRef Full Text | Google Scholar

Jones, P. W., Quirck, F. H., Baveystock, C. M. (1991). The St George’s Respiratory Questionnaire. Respirat. Med. 85 (Suppl B), 25–31. doi: 10.1016/S0954-6111(06)80166-6

CrossRef Full Text | Google Scholar

Jones, P., Miravittles, M., van der Molen, T., Kulich, K. (2012). Beyond FEV1 in COPD: a review of patient-reported outcomes and their measurement. Int. J. COPD 7, 697–709. doi: 10.2147/COPD.S32675

CrossRef Full Text | Google Scholar

Jones, P. W. (2001). Health status measurement in chronic obstructive pulmonary disease. Thorax 56 (11), 880–887. doi: 10.1136/thorax.56.11.880

PubMed Abstract | CrossRef Full Text | Google Scholar

Jones, P. W. (2005). St. George’s respiratory questionnaire: MCID. COPD 2 (1), 75–79. doi: 10.1081/copd-200050513

PubMed Abstract | CrossRef Full Text | Google Scholar

Kocks, J. W., Kerstjens, H. A., Snijders, S. L., de Vos, B., Biermann, J. J., van Hengel, P., et al. (2010). Health status in routine clinical practice: validity of the clinical COPD questionnaire at the individual patient level. Health Qual. Life Outcomes 8, 135. doi: 10.1186/1477-7525-8-135

PubMed Abstract | CrossRef Full Text | Google Scholar

Snoeck-Stroband, J. B., Schermer, T. R. J., Van Schayck, C. P., Muris, J. W., Van der Molen, T., In ‘t Veen, J. C. C. M., et al. (2015). NHG-Werkgroep Astma bij volwassenen en COPD. NHG-Standaard COPD (derde herziening). Huisarts Wet. 58 (4), 198–211.

Google Scholar

US Department of Health and Human Services FDA Center for Drug Evaluation and Research (2006). Guidance for industry: patient-reported outcome measures: use in medical product development to support labelling claims: draft guidance. Health Qual. Life Outcomes 4, 79. doi: 10.1186/1477-7525-4-79

PubMed Abstract | CrossRef Full Text | Google Scholar

Van der Molen, T., Willemse, B. W. M., Schokker, S., ten Hacken, N. H. T., Postma, D. S., Juniper, E. F. (2003). Development, validity and responsiveness of the Clinical COPD Questionnaire. Health Qual. Life Outcomes 1, 13. doi: 10.1186/1477-7525-1-13

PubMed Abstract | CrossRef Full Text | Google Scholar

Welling, J. B., Hartman, J. E., Ten Hacken, N. H., Klooster, K., Slebos, D. (2015). The minimal important difference for the St George’s Respiratory Questionnaire in patients with severe COPD. Eur. Respirat. J. 46 (6), 1598–1604. doi: 10.1183/13993003.00535-2015

CrossRef Full Text | Google Scholar

Keywords: health status, COPD, (MCID) minimal clinically important difference, St. George’s Respiratory Questionnaire, patient-reported outcomes

Citation: Alma H, de Jong C, Kocks J and van der Molen T (2020) Commentary: “Healthcare Professionals’ Preferred Efficacy Endpoints and Minimal Clinically Important Differences in the Assessment of New Medicines for Chronic Obstructive Pulmonary Disease” by Dankers M et al. in Frontiers in Pharmacology 2020; 10: 1519. Front. Pharmacol. 11:827. doi: 10.3389/fphar.2020.00827

Received: 29 March 2020; Accepted: 19 May 2020;
Published: 02 June 2020.

Edited by:

Mauro Maniscalco, Fondazione Salvatore Maugeri (IRCCS), Italy

Reviewed by:

Alessandro Vatrella, University of Salerno, Italy
Antonio Molino, University of Naples Federico II, Italy

Copyright © 2020 Alma, de Jong, Kocks and van der Molen. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Harma Alma, h.j.alma@umcg.nl

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.