Henry Tseng
Corey Stone1,2†
Dédée F. Murrell- 1Department of Dermatology, St. George Hospital, Sydney, NSW, Australia
- 2Faculty of Medicine, University of New South Wales, Sydney, NSW, Australia
Autoimmune blistering diseases (AIBDs) comprise a group of rare conditions marked by autoantibodies that specifically target intercellular adhesion molecules. Despite the progress made in comprehending the disease and the increasing number of treatment options available, there is still no definitive cure for AIBDs such as pemphigus, and it continues to have a devastating impact on those affected. The challenges in achieving new approved therapies for AIBDs are complex and multifaceted. One significant obstacle was the prior lack of validated and standardized outcome measures, which are crucial for ensuring precise comparisons between new and traditional therapies. This gap in knowledge has prompted the development of minimal clinically important differences (MCIDs), which enable efficient and reliable comparison of therapeutic outcomes between trials. MCID is defined as the minimum difference in an outcome measure that indicates a clinically significant improvement/deterioration in disease severity. Additionally, MCIDs provide a patient-centered approach to evaluating treatment efficacy, by considering whether patients experience a subjective improvement in their symptoms. Therefore, this literature review will examine the derivation and significance of MCIDs for various scoring systems in AIBDs.
1 Introduction
This literature review will discuss the importance and derivation of MCIDs of various scoring systems in AIBD. The purpose is to understand the various methods which have been used to calculate MCIDs in AIBD and to recommend strategies to improve the reliability of MCID calculations.
2 Background
Autoimmune blistering diseases (AIBDs) encompass a group of rare conditions that result in the formation of blisters on the skin and mucous membranes, with examples including pemphigus and pemphigoid. The severity of AIBDs can be attributed to the fact that the rupturing of its blisters creates an opening for infections to opportunistically invade, putting the patient at increased risk of sepsis and ultimately death. This substantial infection risk is underscored by the high mortality rate of pemphigus, a type of AIBD, amounting to 1.7 to 3 times that of the general population (1). In addition to its poor outcomes, there is a shortage of international clinical trials exploring novel treatments for pemphigus that have the potential to significantly enhance the disease and the quality of life (QOL) of affected patients (2). This inadequacy can be attributed to the absence of validated and standardized outcome measures that are essential for making accurate comparisons between new and traditional therapies. To address this issue, minimal clinically important differences (MCIDs) of such outcome measures have been developed and used to allow for efficient comparison of therapeutic outcomes between trials. MCIDs not only facilitate comparison, but also provide a patient-centric approach to evaluating treatment efficacy, by taking into account whether the patient subjectively experiences an improvement in their symptoms (3).
3 Scoring systems for autoimmune blistering diseases
In medicine, the ability to accurately assess disease severity is often hindered by the lack of consensus on the best scoring systems to carry out such an assessment, which can lead to inaccurate assessments of patient treatment outcome and efficacy. This is especially true for AIBDs such as pemphigus before the development of its current scoring systems, as a systematic review conducted in 2006 highlighted the lack of universally recognized outcome measures as the contributing factor to the shortage of high-quality clinical trials assessing treatments for pemphigus (4). As a response to this issue, new scoring systems for the disease have since been developed and validated, including the two most-commonly used scoring systems in pemphigus: the Pemphigus Disease Activity Index (PDAI) and Autoimmune Bullous Skin Disorder Intensity Score (ABSIS) (5, 6).
Although assessment tools are valuable in dermatology, it is important to recognize that most of these clinician-reported outcome measures are designed to detect only small differences in severity. However, such minimal differences may not be significant enough to impact patients’ quality of life or measurable disease burden. Therefore, the concept of the MCID was created, which refers to the smallest difference in an outcome measure that reflects a clinically significant improvement in disease severity (7). Values above the MCID are used in clinical studies to show that a novel intervention is helpful for patients, which can be used in clinical practice to direct management.
3.1 How MCIDs are calculated
There are various different methods of calculating MCID which are summarized below in Table 1; however, by far the most common method is the anchor-based method because it directly takes into account the patients’ preferences and values (2). The anchor-based method compares score changes on a scoring system to an external “anchor,” which can be subjective patient-reported outcomes (e.g., QOL) or physician assessments, depending on the study context and outcome measure used (12).
Table 1 Methods used to calculate MCID: summary overview.
Once the anchor’s threshold of clinical significance is established, the MCID is then calculated as the smallest change in the scoring system that corresponds to a meaningful change in clinical improvement indicated by the anchor (12).
4 Previous MCIDs of scoring systems for autoimmune blistering diseases
The following section will explore the variations of methods that previous studies have employed to calculate MCIDs of AIBD.
4.1 PDAI
The PDAI is a scoring system developed by the International Pemphigus Definitions Committee over a period of three years after 2006, to evaluate the extent of pemphigus disease with a potential score ranging from 0-263 (5). It assesses activity and damage associated with the skin, scalp, and mucous membranes. The activity score is determined based on the numbers and sizes of erosions, blisters, or erythema observed during evaluation for each of the 12 anatomic locations. On the other hand, its damage score is determined by the presence (1 point) or absence (0 points) of post-inflammatory hyperpigmentation or erythema on resolving lesions on the skin and scalp (13).
To date, there have been no published studies on calculating the MCIDs for PDAI. The absence of this knowledge is likely because calculating MCID requires data on longitudinal changes in clinical scores in response to treatment from a reasonable sample size, which is difficult to collect and analyze for a disease as rare and severe as pemphigus (14). Determining the MCID for PDAI scores would offer valuable insights to clinicians and patients about the clinical significance of changes in PDAI scores over time. This knowledge can assist in guiding treatment decisions, providing realistic expectations for patients, and ultimately improving the care and quality of life of pemphigus patients.
4.2 ABSIS
The ABSIS is a scoring system that was initially developed for evaluating pemphigus but has since been widely used for all autoimmune blistering diseases following its introduction in 2007 (14, 15). The ABSIS has a possible score from 0-206 and assesses factors such as the patient’s body surface area affected by pemphigus, the location of the lesions on the skin and mucosa, as well as discomfort when eating or drinking designed to reflect the level of mucosal involvement (15). Furthermore, in contrast to the PDAI, the ABSIS incorporates the damage items into the overall score instead of treating them as individual components (16). Nevertheless, several multicenter studies have shown that the PDAI has higher interrater reliability than the ABSIS. For example, in a 2009 study conducted in the United States, the PDAI skin activity had an ICC of 0.86 compared to 0.39 for the ABSIS, demonstrating the superior reliability of the PDAI in comparison (17). In a 2012 study with 100 pemphigus vulgaris patients in Iran, the PDAI had the highest interrater reliability (ICC=0.98) among the scoring systems, followed by ABSIS (ICC=0.97) (18). The PDAI also correlated more strongly with disease extent and anti-desmoglein antibody levels, making it a more reliable tool for assessing pemphigus severity, especially in cases with variable cutaneous disease. In a recent multicenter international study with 116 pemphigus patients over 24 months (2019), both the PDAI and ABSIS showed strong interrater reliability at the beginning of the study. The PDAI had higher ICCs in moderate and extensive cases, while the ABSIS performed better in intermediate and extensive cases (16). Overall, such findings underscore that both the PDAI and ABSIS are valuable assessment tools for measuring pemphigus activity and suggest a preference for using the PDAI in multicenter studies.
There has been one study that calculated MCIDs for ABSIS led by Wijayanti et al. in 2017 (n=27) for bullous pemphigoid (14). The study used the anchor-based method by employing the receiver-operating characteristic (ROC) curve to calculate MCID values of BPDAI anchored against the Physician’s Subjective Assessment of Clinical Improvement (PSACI), a classification system that classifies disease activity as improved, stable, or deteriorated (2). Wijayanti et al. concluded that a 4.75-point decrease in BPDAI scores indicated clinical improvement (MCID=4.75), whereas a 4-point increase indicated clinical deterioration (MCID=4) (14). However, a significant limitation is its small sample size (n=27) which negatively affected the reliability and generalizability of its calculated MCID value due to the reduction in statistical power of analysis. In addition, the chosen anchor in the study assessed the severity of pemphigus from the physician’s rather than the patient’s perspective, which may not be reflective of how meaningful the change in severity is to the patient. Broadly classifying patients as clinically improved/stable/deteriorated is a relatively narrow categorization and may not capture the full range of disease activity/severity.
4.3 EBDASI
The Epidermolysis Bullosa Disease Activity and Scarring Index (EBDASI) is a scoring system that evaluates epidermolysis bullosa disease activity and damage separately in each of its five sections: skin, scalp, mucosa, nails, and other epithelialized surfaces. The skin section is the most comprehensive and includes 12 anatomical sites. The algorithm assigns a combined aggregate score of 506, with 276 for total activity and 230 for damage (19). The EBDASI has shown excellent reliability and validity as a scoring system for epidermolysis bullosa, as confirmed by its consistent performance in two separate Australian studies (19, 20).
There has been one study led by Jain et al. in 2016 (n=29) that likewise used the anchor-based method and ROC analyses to calculate MCIDs for EBDASI by anchoring against the 15-point Likert scale of change (21). A criterion of 3 on the Likert scale was chosen as the threshold for clinical significance, in accordance with various other MCID studies that have used the same established method. In their results, Jain et al. reported that a decrease of 7 or more points in the activity score indicated clinical improvement (MCID=7), while an increase of 3 or more points indicated clinical deterioration (MCID=3) (21). This study is similarly limited by its small sample size, although the anchor used provided broader categorization of disease activity across 15 points on the Likert scale of change.
4.4 BPDAI
The Bullous Pemphigoid Disease Area Index (BPDAI) is a scoring system developed in 2007 for bullous pemphigoid and is scored from 0-360 based on four components: body surface area percentage involved, peak pruritus numerical rating scale (NRS) score, disease extent index, and area index (22). The pruritus component is considered as a subjective aspect of the BPDAI and is evaluated separately from the scores of the other three components (23). In addition, a recent multicenter study in Europe (2021) reaffirmed the reliability of the BPDAI as a robust tool for assessing bullous pemphigoid (BP) severity, which demonstrated a high baseline intraclass correlation coefficient (ICC=0.97) that remained stable up to month 6 (22). The study involved 285 bullous pemphigoid patients from 50 dermatology departments and established cut-off values for categorizing mild, moderate, and severe bullous pemphigoid cases based on BPDAI score percentiles. The calculated cut-off values were 20, 57, and above 57, respectively. Furthermore, the improvement in BPDAI scores correlated with a decrease in anti-BP180 antibodies but not with anti-BP230 antibodies (22). These findings highlight the BPDAI’s precision in assessing BP severity and provide valuable clinical classification cut-off values.
In Wijayanti et al.’s same study in 2017 which also calculated MCIDs for ABSIS, Wijayanti et al. concluded that a 4-point decrease in BPDAI scores indicated clinical improvement (MCID=4), whereas a 3-point decrease indicated clinical deterioration (MCID=3) (14). In addition, Wijayanti et al. also concluded that the BPDAI was a reliable scoring system in assessing bullous pemphigoid severity, demonstrating both strong interrater and intrarater reliability (14).
4.5 IGA
The investigator global assessment (IGA) for pemphigus is a new scoring system that is still in the process of being developed and validated by a previous study as of this writing.
The IGA is scored separately from 0-4 (0=clear; 4=severe) for mucosal and cutaneous lesions, making it more simplistic than PDAI and ABSIS scores. The U.S. Food and Drug Administration (FDA) prefers to use the IGA to assess the primary endpoint of clinical drug trials over the PDAI and ABSIS because the IGA provides a more simplistic and clinically meaningful ordinal score that patients can easily understand (24). Developing and calculating the MCID of a validated IGA for pemphigus will thus aid clinical trials and development of new treatments for pemphigus.
4.6 MCIDs of other commonly used scoring systems in dermatology
The variation in methods which have been used to calculate MCIDs highlighted above underscores the inconsistency of calculating MCIDs of scoring systems for AIBD. This appears to be a common theme for many other scoring systems in dermatology (3). The following table (Supplementary Table 1) compares the various methods used to calculate MCID of commonly used dermatological scoring systems and their advantages and disadvantages.
4.7 The credibility of MCID calculations: an analysis
The inconsistency in calculating MCIDs is evident from the various methods used, as highlighted in the discussion so far. The absence of international guidelines for calculating MCIDs leads to varying sample sizes in studies, which are often on the low end for rarer and more severe diseases like pemphigus. This has negative consequences on the generalizability of MCIDs. It is known that the MCID may exhibit variability based on differences in population and context, and to address this issue, it is imperative to accept that MCIDs can and should be re-calculated should the opportunity arise for sample sizes to be expanded (25, 26).
In addition, from the examples above, it is evident that MCIDs vary with the choice of anchor. Given that MCIDs reflect the level of improvement from the patient’s perspective, it appears to be logical that anchoring to a score that patients give themselves, such as QOL scores, would be more favorable than a physician’s score such as the 15-point Likert scale of change. However, in as much as 41.4% of MCID calculations in dermatology, the anchor was rated by the physician, though there are advantages and disadvantages to both (3). One advantage to having an anchor scored by a physician is ensuring that the MCID calculated reflects an objective improvement in patient’s condition particularly when assessing disease activity, but a disadvantage is the inability of the MCID to reflect a change felt by the patient. It can be argued that having a physician-reported anchor is better suited for severe and potentially life-threatening conditions such as pemphigus as any small change in disease activity would be significant to the patient’s ongoing management more so than whether the patient feels he/she has improved. In addition, patient perception of their level of improvement in disease severity may be skewed by other, unrelated factors such as treatment side effects (27). These factors could potentially bias results of the calculation of MCIDs should patient-reported anchors be used.
4.8 Implications for future MCID calculations
Overall, the inconsistency of MCID calculations identified arises from the use of various methods and anchors in different studies. The absence of standardized guidelines leads to varying sample sizes, particularly in rare and severe diseases, which affects the generalizability of MCIDs.
To address this issue, it is crucial to recognize that MCIDs can and should be recalculated when opportunities arise to expand sample sizes. To improve the credibility of MCID calculations in dermatology, it is recommended to establish international guidelines that provide standardized methods for calculating MCIDs. These guidelines should address issues such as sample size determination and the selection of appropriate anchors based on the specific disease, its severity, and objectives of the studies. The selection of appropriate anchors for MCID calculations should not only consider the specific disease and its severity but also take into account the objectives of the studies. Different objectives may require different types of anchors to be used in calculating MCIDs. For instance, studies aiming to assess the impact of treatments on disease severity may use patient-reported anchors, whereas studies focusing on clinical significance may opt for physician-reported anchors. By acknowledging the influence of study objectives on anchor selection, researchers can ensure that MCID calculations align more closely with the intended applications of the study results. Furthermore, collaboration among researchers and clinicians can facilitate the pooling of data and resources to achieve larger sample sizes, enhancing the accuracy and generalizability of MCIDs. Overall, by promoting consistency and transparency in the calculation of MCIDs, the reliability and usefulness of these values in dermatology research and clinical practice can be enhanced.
5 Conclusion
The current inconsistencies in MCID calculations underscore a significant gap in literature in this area. The determination and application of MCIDs in AIBD are essential for evaluating treatment effectiveness and enhancing patient care. By calculating MCIDs through anchor-based and distribution-based approaches, clinicians can gain insights into clinically significant improvements and make informed treatment decisions. Further research is needed to establish MCIDs for different scoring systems using both patient- and physician-reported anchors and enhancing sample sizes to improve the application of MCID calculations in dermatology.
Author contributions
HT wrote the original manuscript. CS and DFM have made a substantial contribution revising the manuscript critically related to the accuracy or integrity of any part of the work. All authors contributed to the article and approved the submitted version.
Conflict of interest
Author DM was a coinvestigator/cocreator of the PDAI, BPDAI and the EBDASI referenced in this article.
The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
The handling editor TH is currently organizing a Research Topic with the author DM.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Supplementary material
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fimmu.2023.1243581/full#supplementary-material
References
1. Schmidt E, Kasperkiewicz M, Joly P. Pemphigus. Lancet (2019) 394:882–94. doi: 10.1016/S0140-6736(19)31778-7
2. Hanna S, Kim M, Murrell DF. Validation studies of outcome measures in pemphigus. Int J Women’s Dermatol (2016) 2:128–39. doi: 10.1016/j.ijwd.2016.10.003
3. Speeckaert R, Belpaire A, Herbelet S, Speeckaert MM, van Geel N. The meaning and reliability of minimal important differences (MIDs) for clinician-reported outcome measures (ClinROMs) in dermatology-A scoping review. J Personalized Med (2022) 12:167. doi: 10.3390/jpm12071167
4. Martin L, Murrell D. Measuring the immeasurable: a systematic review of outcome measures in pemphigus. Australas J Dermatol (2006) 47:A32–33.
5. Daniel BS, Hertl M, Werth VP, Eming R, Murrell DF. Severity score indexes for blistering diseases. Clinics Dermatol (2012) 30:108–13. doi: 10.1016/j.clindermatol.2011.03.017
6. Murrell DF, Dick S, Ahmed AR, Amagai M, Barnadas MA, Borradori L. Consensus statement on definitions of disease, end points, and therapeutic response for pemphigus. J Am Acad Dermatol (2008) 58:1043–6. doi: 10.1016/j.jaad.2008.01.012
7. Salas Apaza JA, Franco JVA, Meza N, Madrid E, Loézar C, Garegnani L. Minimal clinically important difference: The basics. Medwave (2021) 21:e8149. doi: 10.5867/medwave.2021.03.8149
8. Rai SK, Yazdany J, Fortin PR, Avina-Zubieta JA. Approaches for estimating minimal clinically important differences in systemic lupus erythematosus. Arthritis Res Ther (2015) 17:143. doi: 10.1186/s13075-015-0658-6
9. Mouelhi Y, Jouve E, Castelli C, Gentile S. How is the minimal clinically important difference established in health-related quality of life instruments? Review of anchors and methods. Health Qual Life Outcomes (2020) 18:136. doi: 10.1186/s12955-020-01344-w
10. Woaye-Hune P, Hardouin JB, Lehur PA, Meurette G, Vanier A. Practical issues encountered while determining Minimal Clinically Important Difference in Patient-Reported Outcomes. Health Qual Life Outcomes (2020) 18:156. doi: 10.1186/s12955-020-01398-w
11. Malec JF, Ketchum JM. A standard method for determining the minimal clinically important difference for rehabilitation measures. Arch Phys Med Rehabil (2020) 101:1090–4. doi: 10.1016/j.apmr.2019.12.008
12. Ousmen A, Touraine C, Deliu N, Cottone F, Bonnetain F, Efficace F. Distribution-and anchor-based methods to determine the minimally important difference on patient-reported outcome questionnaires in oncology: a structured review. Health Qual Life Outcomes (2018) 16:1–12. doi: 10.1186/s12955-018-1055-z
13. Grover S. Scoring systems in pemphigus. Indian J Dermatol (2011) 56:145–9. doi: 10.4103/0019-5154.80403
14. Wijayanti A, Zhao CY, Boettiger D, Chiang YZ, Ishii N, Hashimoto T. The reliability, validity and responsiveness of two disease scores (BPDAI and ABSIS) for bullous pemphigoid: which one to use? Acta Dermato-Venereologica (2017) 97:24–31. doi: 10.2340/00015555-2473
15. Pfutze M, Niedermeier A, Hertl M, Eming R. Introducing a novel Autoimmune Bullous Skin Disorder Intensity Score (ABSIS) in pemphigus. Eur J Dermatol (2007) 17:4–11. doi: 10.1684/ejd.2007.0090
16. Hebert V, Boulard C, Houivet E, Lehembre SD, Borradori L, Della Torre R. Large international validation of ABSIS and PDAI pemphigus severity scores. J Invest Dermatol (2019) 139:31–7. doi: 10.1016/j.jid.2018.04.042
17. Rosenbach M, Murrell DF, Bystryn J-C, Dulay S, Dick S, Fakharzadeh S. Reliability and convergent validity of two outcome instruments for pemphigus. J Invest Dermatol (2009) 129:2404–10. doi: 10.1038/jid.2009.72
18. Chams-Davatchi C, Rahbar Z, Daneshpazhooh M, Mortazavizadeh SMA, Akhyani M, Esmaili N. Pemphigus vulgaris activity score and assessment of convergent validity. Acta Med Iranica (2013) 51(4):224–30.
19. Loh CC, Kim J, Su JC, Daniel BS, Venugopal SS, Rhodes LM. Development, reliability, and validity of a novel Epidermolysis Bullosa Disease Activity and Scarring Index (EBDASI). J Am Acad Dermatol (2014) 70:89–97. doi: 10.1016/j.jaad.2013.09.041
20. Rogers CL, Gibson M, Kern JS, Martin LK, Robertson SJ, Daniel BS. A comparison study of outcome measures for epidermolysis bullosa: Epidermolysis Bullosa Disease Activity and Scarring Index (EBDASI) and the Instrument for Scoring Clinical Outcomes of Research for Epidermolysis Bullosa (iscorEB). JAAD Int (2021) 2:134–52. doi: 10.1016/j.jdin.2020.12.007
21. Jain S, Harris AG, Su JC, Orchard D, Warren LJ, McManus H, et al. The Epidermolysis Bullosa Disease Activity and Scarring Index (EBDASI): grading disease severity and assessing responsiveness to clinical change in epidermolysis bullosa. J Eur Acad Dermatol Venereology (2017) 31:692–8. doi: 10.1111/jdv.13953
22. Masmoudi W, Vaillant M, Vassileva S, Patsatsi A, Quereux G, Moltrasio C. International validation of the Bullous Pemphigoid Disease Area Index severity score and calculation of cut-off values for defining mild, moderate and severe types of bullous pemphigoid. Br J Dermatol (2021) 184:1106–12. doi: 10.1111/bjd.19611
23. Murrell DF, Daniel BS, Joly P, Borradori L, Amagai M, Hashimoto T. Definitions and outcome measures for bullous pemphigoid: recommendations by an international panel of experts. J Am Acad Dermatol (2012) 66:479–85. doi: 10.1016/j.jaad.2011.06.032
24. Futamura M, Leshem YA, Thomas KS, Nankervis H, Williams HC, Simpson EL. A systematic review of Investigator Global Assessment (IGA) in atopic dermatitis (AD) trials: Many options, no standards. J Am Acad Dermatol (2016) 74:288–94. doi: 10.1016/j.jaad.2015.09.062
25. Schram M, Spuls Ph I, Leeflang MMG, Lindeboom R, Bos JD, Schmitt J. EASI,(objective) SCORAD and POEM for atopic eczema: responsiveness and minimal clinically important difference. Allergy (2012) 67:99–106. doi: 10.1111/j.1398-9995.2011.02719.x
26. Silverberg JI, Lei D, Yousaf M, Janmohamed SR, Vakharia PP, Chopra R. What are the best endpoints for Eczema Area and Severity Index and Scoring Atopic Dermatitis in clinical practice? A prospective observational study. Br J Dermatol (2021) 184:888–95. doi: 10.1111/bjd.19457
Keywords: MCID, autoimmune blistering diseases, pemphigus, pemphigus disease area index, investigator global assessment, outcome measures
Citation: Tseng H, Stone C and Murrell DF (2023) Precision in treatment evaluation: importance of minimal clinically important differences (MCIDs) of outcome measures for autoimmune blistering diseases. Front. Immunol. 14:1243581. doi: 10.3389/fimmu.2023.1243581
Received: 21 June 2023; Accepted: 05 September 2023;
Published: 25 September 2023.
Edited by:
Takashi Hashimoto, Osaka City University, JapanReviewed by:
Branka Marinovic, University Hospital Centre Zagreb, CroatiaYen Loo Lim, National Skin Centre, Singapore
Copyright © 2023 Tseng, Stone and Murrell. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Dédée F. Murrell, d.murrell@unsw.edu.au
†ORCID: Henry Tseng, orcid.org/0009-0004-8074-3759
Corey Stone, orcid.org/0000-0002-4847-8447
Dédée F. Murrell, orcid.org/0000-0003-2971-0199