What Is the Minimum Clinically Important Change in Negative Symptoms of Schizophrenia? PANSS Based Post-hoc Analyses of a Phase III Clinical Trial

Introduction: Minimum clinically important difference (MCID) is a measure that defines the minimum amount of change in an objective score of a clinical test that must be reached for that change to be clinically noticeable. We aimed to find the MCID for patients with predominantly negative symptoms of schizophrenia at its earliest occurrence.

Methods: Data of a 26-week long, double-blind study with 454 patients [Positive and Negative Symptom Scale Negative Factor Score (PANSS-FSNS) ≥24, Positive and Negative Symptom Scale Positive Factor Score (PANSS-FSPS) ≤ 19] treated with cariprazine 4.5 mg/d or risperidone 4 mg/d were analyzed. The Clinical Global Impression—Improvement scale was used to quantify minimum improvement (CGI-I = 3) and no clinical change (CGI-I = 4) on the PANSS-FSNS, and the MCID was estimated with the following methods: as the mean PANSS-FSNS changes corresponding to the first instance of minimal improvement across all visits (MCID₁); as the difference between the PANSS-FSNS change associated with the first instance and the PANSS-FSNS changes associated with the last recorded clinically unchanged status across all visits (MCID₂); with the effect size approach (MCID₃); as the Youden Index based cut-off value between no clinical change and minimal improvement (MCID₄); as the relative likelihood of minimal improvement (MCID₅).

Results: The MCID₁ and MCID₂ resulted in, respectively, a 3.8-point (18.5%) and a 1.5-point (7.3%) decrease from baseline severity on the PANSS-FSNS. Greater values were required for the MCID at later evaluation times. The cut-off between minimum improvement and no clinical change defined by the Youden Index was a−3-point (15%) change in the PANSS-FSNS. The effect size approach indicated the 1.5-point difference between minimally improved and unchanged patients to be a medium effect (ES = 0.6).

Conclusion: Applying different methods led to different results, ranging between 7.3 and 18.5% improvement from the baseline for the MCID at its earliest occurrence in patients with predominantly negative symptoms of schizophrenia.

Introduction

The efficacy of various treatment interventions can be assessed in clinical trials by testing for statistical significance, yet a statistically significant change on a symptom scale score does not necessarily indicate a clinically relevant improvement (1). Thus, various approaches have been developed across different diseases to define the smallest beneficial effect for patients. One of the first attempts to obtain the slightest empirically observed, “clinically important” effects of intervention was published in 1989 by Jaeschke et al. (2), who defined the Minimum Clinically Important Difference (MCID) as “the smallest difference in score in the domain of interest which patients perceive as beneficial and which would mandate, in the absence of troublesome side effects and excessive cost, a change in the patient's management.” It is, therefore, a within-person, “before-after” change and conceptually distinct from minimum between-group differences that can be expected between two different treatments. The latter would be called the smallest worthwhile difference (SWD) (3). While the MCID is scale specific and assumes no substantial adverse effects or costs, the SWD represents the ratio of benefits and negative effects of two alternative treatments (4). Although other reports described the MCID with very similar names, such as the Minimal Clinically Important Difference (5, 6) Minimum Important Difference (7) or Minimal Important Change (6), the intended meaning is the same: MCID is a measure that defines the minimum amount of change in the objective score of a clinical test that must be reached for that change to be clinically noticeable.

Calculations of the MCID are usually divided into two groups: anchor- and distribution-based approaches. In anchor-based methods, an objective outcome measure of change is linked to a clinically meaningful external anchor, largely corresponding to patient perception (3, 6, 7) or in case of impaired insight, e.g., in dementia or schizophrenia, to clinical opinion (8–10). Distribution-based methods use statistical properties of study results, e.g., effect size or standard error of measurement, to calibrate the MCID (11–15).

In schizophrenia, the Positive and Negative Syndrome Scale (PANSS) measuring positive, negative, and general psychopathology, is the gold-standard instrument for assessing symptom severity (16). The clinical relevance of changes in the PANSS total score has been previously evaluated using the Clinical Global Impression (CGI) rating scale as an anchor (17, 18). It largely varies across different patient populations. Based on the CATIE study, where a very heterogeneous patient population was analyzed, a change of a 34% decrease on the PANSS total score was established as necessary to improve one category on the CGI-Severity scale (CGI-S) (18). For patients with florid positive symptoms, a 19–28% decrease in the PANSS total score was necessary to reach “minimal improvement” on the CGI-Improvement scale (CGI-I) (17). To specifically assess negative symptoms of schizophrenia, the PANSS factor score for negative symptoms (PANSS-FSNS) has been widely used in clinical trials (19). Leucht et al. found that minimal improvement corresponded to a change from baseline in PANSS-FSNS scores of −27 and −41%, as measured by the CGI-I and CGI-S (20).

Depending on the diagnostic criteria applied, negative symptoms of schizophrenia are present in 5–60% of patients with schizophrenia (21). These symptoms significantly affect a patient's quality of life as they limit functional recovery and are associated with poor functional outcomes (22). Patients with negative symptoms use more healthcare resources than patients with positive symptoms (such as: primary care, emergency care, laboratory and radiology tests, and prescription drugs), and their treatment is usually not simple, causing a clinical challenge for physicians. In contrast, positive symptoms have remarkably little association with real-life functioning and are easier to treat (23–25).

Finding the minimum clinically important change for negative symptoms may help physicians in better assessing treatment results as well as fostering the development of new instruments. In this paper, we further analyse the MCID in negative symptoms of schizophrenia, hypothesizing that as patients get better by taking their medication, more extensive changes in the PANSS-FSNS are needed to be considered clinically relevant. The previous estimation by Leucht et al. took all PANSS-FSNS changes associated with minimal improvement into account, regardless of their timepoint (20) meaning that the 27 and 41% improvement in the PANSS-FSNS associated with minimal clinical changes represent weighted averages from the first to the last instance of minimal improvement. Thus, those percentages may have overestimated the MCID in patients with predominantly negative symptoms of schizophrenia. In this work, we focus on the first instance of the MCID in patients with predominantly negative symptoms. Furthermore, to date, there is no consensus on the best method to calculate the MCID, and we apply both anchor- and distribution-based methods to get a more comprehensive picture.

Methods

Study Design

Data were analyzed from a large randomized, double-blind clinical trial treating patients with schizophrenia with predominantly negative symptoms; the study's methods and results have been previously published (26). The study was conducted at 66 study centers in 11 European countries from May 2013 to November 2014. The clinical study was approved by local independent ethics committees and was completed following clinical practice guidelines by the International Conference of Harmonization. All patients provided written informed consent. The study consisted of a 4-week lead-in period, a 26-week double-blind treatment period, and a 2-week safety follow-up with a total of 14 visits. The primary aim of the study was to assess the efficacy and safety of cariprazine treatment vs. risperidone treatment in primary, persistent, predominantly negative symptoms of schizophrenia. The primary efficacy outcome was the change from baseline in the PANSS-FSNS score at the end of the double-blind period (26 weeks). The secondary outcome parameter was change from the baseline on the Personal and Social Performance (PSP) scale. The study was well-controlled for secondary negative symptoms, assessing depression (Calgary Depression Scale for Schizophrenia), movement disorders (Simpson Angus Scale, Abnormal Involuntary Movement Scale, and Barnes Akathisia Rating Scale) and positive symptoms [PANSS factor score for positive symptoms (PANSS-FSPS)] throughout the study. Safety and tolerability were also assessed including adverse event reports, laboratory assessment, vital signs, and EEG. Patients were randomized to cariprazine 4.5 mg/d or risperidone 4 mg/d (1:1) with 2 weeks of up-titration (26).

Patients

Male and female patients with schizophrenia and predominantly negative symptoms, between 18 and 65 years of age, who were diagnosed with schizophrenia (as defined by DSM-IV-TR) and an onset of illness ≥2 years prior, were included in the study. Patients also needed to be in stable condition for at least 6 months with no hospitalisations. For study inclusion, patients must have presented with predominantly negative symptoms for ≥6 months, a PANSS-FSNS ≥24, and a score ≥4 on at least 2 of the following 3 PANSS negative symptom items: blunted affect, passive/apathetic social withdrawal, and lack of spontaneity and flow of conversation. The presence of a current DSM-IV-TR axis I disorder other than schizophrenia, and an unstable condition (such as a hospital admission in the previous 6 months), a PANSS factor score for positive symptoms (PANSS-FSPS) >19, or a PANSS-FSPS score increase of ≥25% during study lead-in were grounds for study exclusion. Other clinical exclusion criteria included substance abuse/dependence, treatment with clozapine during the 12 months before the study, a history of non-response to an adequate trial of risperidone for a psychotic episode, or treatment with risperidone within 6 weeks of screening (26).

MCID Analyses

The clinician-rated CGI-I scale was used to quantify minimum improvement (CGI-I = 3) and no clinical change (CGI-I = 4) on the PANSS-FSNS. To demonstrate any meaningful results by linking an objective scale measuring symptom severity to a subjective scale that estimates the clinical state correlation between the two scales must be demonstrated in the target population. This was done by Leucht et al., who performed the correlation analysis in the same population on which this work is based (20).

According to the definition of the MCID, a within-subject design was applied to estimate the difference between no clinical change and minimal improvement. Our observation was that PANSS-FSNS changes corresponding to minimal improvement get higher and higher over time, and thus, in order to capture the minimum clinically important difference, the MCID should be calculated on the basis of PANSS-FSNS changes associated with the earliest instance of minimal improvement. The MCID was estimated with the following methods.

Anchor-Based Methods

• MCID₁: The mean PANSS-FSNS changes corresponding to the first instance of minimal improvement (CGI-I = 3) across all visits. In other words, MCID₁ is based on the original definition by Jaeschke et al. (2) for MCID, as it represents a change from baseline in the original score units of the PANSS-FSNS scale.

• MCID₂: The difference between the PANSS-FSNS change associated with the first instance of improvement (i.e., CGI-I = 3) and the PANSS-FSNS changes associated with the last recorded clinically unchanged status (CGI-I = 4) across all visits. Thus, MCID₂ represents the mean score method of Redelmeier and Lorig (27) i.e., it shows the score difference of the “slightly better” group minus that of the “about the same” group (28). Accordingly, MCID₂ is also expressed in the original scale units.

To test the statistical significance of the difference between unchanged and minimally improved PANSS-FSNS values, we applied a mixed model for repeated measures (MMRM) analysis with improvement and visit as fixed effects. The subject was used as random effect in the model. To avoid losing cases with minimal improvement at the first visit after baseline (Week 1), zero PANSS-FSNS change was imputed for the baseline visit (where no improvement can be present by definition).

Distribution-Based Methods

• MCID₃: the effect size approach, based on the standardized response mean difference, a widely used distribution-based method to estimate the MCID, where MCID is the mean difference between the last unchanged (CGI-I = 4) and the first minimally improved (CGI-I = 3) PANSS-FSNS values divided by the pooled standard deviation (SD) of the two. It formally corresponds to the effect size calculation, making it possible to interpret the MCID in terms of the effect sizes (28).

• MCID₄: dichotomous variable indexing minimal improvement (not obtained = 0, obtained = 1) based on the cut-off value between no clinical change (CGI-I = 4) and minimal improvement (CGI = 3). A logistic regression model, with CGI-I as dependent and PANSS-FSNS as independent variables as well as baseline PANSS-FSNS as a covariate, was fitted to the data. To examine the accuracy of predicting improvement based on the PANSS-FSNS change, a receiver operating characteristics (ROC) curve was derived. The strategy used in the ROC analysis was to maximize both sensitivity and specificity, and the MCID was estimated as a cut-off value corresponding to the maximal Youden's index (29, 30).

• MCID₅ as expressed in terms of ratio of odds values (p/1-p) of being in improved vs. unchanged state at a certain degree of FSNS decrease. This estimation is based on the logistic regression model as above and expresses the strength of the predictive power of unity change in FSNS for clinical improvement.

Results

The primary results of the study were previously published (26): change from baseline to week 26 in PANSS-FSNS was significantly greater with cariprazine than with risperidone [least squares mean difference (LSMD) −1.46, 95% CI −2.39 to −0.53; p = 0.0022; effect size = 0.31]. Also, for the secondary efficacy parameter, least squares mean change from baseline to endpoint in PSP total score, was greater for cariprazine than risperidone (LSMD 4.63, 2.71–6.56; p < 0.0001; statistical effect size = 0.48). In the parameters controlling for secondary negative symptoms, least squares mean changes from baseline for PANSS-FSPS, CDSS total score, and movement scales were small and similar for cariprazine and risperidone.

A total of 454 patients from the intent-to-treat population with at least one post-baseline PANSS assessment were pooled for this analysis from both the cariprazine and risperidone treatment groups; the mean factor score on the PANSS-FSNS at baseline was 27.6, with points decreasing to 19.0 points at week 26.

By Visit Analyses

Minimal improvement on the clinical global impression scale (CGI-I = 3) was associated with PANSS-FSNS changes ranging from −2.5 points (Week 1) to −7.1 points (Week 26), consistent with our hypothesis of the MCID to be smaller at its earliest occurrence (Figure 1).

FIGURE 1

Figure 1. Change from baseline in PANSS-FSNS (Positive- and Negative Syndrome Scale—Factor Score for Negative Symptoms) as a function of minimal change vs. no change.

Quantifying the MCID by Anchor-Based Methods

MCID_{1 and 2}

The mean PANSS-FSNS change from baseline corresponding to the first occurrence of minimal improvement (CGI-I = 3) and the last recorded unchanged status across all visits were −3.8 and −2.3 points, respectively. The statistical analysis of the two arithmetic PANSS-FSNS means showed a significant difference (Table 1).

TABLE 1

Table 1. Anchor based calculations of the MCID.

Quantifying the MCID by Distribution-Based Methods

MCID₃

Based on the observed PANSS-FSNS changes the standardized effect size was determined according to the following formula: (PANSS-FSNS change from baseline at the first instance minimal improvement) − (Mean PANSS-FSNS Change from baseline at the last recorded unchanged status)/pooled SD. Our computation resulted in a standardized effect size for the improvement with a value of 0.6 [i.e., −3.8 – (−2.3)/2.5 = −1.5/ 2.5 = −0.6].

MCID₄

The ROC curve indicated statistically robust predictive values of PANSS-FSNS changes, as the model fitted to our data (Model) was highly significantly different from the reference line (ROC1) (Figure 2). Based on the maximal Youden's index method (29) we identified a −3 point decrease from the PANSS-FSNS at baseline as the cut-off value most effectively differentiated between true positive and false positive classifications, i.e., minimally improved and unchanged statuses (Figure 3).

FIGURE 2

Figure 2. Receiver Operating Characteristic (ROC) curve: predictive accuracy of the PANSS-FSNS (Positive and Negative Syndrome Scale Factor Score for Negative Symptoms) scale for differentiating minimally improved vs. clinically unchanged status. The values on the vertical and horizontal axis, respectively, depict the sensitivity (“true positive rate”) and 1− specificity (“false positive rate”) values for the differentiation as a function of change from baseline in the PANSS-FSNS scale. The leftmost part of the ROC curve represents the highest empirically observed improvements in the sample as compared to baseline while the rightmost part represents no improvement (or even deterioration). Please note that the ROC curve for differentiating the minimally improved from the clinically unchanged status based on the PANSS-FSNS (ROC model, depicted in blue in the figure) significantly outperforms the random classification (ROC1 model, in red), with an area under the curve (AUC, labeled as “Area”) value of 0.7232 vs. 0.5000 (p < 0.0001).

FIGURE 3

Figure 3. Cut-off values (Youden's indices) for predicting improvement from no clinical change to minimal improvement. To differentiate minimal improvement from no clinical change, Youden's J indices were computed at different cut-off points based on the PANSS-FSNS change. The sensitivity (vertical axis) and 1- specificity (horizontal axis) values for the differentiation of minimal improvement from no clinical change are depicted in the figure for various values of change from baseline in the PANSS-FSNS (labeled as “Cutoff”). Please note that the Youden's J index, which shows the efficiency of differentiation based on the combination of sensitivity and specificity, first increases then decreases with increasingly greater improvements (i.e., with greater negative values) as compared to baseline. The highest value of the Youden's J Index is reached at the cut-off value of −3 (i.e., at a 3 point reduction of symptom severity from baseline in the PANSS-FSNS), which identifies the optimal change value that maximizes sensitivity and specificity simultaneously.

MCID₅ as Odds Ratio

The odds ratio (OR) indicates the strength of association between the decrease in FSNS and the improved state based on CGI-I. For the −1.7 shift in FSNS obtained above as estimated MCID based on the Least Squares Mean Difference (LSMD), the OR is 1.86 (95% CI 1.62, 2.13; p < 0.05) favoring CGI-I = 3 vs. 4, The logistic regression analysis described above yielded an estimated OR of 2.07 (95% CI = 1.76 – 2.44; p < 0.05) for a 2-point decrease in the PANSS-FSNS factor score during the study. Thus, this analysis indicates that such an improvement (i.e., 2 points) in the PANSS-FSNS factor score more than doubles the likelihood of achieving minimal clinical improvement in the study.

Discussion

Previous work has established the minimal improvement (CGI-I = 3) as being associated with a 27% decrease in the PANSS-FSNS for patients with predominantly negative symptoms of schizophrenia (20) which may still be an overestimation if the time effect of improvement is considered as well. The current analyses estimated the MCID with several methods by looking at it at its earliest occurrence.

It is important to note that the five different approaches that we adopted in the current investigation to characterize MCID complement each other and delineate the MCID from various vantage points. The distribution-free approaches characterize change over time in terms of the original units on the scale of interest (PANSS-FSNS), either as a baseline-end point difference associated with the minimal improvement on the CGI-I (score = 3) (MCID1) or the difference between unchanged and minimally improved PANSS-FSNS values (MCID2). The first one of the distribution-based methods that we adopted (MCID3) expresses the difference between unchanged and minimally improved PANSS-FSNS values (measured in the original scale units) in terms of standard deviation (statistical) units. The additional two distribution-based approaches employ logistic regression modeling in order to predict the minimally improved status on the basis of the PANSS-FSNS change over time. They express MCID either as measures of the ROC Curve (AUC, Youden's index in case of MCID4) or an OR (MCID5).

Applying these approaches, we confirmed that over time, more and more prominent symptom changes were needed to achieve minimal clinical improvement. In view of this finding, we conclude that the absolute minimum clinically meaningful difference should be considered at the earliest instance, since this approach provides the highest assay sensitivity to detect clinically important changes of symptom severity over time (i.e., it allows to capture the lowest symptom change threshold for minimal clinical improvement). We note that the PANSS-FSNS of the patients whose clinical status remained the same slightly decreased as well, although to a much smaller extent. This slight symptom reduction of the clinically not improving patients may be attributable to unspecific changes, such as the regression to the mean effect, a phenomenon often seen when applying strict inclusion criteria for the patients regarding their symptom severity (31). The presence of such unspecific changes, which evolve gradually with time, may make it more difficult to establish a clinically important difference at later times in a trial, thereby providing additional rationale for focusing on the earlier time points. Additionally, because improvement is rated against the initial baseline, anchoring changes of symptom severity to clinical improvement can be more and more difficult and accompanied by greater variation as the patients progress over time during the study.

Overall, applying various underlying concepts to calculate the MCID, different methods led to different estimates with respect to the PANSS-FSNS change that need to be considered as minimally clinically significant. Our estimates from the anchor-based analyses, ranging from 7.3 to 18.5% for patients with predominant negative symptoms of schizophrenia, were below the estimates reported from studies that relied on more heterogeneous patient populations and did not take the time effect into consideration. However, it is important to bear in mind that the distribution-based statistical approaches showed a marked separation (MCID₃, i.e., statistical effect size in terms of standardized mean difference = 0.6); and highly significant predictive power for the PANSS-FSNS scale for differentiating Minimally Improved from Clinically Unchanged status in terms of ROC measures (MCID₄; e.g., AUC = 0.7232) and the odds ratio (MCID₅; OR = 2.07).

An important limitation of this investigation is that, although experienced raters met the training requirements and qualification criteria set forth before the rater training and administered the instruments, potential rating bias might have occurred. For example, the increasing PANSS-FSNS changes over time could be attributed to a possible oversight by comparing patients' clinical statuses to the previous visit instead of the baseline status (32). Furthermore, one could also argue that very early improvements might not have been real drug effects due to the two drugs' different pharmacokinetics and the onset of effect. Nevertheless, the MCID, by definition, is not about treatment-effect and could be driven by complex factors. Finally, since the CGI ratings were performed by the clinician, a further limitation of the study is that the minimally clinically important difference in the current investigation was evaluated from the clinician's perspective, not from the patient's. Patient-related outcomes were not assessed to determine the minimally clinically important change. However, we note that one study which examined patient- and clinician-rated CGI assessments simultaneously in patients with schizophrenia found only slight differences between the two approaches (18). Nonetheless, a specifically designed study is clearly needed to investigate this issue further. Further, an additional limitation derives from the use of only one particular scale. In this study we adopted one of the benchmark scales for negative symptoms of schizophrenia, the PANSS-FSNS. Assessing negative symptoms with different scales (such as the SANS, BNSS, CAINS, etc.) would yield different values as minimally important change, as scoring on these scales is different. Consequently, further research is needed to identify minimally clinically important changes on different negative symptom scales.

Negative symptoms, that are not secondary to positive ones, are known for being less responsive to antipsychotic therapy (33–35). Their presence often challenges the therapeutic strategy and makes clinicians switch the patients' medication. Our findings may help clinicians and drug developers have a more precise idea about improving predominant negative symptoms in clinical decision-making, or in terms of designing trials for patients with predominant negative symptoms.

Conclusion

Applying different methods lead to different results, ranging between 7.3 and 18.5% improvement from baseline for the MCID at its earliest occurrence in patients with predominant negative symptoms of schizophrenia, suggesting even lower thresholds than previously thought.

Data Availability Statement

The datasets presented in this article are not readily available because they are part of a bigger randomized clinical study dataset, and are owned by the company Gedeon Richter. The datasets have been provided to authors to perform the presented analyses only. While the full datasets can therefore not be shared, all statistical analyses and data outputs generated for the present study/publication can be requested by the authors.

Ethics Statement

The clinical study protocol was approved by nine central and 37 local independent Ethics Committees in relation to the 66 sites that recruited at least one patient; the study was done in accordance with good clinical practice guidelines and the principles of the International Conference on Harmonization. All patients provided written informed consent. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

SL, PC, ÁB, IL, and GN developed the concept of the current investigation. Methodology contribution was provided by KA, PC, SL, and TAF. The statistical plan was outlined by PC and KA. KA conducted the statistical analyses under the supervision of PC. Software preparation was done by KA and PC. KA, PC, BS, and IL had a leading role in the preparation of the figures and table. Project administration was provided by ÁB, BS, IL, and GN. PC and KA interpreted the statistical outputs. Conceptual supervision for manuscript preparation and writing was provided SL, PC, and TAF. BS, IL, ÁB, and PC performed the writing. All authors contributed to the article and approved the submitted version.

Conflict of Interest

PC, BS, KA, ÁB, IL, and GN reports personal fees from Gedeon Richter Plc., outside the submitted work. TAF reports grants and personal fees from Mitsubishi-Tanabe and from Shionogi, personal fees from MSD and from SONY, outside the submitted work. In addition, TAF has a patent 2020-548587 concerning smartphone CBT apps pending, and intellectual properties for Kokoro-app licensed to Mitsubishi-Tanabe. SL reports honoraria as a consultant/advisor and/or for lectures from Angelini, Böhringer Ingelheim, Geodon Richter, Janssen, Johnson & Johnson, Lundbeck, LTS Lohmann, MSD, Otsuka, Recordati, SanofiAventis, Sandoz, Sunovion, TEVA, Eisai, Rovi, and Medichem. GN and IL have issued patents for cariprazine.

This study was sponsored by Gedeon Richter Plc. Gedeon Richter was involved in the study design, collection (via contracted clinical investigator sites), analysis, and interpretation of data and decided to submit it for publication.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

1. Correll CU, Kishimoto T, Nielsen J, Kane JM. Quantifying clinical relevance in the treatment of schizophrenia. Clin Ther. (2011) 33:B16–39. doi: 10.1016/j.clinthera.2011.11.016

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Jaeschke R, Singer J, Guyatt GH. Measurement of health status. Ascertaining the minimal clinically important difference. Control Clin Trials. (1989) 10:407–15. doi: 10.1016/0197-2456(89)90005-6

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Ferreira ML, Herbert RD, Ferreira PH, Latimer J, Ostelo RW, Nascimento DP, et al. A critical review of methods used to determine the smallest worthwhile effect of interventions for low back pain. J Clin Epidemiol. (2012) 65:253–61. doi: 10.1016/j.jclinepi.2011.06.018

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Furukawa TA. Measuring clinical importance in a trial of interventions for mixed urinary incontinence. J Am Med Assoc. (2020) 323:479. doi: 10.1001/jama.2019.19730

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Beurskens AJHM, De Vet HCW, Köke AJA. Responsiveness of functional status in low back pain: a comparison of different instruments. Pain. (1996) 65:71–6. doi: 10.1016/0304-3959(95)00149-2

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Wright A, Hannon J, Hegedus EJ, Kavchak AE. Clinimetrics corner: a closer look at the minimal clinically important difference (MCID). J Man Manip Ther. (2012) 20:160–6. doi: 10.1179/2042618612Y.0000000001

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Cella D, Bullinger M, Scott C, Barofsky I, Aaronson N, Berzon R, et al. Group vs individual approaches to understanding the clinical significance of differences or changes in quality of life. Mayo Clin Proc. (2002) 77:384–92. doi: 10.4065/77.4.384

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Falissard B, Sapin C, Loze JY, Landsberg W, Hansen K. Defining the minimal clinically important difference (MCID) of the Heinrichs-carpenter quality of life scale (QLS). Int J Methods Psychiatr Res. (2016) 25:101–11. doi: 10.1002/mpr.1483

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Andrews JS, Desai U, Kirson NY, Zichlin ML, Ball DE, Matthews BR. Disease severity and minimal clinically important differences in clinical outcome assessments for Alzheimer's disease clinical trials. Alzheimer's Dement Transl Res Clin Interv. (2019) 5:354–63. doi: 10.1016/j.trci.2019.06.005

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Amri I, Millier A, Toumi M. Minimum clinically important difference in the global assessment functioning. Value Health. (2014) 17:A765–6. doi: 10.1016/j.jval.2014.08.285

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Jacobson NS, Roberts LJ, Berns SB, McGlinchey JB. Methods for defining and determining the clinical significance of treatment effects: description, application, and alternatives. J Consult Clin Psychol. (1999) 67:300–7. doi: 10.1037/0022-006X.67.3.300

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Jacobson NS, Truax P. Clinical significance: a statistical approach to defining meaningful change in psychotherapy research. J Consult Clin Psychol. (1991) 59:12–9. doi: 10.1037/0022-006X.59.1.12

PubMed Abstract | CrossRef Full Text | Google Scholar

13. de Vet HC, Terwee CB, Ostelo RW, Beckerman H, Knol DL, Bouter LM. Minimal changes in health status questionnaires: distinction between minimally detectable change and minimally important change. Health Qual Life Outcomes. (2006) 4:54. doi: 10.1186/1477-7525-4-54

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Wyrwich KW, Nienaber NA, Tierney WM, Wolinsky FD. Linking clinical relevance and statistical significance in evaluating intra-individual changes in health-related quality of life. Med Care. (1999) 37:469–78. doi: 10.1097/00005650-199905000-00006

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Muller K, Cohen J. Statistical power analysis for the behavioral sciences. Technometrics. (1989) 31:499–500. doi: 10.1080/00401706.1989.10488618

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Suzuki T. Which rating scales are regarded as “the standard” in clinical trials for schizophrenia? a critical review. Psychopharmacol Bull. (2011) 44:18–31.

PubMed Abstract | Google Scholar

17. Leucht S, Kane JM, Kissling W, Hamann J, Etschel E, Engel RR. What does the PANSS mean? Schizophr Res. (2005) 79:231–8. doi: 10.1016/j.schres.2005.04.008

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Hermes EDA, Sokoloff D, Stroup TS, Rosenheck RA. Minimum clinically important difference in the positive and negative syndrome scale with data from the Clinical Antipsychotic Trials of Intervention Effectiveness (CATIE). J Clin Psychiatry. (2012) 73:526–32. doi: 10.4088/JCP.11m07162

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Marder SR, Daniel DG, Alphs L, Awad AG, Keefe RSE. Methodological issues in negative symptom trials. Schizophr Bull. (2011) 37:250–4. doi: 10.1093/schbul/sbq161

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Leucht S, Barabássy Á, Laszlovszky I, Szatmári B, Acsai K, Szalai E, et al. Linking PANSS negative symptom scores with the Clinical Global Impressions Scale: understanding negative symptom scores in schizophrenia. Neuropsychopharmacology. (2019) 44:1589–96. doi: 10.1038/s41386-019-0363-2

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Rabinowitz J, Werbeloff N, Caers I, Mandel FS, Stauffer V, Menard F, et al. Negative symptoms in schizophrenia - the remarkable impact of inclusion definitions in clinical trials and their consequences. Schizophr Res. (2013) 150:334–8. doi: 10.1016/j.schres.2013.06.023

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Correll CU, Schooler NR. Negative symptoms in schizophrenia: a review and clinical guide for recognition, assessment, and treatment. Neuropsychiatr Dis Treat. (2020) 16:519–34. doi: 10.2147/NDT.S225643

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Harvey PD, Bellack AS. Toward a terminology for functional recovery in schizophrenia: is functional remission a viable concept? Schizophr Bull. (2009) 35:300–6. doi: 10.1093/schbul/sbn171

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Harvey PD, Strassnig M. Predicting the severity of everyday functional disability in people with schizophrenia: cognitive deficits, functional capacity, symptoms, and health status. World Psychiatry. (2012) 11:73–9. doi: 10.1016/j.wpsyc.2012.05.004

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Galderisi S, Rossi A, Rocca P, Bertolino A, Mucci A, Bucci P, et al. The influence of illness-related variables, personal resources and context-related factors on real-life functioning of people with schizophrenia. World Psychiatry. (2014) 13:275–87. doi: 10.1002/wps.20167

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Németh G, Laszlovszky I, Czobor P, Szalai E, Szatmári B, Harsányi J, et al. Cariprazine versus risperidone monotherapy for treatment of predominant negative symptoms in patients with schizophrenia: a randomised, double-blind, controlled trial. Lancet. (2017) 389:1103–13. doi: 10.1016/S0140-6736(17)30060-0

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Redelmeier DA, Lorig K. Assessing the clinical importance of symptomatic improvements: an illustration in rheumatology. Arch Intern Med. (1993) 153:1337–42. doi: 10.1001/archinte.1993.00410110045008

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Angst F, Aeschlimann A, Angst J. The minimal clinically important difference raised the significance of outcome effects above the statistical level, with methodological implications for future studies. J Clin Epidemiol. (2017) 82:128–36. doi: 10.1016/j.jclinepi.2016.11.016

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Schisterman EF, Perkins NJ, Liu A, Bondell H. Optimal cut-point and its corresponding Youden index to discriminate individuals using pooled blood samples. Epidemiology. (2005) 16:73–81. doi: 10.1097/01.ede.0000147512.81966.ba

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Powers DMW. Evaluation: From Precision, Recall and F-Factor. Tech Rep SEI-07-001. Adelaide, SA: Flinders University of South Australia (2007).

Google Scholar

31. Bland JM, Altman DG. Regression towards the mean. BMJ. (1994) 308:1499. doi: 10.1136/bmj.308.6942.1499

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Busner J, Targum SD. The clinical global impressions scale: applying a research tool in clinical practice. Psychiatry. (2007) 4:28–37.

PubMed Abstract | Google Scholar

33. Fusar-Poli P, Papanastasiou E, Stahl D, Rocchetti M, Carpenter W, Shergill S, et al. Treatments of negative symptoms in schizophrenia: meta-analysis of 168 randomized placebo-controlled trials. Schizophr Bull. (2015) 41:892–9. doi: 10.1093/schbul/sbu170

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Aleman A, Lincoln TM, Bruggeman R, Melle I, Arends J, Arango C, et al. Treatment of negative symptoms: Where do we stand, and where do we go? Schizophr Res. (2017) 186:55–62. doi: 10.1016/j.schres.2016.05.015

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Barnes TR. Evidence-based guidelines for the pharmacological treatment of schizophrenia: recommendations from the British Association for Psychopharmacology. J Psychopharmacol. (2011) 34:3–78. doi: 10.1177/0269881119889296

PubMed Abstract | CrossRef Full Text | Google Scholar