This article is a commentary on:
Does Cognitive Behavior Therapy for psychosis (CBTp) show a sustainable effect on delusions? A meta-analysis
Keith R. LawsA commentary on
Does Cognitive Behavior Therapy for psychosis (CBTp) show a sustainable effect on delusions? A meta-analysis
by Mehl, S., Werner, D., and Lincoln, T. M. (2015). Front. Psychol. 6:1450. doi: 10.3389/fpsyg.2015.01450
In their meta-analysis, Mehl et al. (2015) examine whether CBT for psychosis is an effective intervention for reducing delusions. The authors reported a small but significant effect size for CBT in comparison to TAU at end-of-therapy (k = 13; d = 0.27) and at follow-up (k = 12; d = 0.25). By contrast, no significant benefit emerged for CBT when compared to active control conditions either at end-of-therapy (k = 8; d = 0.16) or at follow-up (k = 5; d = −0.04). The meta-analysis, however, contains errors and omissions that when rectified, cast doubt on the reliability of the reported significant effects comparing CBT to Treatment as Usual (TAU) at end-of treatment and at follow-up.
First, for the end-of-trial analysis, Mehl et al. demonstrate the presence of significant publication bias. Their funnel plot is asymmetric and a trim-and-fill analysis points to four possible missing unpublished studies (a large number when only 13 RCTs studies were analyzed). They note that when effect sizes for the four “missing” studies are included, the original effect size almost halves from d = 0.27 to d = 0.14. Mehl et al. did not, however, report the 95% Confidence Intervals and whether the revised effect size is significant (rather they report the standard error: SE = 0.12)—the required analysis reveals that the revised effect size of 0.14 becomes non-significant (95%CI −0.07 to 0.35).
Second, Mehl et al. also highlight the moderate level of heterogeneity in their end-of-trial analysis and attribute this to one outlying study by Kråkvik et al. (2013). After removing Kråkvik et al. they report that heterogeneity reduces with I2 falling from 42 to 11.7%. Key information concerning the revised effect size is, however, missing. A random effects recalculation shows that removal of Kråkvik reduces the effect size from 0.27 to 0.19 (95% CI 0.03 to 0.35), which is small but significant; however re-calculation of I2 did not seem to fall to 11.7% but 21.6%. Whatever the reduction in the percentage of I2, the key point here, is why Krakvik (d = 0.94) alone is considered an outlier, but not Foster (d = 0.90) or Waller et al. (d = 0.89). The removal of a single outlier here is somewhat opaque and atheoretical.
Third, Mehl et al. give only a fleeting mention to the most well-documented, “genuine” and significant source of heterogeneity in CBTp trials—whether outcomes are measured blind or not (see Lynch et al., 2010; Jauhar et al., 2014). Indeed, the issue of blinding pertains both to the issue of Kråkvik et al. and the asymmetric funnel plot mentioned above. Mehl et al. state that “Only one of these studies did not use single-blind assessment (Foster et al., 2010)”; however, it is clear that Mehl and colleagues misclassify two non-blind trials as blind. In their discussion, Mehl et al. note that Kråkvik et al. (2013) “produced a quite large effect size (d = 0.94), which might also have been influenced by difficulties in maintaining the blinding procedure.” If we turn to the Kråkvik et al paper itself, those authors clearly state that “All four professionals were trained in the use of assessment measures, but it was not possible to keep them blind to the treatment condition.” Mehl et al. also incorrectly classify Waller et al. (2015) as a blind trial when it was non-blind. These 3 of 13 non-blind trials (Foster 0.90; Kråkvik 0.94; Waller 0.89) produced the largest effects reported for CBT on delusions. Further analysis of Mehl et al.'s data shows that blind trials (k = 9) elicit a non-significant effect size of d = 0.13 (95% CI −0.028 to 0.29) while non-blind trials (K = 4 adding Lincoln, which is self-rating) produce an effect size five-times larger with d = 0.65 (95% CI 0.21 to 1.09). Additionally, it is notable that amongst the nine blind trials, heterogeneity is virtually non-existent I2 = 4.8. In sum, the minority of non-blind trials underpins the inflation of effect sizes and their reported heterogeneity.
Turning to the follow-up analysis, where Mehl et al. claim an overall significant effect of CBT on delusions with d = 0.25. Their analysis reports a significant effect size of 0.43 for Turkington et al. (2006) which is the largest sample included and hence most heavily weighted effect size in their meta-analysis. This effect size is, however, difficult to reconcile with the data presented in the Turkington paper itself- where the Psychotic Symptom Rating Scale (PSYRATS: Haddock et al., 1999) delusion score was not only non-significant, but in the opposite direction—with a greater decrease of delusions in controls than CBT (confirmed in personal communication by one of the authors). A recalculation using the Turkington data reveals an effect size of d = −0.11 (−0.33 to 0.11) in favor of controls. If we add this new value to replace the apparently erroneous effect size reported by Mehl et al. a random effects model now shows the overall effect size reduces to 0.16 (−0.03 to 0.34) and becomes non-significant.
To summarize, examination of Mehl et al.'s end-of-trial data comparing CBT and TAU shows that if the overall effect size is adjusted for potential publication bias, then it becomes non-significant. Further analysis of the same data also shows that the significant heterogeneity reported by Mehl et al. is likely to reflect the inclusion of 4 non-blind trials, which elicit effects sizes five-times larger than in blinded trials. Analysis of nine blind trials revealed no heterogeneity and no CBT efficacy. Turning to the follow-up data, adjusting the effect size for Turkington et al. (2006) means the overall CBT efficacy becomes non-significant. In other words, CBT fails to reduce delusional thinking compared to “active” controls at either end-of-trial or at follow-up, and further fails to reduce delusional thinking when compared to TAU at follow-up and shows no efficacy at end-of-trial except where there is high risk of bias.
Author Contributions
The author confirms being the sole contributor of this work and approved it for publication.
Conflict of Interest Statement
The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
The reviewer, Maria Semkovska, and handling Editor declared their shared affiliation, and the handling Editor states that the process nevertheless met the standards of a fair and objective review.
References
Foster, C., Startup, H., Potts, L., and Freeman, D. (2010). A randomised controlled trial of a worry intervention for individuals with persistent persecutory delusions. J. Behav. Ther. Exp. Psychiatry 41, 45–51. doi: 10.1016/j.jbtep.2009.09.001
Haddock, G., McCarron, J., Tarrier, N., and Faragher, E. B. (1999). Scales to measure dimensions of hallucinations and delusions: the psychotic symptom rating scales (PSYRATS). Psychol. Med. 29, 879–889. doi: 10.1017/S0033291799008661
Jauhar, S., McKenna, P. J., Radua, J., Fung, E., Salvador, R., and Laws, K. R. (2014). Cognitive-behavioural therapy for the symptoms of schizophrenia: systematic review and meta-analysis with examination of potential bias. Br. J. Psychiatry 204, 20–29. doi: 10.1192/bjp.bp.112.116285
Kråkvik, B., Gråwe, R. W., Hagen, R., and Stiles, T. C. (2013). Cognitive behaviour therapy for psychotic symptoms: a randomized controlled effectiveness trial. Behav. Cogn. Psychother. 41, 511–524. doi: 10.1017/S1352465813000258
Lynch, D., Laws, K. R., and McKenna, P. J. (2010). Cognitive behavioural therapy for major psychiatric disorder: does it really work? A meta-analytical review of well-controlled trials. Psychol. Med. 40, 9–24. doi: 10.1017/S003329170900590X
Mehl, S., Werner, D., and Lincoln, T. M. (2015). Does Cognitive Behavior Therapy for Psychosis (CBTp) show a sustainable effect on delusions? A meta-analysis. Front. Psychol. 6:1450. doi: 10.3389/fpsyg.2015.01450
Turkington, D., Kingdon, D., Rathod, S., Hammond, K., Pelton, J., Mehta, R., et al. (2006). Outcomes of an effectiveness trial of cognitive-behavioural intervention by mental health nurses in schizophrenia. Br. J. Psychiatry 189, 36–40. doi: 10.1192/bjp.bp.105.010884
Waller, H., Emsley, R., Freeman, D., Bebbington, P., Dunn, G., Fowler, D., et al. (2015). Thinking well: a randomised controlled feasibility study of a new CBT therapy targeting reasoning biases in people with distressing persecutory delusional beliefs. J. Behav. Ther. Exp. Psychiatry 48, 82–89. doi: 10.1016/j.jbtep.2015.02.007
Keywords: cbt, CBTp, delusions, follow-up, CBT for psychosis, schizophrenia, psychosis
Citation: Laws KR (2016) Commentary: Does Cognitive Behavior Therapy for psychosis (CBTp) show a sustainable effect on delusions? A meta-analysis. Front. Psychol. 7:59. doi: 10.3389/fpsyg.2016.00059
Received: 21 October 2015; Accepted: 12 January 2016;
Published: 01 February 2016.
Edited by:
Rachel M. Msetfi, University of Limerick, IrelandReviewed by:
Maria Semkovska, University of Limerick, IrelandJane Simpson, Lancaster University, UK
Copyright © 2016 Laws. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Keith R. Laws, k.laws@herts.ac.uk