Gerhard Andersson
Per Carlbring
Alexander Rozental- 1Department of Behavioural Sciences and Learning, Linköping University, Linköping, Sweden
- 2Department of Clinical Neuroscience, Karolinska Institutet, Stockholm, Sweden
- 3Department of Psychology, Stockholm University, Stockholm, Sweden
- 4Department of Psychology, University of Southern Denmark, Odense, Denmark
- 5Institute of Child Health, University College London, London, United Kingdom
Background: Internet-delivered cognitive behavior therapy (ICBT) was developed over 20 years ago and has since undergone a number of controlled trials, as well as several systematic reviews and meta-analyses. However, the crucial question of response rates remains to be systematically investigated. The aim of this individual patient meta-analysis (IPDMA) was to use a large dataset of trials conducted in Sweden to determine reliable change and recovery rates across trials for a range of conditions.
Methods: We used previously collected and aggregated data from 2,866 patients in 29 Swedish clinical trials of ICBT for three categories of conditions: anxiety disorders, depression, and others. Raw scores at pre-treatment and post-treatment were used in an IPDMA to determine the rate of reliable change and recovery. Jacobson and Truax’s, (1991) reliable change index (RCI) was calculated for each primary outcome measure in the trials as well as the recovery rates for each patient, with the additional requirement of having improved substantially. We subsequently explored potential predictors using binomial logistic regression.
Results: In applying an RCI of z = 1.96, 1,162 (65.6%) of the patients receiving treatment were classified as achieving recovery, and 620 (35.0%) were classified as reaching remission. In terms of predictors, patients with higher symptom severity on the primary outcome measure at baseline [odds ratio (OR) = 1.36] and being female (OR = 2.22) increased the odds of responding to treatment. Having an anxiety disorder was found to decrease the response to treatment (OR = 0.51). Remission was predicted by diagnosis in the same direction (OR = 0.28), whereas symptom severity was inversely predictive of worse outcome (OR = 0.81). Conclusions: Response seems to occur among approximately half of all clients administered ICBT, whereas about a third reach remission. This indicates that the efficacy of ICBT is in line with that of CBT based in prior trials, with a possible caveat being the lower remission rates. Having more symptoms and being female might increase the chances of improvement, and a small negative effect of having anxiety disorder versus depression and other conditions may also exist. A limitation of the IPDMA was that only studies conducted in Sweden were included.
Introduction
Internet-delivered cognitive behavior therapy (ICBT) has existed for more than 20 years (1), and treatment programs have been developed for a wide range of clinical and non-clinical conditions. Most forms of ICBT are administered in the form of guided self-help, with text and video presentations boosted by email support in a secure online platform resembling Internet banking (2). One way to describe ICBT is to pinpoint the similarities with online education, even if the treatment is largely based on self-help texts and cognitive behavioral therapy (CBT) manuals (3). Thus, ICBT programs tend to rely on psychoeducation and instructions for how to change thoughts, emotions, and behaviors in everyday life, and, like CBT, programs usually last 5 to 15 weeks with homework assignments and therapist feedback provided on a weekly basis (3).
Research suggests that therapist-supported ICBT—in contrast to unguided treatments (4)—can be as effective as face-to-face cognitive behavior therapy (5), yield long-term results (6), and work under clinically representative conditions (7). ICBT has also been tested for different target groups—for example, young persons (8), adults (9), older persons (10), and immigrants (11). Treatment programs have focused on specific problems, such as procrastination (12); diagnoses like post-traumatic stress disorder (PTSD) (13); or tailored according to a client’s specific problem profile (14). Another approach has been to deliver transdiagnostic treatments in which one treatment is used to target underlying common processes, such as avoidance (15). There are also studies on other psychotherapy forms, including psychodynamic psychotherapy (e.g., 16), interpersonal psychotherapy (17), versions of CBT such as acceptance and commitment therapies (18), and attention training (19). Finally, there are also programs based on physical exercise (20), mindfulness (21), and relaxation (22), even if the latter two are often incorporated into ICBT protocols (23). In addition to the controlled trials, several studies exist on moderators and mediators of outcome (e.g., 24), as well as some qualitative studies on the client’s experience during ICBT (25).
While it is common to report effects in clinical trials, there are also studies and reviews reporting negative effects and non-response to ICBT. In two previous individual patient data meta-analyses (IPDMA), we studied these two outcomes (26, 27). With regards to negative effects in the form of deterioration, 5.8% of treated research participants showed such effects (26), and non-response was present among 26.8% of participants (27). When completing these two reviews based on our dataset of controlled trials, a question emerged regarding response rates in our ICBT studies as this was not reported in the previous ones given uncertainties regarding definitions and scope of the two previous reviews. In contrast to the standard of reporting mean standardized differences with metrics like Cohen’s d, there is far less agreement on how to define response in psychological treatments studies in general, and specifically in CBT (28). Several questions emerge when defining what constitutes a “response” to treatment. In a review, 26, p. 73-74) mentioned several issues, such as (a) number of measures used to define response, (b) number of modalities (e.g., self-report versus observed behavior), (c) blinding of assessors, (d) degree of change from baseline, and (e) use of a clinical cut-off to determine if a client has reached a non-clinical state (sometimes referred to as high end-state or remission). In a seminal paper, Jacobson and Truax (29) outlined guidelines for defining change from baseline (reliable change index-RCI) and different ways to define what constitutes being within a non-clinical range or having reached high end-state function/remission. In the present review, we will use the term “remission” to refer to what can be counted as high end-state function, which allows us to be consistent with a previous IPDMA on depression by Karyotaki et al. (30).
In their IPDMA on guided ICBT for depression focusing on response and remission, Karyotaki et al. (30) included 24 RCTs (4,889 participants) and compared guided ICBT with a control group. The mean pooled response rate (based on RCI) at post-treatment was 56.19%, and the mean remission rate at post-treatment in the treatment groups (N = 26) was 38.51% based on the RCI criteria (1.96)—two standard deviations improvement from baseline for each measure. Given the dataset we coded based on our own trials, we decided to conduct a new IPDMA (31) knowing that we could use original data across studies to investigate reliable change and remission (high end-state function). As in our previous two IPDMAs (26, 27), we used data from 29 clinical trials. The final dataset with complete data consisted of 1,535 patients who had received ICBT, and trials were categorized into three groups: anxiety disorders, depression/mood disorders, and other conditions (i.e., erectile dysfunction, relationship problems, and gambling disorder). The aim of the current study was to determine the rates of treatment response and remission in clients who had received ICBT in clinical trials conducted by our group in Sweden. As a secondary exploratory aim, we examined potential predictors of response.
Materials And Methods
Individual Patient Data Meta-Analysis
As described in our two previous IPDMAs (26, 27), we used the scores for individual client and outcome variable across studies (32). In IPDMAs, it is study factors that might be predictive of treatment outcome using the raw data instead of group means, as has been done in previous IPDMAs—for example, on low-intensity psychological treatments (33) and Internet interventions for problem drinking (34). As in the previous IPDMAs, we aggregated available data from clinical trials that we conducted. A complete description surrounding our data collection procedure is presented in Rozental et al. (26). An obvious limitation of using this method is that we cannot assess the risk of bias, which is a common practice in systematic reviews (32). However, by including trials from our own group, we were able to obtain an overall view of response and remission, which we assume could be representative, particularly for Swedish settings. For a complete description of the inclusion and exclusion criteria used, see 26, p.163).
The raw scores from each client in the included trials were entered into one data matrix, and codes for background variables were aligned. Given the complexity of modeling reasons for missing data in such a heterogeneous group of research participants and the fact that we were focused on binary outcomes in this review, we decided to use a complete case approach instead of multiple imputation, as we were convinced that data were not missing completely at random (35). Moreover, complete case analysis has been recommended as the first approach when conducting meta-analyses, even if this approach is followed by sensitivity analyses to detect possible bias in estimates (36).
Sociodemographic variables were occasionally collapsed to facilitate comparisons and to obtain consistency across trials (see 26, 27). Trials were categorized into three categories: (1) anxiety disorders, (2) depression and mood disorders, and (3) other (i.e., erectile dysfunction, relationship problems, and gambling disorder). In Table 1, we present an overview of the clients’ sociodemographic variables in the trials included (the table overlaps with Table 3 in 26, p. 169 but does not include the control groups). We present data from baseline in the trials and also the amount of missing data for the full sample.
Table 1 Sociodemographic characteristics of the participants included in the analysis (based on Table 3 in 26).
Statistical Analysis
The RCI was chosen based on its widespread use for assessing reliable change (29, 37). As is common practice, the RCI was calculated by taking each individual change score and then dividing this score by the standard error of the difference, i.e., SEdiff = SD1√2√1-r. In the formula, SD1 corresponds to the standard deviation of a condition at pre-treatment, and r is the reliability estimate (38). We calculated RCIs for the primary outcome measure of each included trial and used the test–retest reliability for that specific trial measure (see Table 2; also reported in 27). Basically, the RCI sets the limit for when a change score is unlikely to be real (p = .05). Following the usual standards, we calculated RCIs for which a change equal to z = 1.96 on the basis of a standard deviation unit was used. Following this, each participant in the trials could be classified as either a responder or a non-responder to treatment, with the definition of response being specific to each study and measure used, a similar method to that used by Karyotaki et al. (30). Heterogeneity was tested by entering response rates into the program comprehensive meta-analysis (version 2.2.021; CMA).
Table 2 Test-retest reliabilities used for calculating improvement rates based on the reliable change index (same Table as in 27, p. 6).
We also followed the methods of Karyotaki et al. (30) when calculating remission, again using criteria set by Jacobson and Truax (29). Participants were classified as remitters if they moved two standard deviations below the mean of the clinical group to which they belonged in each study. The resulting cut-off scores indicates remission, which is a hard criterion of remission, often being equivalent to a symptom-free state. For six of the studies, it was not possible to use the two standard deviation criteria due to floor effects, so instead, we used one standard deviation as the criterion in these cases.
In order to investigate possible predictors, we applied binomial logistic regression and used either response or recovery rates as dependent variables. All variables were entered into the model simultaneously. In terms of the variables used, we selected a few clinically relevant demographic and clinical predictors (54) (31). We selected the same variables as in our previous IPDMAs on deterioration (26) and non-response (27). The predictors were (a) symptom severity at baseline, (b) civil status, (c) age, (d) sick leave, (e) previous psychological treatment, (f) previous or ongoing psychotropic medication, (g) educational level, (h) diagnosis, and (i) gender. We also added a separate analysis of the association between publication year and the two outcomes response and recovery.
We present odds ratios (OR) with 95% confidence intervals (CI) in order to reflect an increase or decrease in odds of response and remission in relation to a reference category. In the case of dichotomous predictors (such as gender), the OR reflects the odds of response or remission when a client is female versus male (reference). A positive OR thus means better response in women. For continuous predictors (symptom severity at baseline and age), the OR instead represents an increase of one standard deviation above their respective mean. The statistical analyses on predictors were performed using jamovi version 0.9.2.9 (55), with the proportions of response and remission performed on a complete case basis (see online Supplementary Material).
Ethics
The data in the current IPMA were derived from several clinical trials, all of which had received ethical approval from the respective regional ethical review boards at each study location.
Results
Study Characteristics
The 29 clinical trials were coded according to the previously predefined inclusion and exclusion criteria and deemed eligible for the current IPDMA (see 26, 27 for further details). Raw scores from all clients were entered into one spread sheet. The exception was 46 clients who had received either psychodynamic psychotherapy or interpersonal psychotherapy via the Internet as a control condition. In total, 1,535 clients were included in this IPDMA. The following diagnoses were included (clinical trials, k): social anxiety disorder (9), depression (with/without dysthymia) (5), generalized anxiety disorder (3), anxiety disorder (with/without depression) (3), mixed anxiety disorders (e.g., panic disorder as well as social anxiety disorder) (2), specific phobia (2), posttraumatic stress disorder (1), panic disorder (with/without agoraphobia) (1), gambling disorder (1), erectile dysfunction (1), and relationship problems (1) (see 26, p.6). Briefly, most participants in the trials had been recruited from the general population based on self-referral (n = 27). A common practice in the trials was to use structured telephone interviews such as the Structured Clinical Interview for DSM-IV-Axis I Disorders (56) or the MINI-International Neuropsychiatric Interview (57). Some studies had used diagnosis-specific instruments such as the Clinician-Administered PTSD Scale (58). For a description of treatment content original studies, (see 26, 27). The amount of missing data for the primary outcome measures at post-treatment was 12.9%. A complete overview of the clinical trials is presented in Table 3 (which to some extent overlaps with Table 3 in 27 but with different results presented).
Table 3 Characteristics and rates of improvement and recovery for the clinical trials included in the individual patient data meta-analysis.
Response and Remission Rates
Using the RCI criteria for detecting response, 1,027 (69.9%; 95% CI: 67.61–72.19) of the 1,535 participants receiving treatment were categorized as treatment responders when using an RCI of z = 1.96. The lowest rates were found in the trials on erectile dysfunction (12.1%) and older adults with anxiety (27.3%), whereas the highest rates were found in the trials on gambling (93.6%) and spider phobia (92.3%). As seen in Table 3, the proportions varied, which was confirmed by the CMA program showing a significant heterogeneity (I2 = 87.5%; Q = 223, p < .001).
Remission was achieved in 540 participants (35.2%; 95% CI: 32.81–37.59) when adjusting for floor effects (the unadjusted proportion was 31.9%). There was a large variation with ranging from 0% (erectile dysfunction, depression, and bias modification for social anxiety disorder) to 82% (gambling) and 69% (spider phobia). As with the response rates, a significant heterogeneity was found (I2 = 91.6%; Q = 334, p < .001).
Predictors of Response
Binomial logistic regressions were calculated with the predefined variables entered as predictors of response. Results are presented in Table 4 with OR and 95% CI for each predictor. The results that indicate a higher symptom severity on the primary outcome measure at baseline was predictive of better outcome. The odds for responding to treatment decreased when having an anxiety disorder as compared to depression/mood disorder and other diagnoses (i.e., erectile dysfunction, relationship problems, and gambling disorder), and the odds increased if the subject was female. The other variables were not predictive of response.
Table 4 Significance levels, odds ratios, and 95% confidence intervals (CI) for predictors of response.
We also repeated the analyses with remission as outcome (see Table 5). In this analysis, symptom severity at pre-treatment was marginally associated with less remission (OR = 0.81). As with the analysis for response, the odds for remission were lower if the subject suffered from anxiety. Gender and the other variables did not reach statistical significance. Publication year was unrelated to response and remission.
Table 5 Significance levels, odds ratios, and 95% confidence intervals (CI) for predictors of recovery.
Discussion
The aim of this IPDMA was to obtain estimates of response and remission rates for ICBT with a range of conditions categorized into three groups (anxiety, depression, and other). In line with a previous IPDMA on depression by Karyotaki et al. (30) which included trials from different countries, we found that 65.6% of the treated research participants could be classified as treatment responders. This is slightly higher than the 56.19% reported by Karyotaki et al. (30). While Karyotaki et al. (30) imputed missing data, they also reported that the estimates between complete case analysis and the imputed dataset were minor. In addition, we found that 35.0% of participants could be classified as having remitted, which is slightly lower than the 38.51% remission rate reported by Karyotaki et al. (30). Moreover, we had a problem with floor effects and if not considering that our estimate is even lower (31.9%). Overall, the results are rather similar to Karyotaki et al. (30) in that roughly half of clients showed improvement following ICBT and remission was achieved by a third.
Given these estimates, the outstanding question is how well this compares against face-to-face CBT. As previously mentioned, there are a few studies in which clients have been randomly assigned either face-to-face CBT or therapist-guided CBT. In the most recent of such study, Carlbring et al. (5) found no differences in effect. The response rates in CBT across different disorders and conditions are difficult to estimate; as to our knowledge, there is no similar IPDMA on response rates in face-to-face CBT. The closest we can get is a meta-analysis on anxiety disorders based on published data (28). In that review, 31% of the studies had defined response using RCI, and the results were similar to the range found in this analysis (44.5–51.1%). However, Loerinc et al. (28) also reported that the use of RCI in combination with a clinical cut-off (as was done in the present study) was associated with a 28% lower response rate (or, as expressed in this study, the remission rate was 28% lower than the RCI response rate). The difference we found was somewhat larger, as about half of the responders also showed remission. From a more naturalistic perspective, Gyani et al. (82) reported that 63.7% of participants in their clinical sample showed reliable improvement following evidence-based face-to-face treatment. In a recent meta-analysis, Springer et al. (83) reported a remission estimate as high as 51% (compared to our estimate of 35%). As stated by Loerinc et al. (28), there is a large variation in how to define both response and remission in CBT trials; thus, one advantage of the approach taken here is that we can use the same approach across studies. However, the definition of remission used (two SDs below the pre-treatment mean) was impossible in some studies (leading us to correct that estimate) and unrealistic in others. Another disadvantage of a statistical definition of response and remission is that it is heavily dependent on the sample upon which it is calculated. This pertains to both RCI and recovery as outlined by Jacobson and Truax (29). Another approach would have been to determine criteria for response and remission independently of the study, which is possible when there is data on non-clinical samples. For example, on the Beck Depression Inventory (44), a 10-point reduction could be seen as indicative of response, and 13-point reduction indicative of recovery. Applying these criteria for the Stella trial (see Table 3; 63), in which 57.5% of participants responded according to the RCI and 0% remitted, the corresponding figures of using a 10-point reduction (53.1%) and a score of 13 or below on the BDI (59.4%) paints a slightly different picture (although the results are similar between the RCI criteria and the 10-point reduction). In particular, the difference between the 0% classified as remitters versus the 59.4% having a score indicating minimal depression (44) shows the importance of response definitions (for a detailed discussion, see 37). However, in the present IPDMA, we found it to be of value to use similar criteria across trials. Future research could focus more on the external criteria of improvement instead of study-specific criteria. Unfortunately, the BDI is an exception, being widely used in many trials, and for some conditions and measures included in this IPDMA, cut-off scores and non-clinical norms have not been established.
In the present IPDMA, we conducted exploratory analyses to see if response could be predicted. It is important to note that there was no firm theoretical basis for our selection of predictors and, therefore, our findings must be interpreted with caution, as the identification of a significant predictor could have practical implications for future treatment recommendations. This is particularly the case when findings are based on samples larger than is commonly used in psychotherapy trials. However, the finding that symptom severity was not a negative predictor, but rather the opposite is in line with a previous IPDMA on low-intensity interventions for depression (33). From a clinical point of view, this makes sense, as having symptoms makes treatment more relevant than if subclinical or even different symptoms are present in the client. However, it also suggests that our ICBT trials probably included participants without severe symptoms. It is therefore possible that this finding is based on selection criteria used in trials and not necessarily relevant for clinical practice when treatment is offered with fewer restrictions. In contrast to the finding for response, we found a small negative effect of pre-treatment severity for the prediction of remission. These two conflicting results may indicate that large improvements are less likely if the client has more symptoms. However, as remission requires a low level of symptoms, it is less likely that a client with many symptoms will reach that low level. As in treatment research in general, it could be that the search for predictors is best pursued in ordinary clinical settings rather than in well-controlled trials. On a promising note, large ICBT effectiveness studies are being reported (84), which may provide clearer insight into the efficacy of different approaches for different populations.
The odds ratio in favor of female participants was surprisingly high (OR = 2.22), which is hard to explain as this is not a consistent result from previous individual trials. One possible explanation is the fact that the gender proportions in trials are nested with the condition treated. For example, in a typical depression trial, there is a majority of women, whereas in other conditions, there are more or less equal proportions of men and women, while some studies have only men (e.g., erectile dysfunction). The overall takeaway message here is that this finding needs further investigation in future trials to confirm its veracity. Gender was not found to be predictive of remission.
Interestingly, while the IPDMA by Karyotaki et al. (30) found old age to be weakly associated with better response (OR = 1.01), there was no such effect in this study suggesting that age is not a predictor of outcome. In Karyotaki et al.’s (30) complete case analyses, baseline severity (OR = 1.16) was found to predict better outcome, which was in line with our findings (but not in their intention to treat analyses). Gender did not predict outcome in Karyotaki et al.’s (30) IPDMA, but, again, their review focused on only depression which means a larger portion of the population was likely female.
In this IPDMA, we found decreased odds for responding to treatment when having an anxiety disorder as compared to depression/mood disorders or other (erectile dysfunction, relationship problems, and gambling disorder). This was a small effect, but still puzzling and hard to understand given that the overall picture is the opposite—indicating a higher response to ICBT in clients with anxiety disorders. Anxiety disorders were also found to be predictive of lower rates of remission.
While non-significant findings cannot be viewed as proof of absence of an effect, it is still interesting that ICBT has had marginal success in finding predictors of change as well as moderators and mediators of outcome (1). There are several possible reasons for this. First, participants in trials are not selected for their differences. Rather, they are chosen based on diagnostic criteria, access to the Internet, willingness to be a research participant, etc., all of which likely reduce the chance of identifying accurate predictors. Second, predictors of dropout (85) may ultimately be more illuminating, even if a given IPDMA was on self-guided ICBT for depressive symptoms. It is, however, interesting to note that they found that male gender (RR = 1.08) and co-morbid anxiety symptoms (RR = 1.18) significantly increased the risk of dropping out of the study, which is in line with our findings with regards to gender and anxiety disorders. In spite of the overall inconsistency in findings and lack of findings, we believe that IPDMA is a powerful and reliable tool for answering questions regarding predictors of response (86). However, this will require concerted efforts to align both outcome measures as well as data on predictors in order to make studies comparable. Even if we were co-workers and principal investigators in the trials included in this IPDMA, the data were not consistent with regards to background variables etc. It becomes even more problematic when attempting to combine datasets from different research groups.
This study had several limitations. We focus on four, knowing that there are further objections that could be raised. First, we only included our own trials, which were conducted in Sweden. IPDMAs are often selective, as original data sometimes cannot be obtained from authors, but in this case, we cannot generalize the results outside of our own culture and setting. In addition to not including trials from international groups, we also did not include the most recent and unpublished trials from our own groups, and there are other groups in Sweden with trials that were also not included. Furthermore, with the focus on our own studies conducted over a period close to 20 years, there is a possibility of time trends, which we did not focus on in this study. Technological changes may not influence effects, but, to give an example, early studies relied more on printing out text materials (87), whereas recent studies are typically delivered via online platforms (e.g., responsive to the presentation format, such as smartphones or computers) (2). Second, even if this limitation is not unique for ICBT, outcome measures were largely based on self-report. Such measures are useful and generally have good psychometric properties but still present a possible risk that self-reported changes do not correspond with actual behavior changes and do not conform to the findings of an interview. Third, as commented on by Loerinc et al. (28), treatment response can be defined by several measures, but here, we focused only on the primary outcomes. Our dataset on IPDMA could also be used to analyze effects on other secondary measures of constructs, like quality of life, as several of our studies have used the same measure for this construct (e.g., 88), and it has been reported that the effects of ICBT might be lower on that construct (89). The fourth limitation relates to the statistical methods used. We decided to report on a complete case basis, but we could not exclude bias, as missing data was not considered.
Conclusions
In spite of the limitations, the present IPDMA suggests that ICBT can lead to major reductions in symptoms and that more than half of clients, on average, respond to treatment. A lower proportion remits; thus, there is room for improvement. It is possible that women benefit more from ICBT based on our findings, and symptom severity seems to be predictive of outcome, but these findings should be interpreted with caution. Our findings that studies on clients with anxiety disorders were associated with less response is also notable but should be regarded with care. Future IPDMAs should include more trials and should also consider secondary outcomes, such as quality of life.
Data Availability Statement
The datasets analyzed in this manuscript are not publicly available. Requests to access the datasets should be directed to gerhard.andersson@liu.se.
Author Contributions
All of the authors contributed in the process of completing the current study and writing the final manuscript. AR conducted the aggregation of raw scores into a single data matrix and completed the statistical analysis with input from GA and PC. GA drafted the first version of the text and did additional calculations, while AR and PC provided feedback and reviewed and revised the manuscript.
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Acknowledgments
The authors would like to thank all of the students, researchers, clinicians, and administrative staff who, over the years, have been involved in the clinical trials that were included in the individual patient data meta-analysis. GA also thanks Linköping University for support in the form of a professor contract. The code to analyse the data using jamovi can be obtained found in the online supplement.
Supplementary Material
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyt.2019.00749/full#supplementary-material
References
(* References marked with an asterisk indicate studies included in the meta-analysis.)
1. Andersson G. Internet interventions: past, present and future. Internet Interv (2018) 12:181–8. doi: 10.1016/j.invent.2018.03.008
2. Vlaescu G, Alasjö A, Miloff A, Carlbring P, Andersson G. Features and functionality of the Iterapi platform for internet-based psychological treatment. Internet Interv (2016) 6:107–14. doi: 10.1016/j.invent.2016.09.006
3. Andersson G. The internet and CBT: a clinical guide. Boca Raton: CRC Press (2015). doi: 10.1201/b13645
4. Baumeister H, Reichler L, Munzinger M, Lin J. The impact of guidance on Internet-based mental health interventions—a systematic review. Internet Interv (2014) 1:205–15. doi: 10.1016/j.invent.2014.08.003
5. Carlbring P, Andersson G, Cuijpers P, Riper H, Hedman-Lagerlöf E. Internet-based vs. face-to-face cognitive behavior therapy for psychiatric and somatic disorders: an updated systematic review and meta-analysis. Cogn Behav Ther (2018) 47:1–18. doi: 10.1080/16506073.2017.1401115
6. Andersson G, Rozental A, Shafran R, Carlbring P. Long-term effects of internet-supported cognitive behavior therapy. Expert Rev Neurother (2018) 18:21–8. doi: 10.1080/14737175.2018.1400381
7. Andersson G, Hedman E. Effectiveness of guided Internet-delivered cognitive behaviour therapy in regular clinical settings. Verhaltenstherapie (2013) 23:140–8. doi: 10.1159/000354779
8. Vigerland S, Lenhard F, Bonnert M, Lalouni M, Hedman E, Ahlen J, et al. Internet-delivered cognitive behavior therapy for children and adolescents: a systematic review and meta-analysis. Clin Psychol Rev (2016) 50:1–10. doi: 10.1016/j.cpr.2016.09.005
9. Andersson G, Carlbring P, Titov N, Lindefors N (2019). Internet interventions for adults with anxiety and mood disorders: A narrative umbrella review of recent meta-analyses. Can J Psychiatry 64: 465–470. doi: 10.1177/0706743719839381
10. Dear BF, Zou JB, Ali S, Lorian CN, Johnston L, Sheehan J, et al. Clinical and cost-effectiveness of therapist-guided internet-delivered cognitive behavior therapy for older adults with symptoms of anxiety: a randomized controlled trial. Behav Ther (2015b) 46:206–17. doi: 10.1016/j.beth.2014.09.007
11. Nygren T, Brohede D, Koshnawa K, Osmana SS, Johansson R, Andersson G. Internet-based treatment of depressive symptoms in a Kurdish population: a randomized controlled trial. J Clin Psychol (2019) 75:985–98. doi: 10.1002/jclp.22753
12. Rozental A, Forsell E, Svensson A, Andersson G, Carlbring P. Internet-based cognitive behavior therapy for procrastination: a randomized controlled trial. J Consult Clin Psychol (2015) 83:808–24. doi: 10.1037/ccp0000023
13. Sijbrandij M, Kunovski I, Cuijpers P. Effectiveness of internet-delivered cognitive behavioral therapy for posttraumatic stress disorder: a systematic review and meta-analysis. Depress Anxiety (2016) 33:783–91. doi: 10.1002/da.22533
14. Păsărelu C, Andersson G, Bergman Nordgren L, Dobrean A. Internet-delivered transdiagnostic and tailored cognitive behavioral therapy for anxiety and depression: a systematic review and meta-analysis. Cogn Behav Ther (2017) 46:1–28. doi: 10.1080/16506073.2016.1231219
15. Dear BF, Staples LG, Terides MD, Karin E, Zou J, Johnston L, et al. Transdiagnostic versus disorder-specific and clinician-guided versus self-guided internet-delivered treatment for generalized anxiety disorder and comorbid disorders: a randomized controlled trial. J Anxiety Disord (2015a) 36:63–77. doi: 10.1016/j.janxdis.2015.09.003
16. Johansson R, Hesslow T, Ljotsson B, Jansson A, Jonsson L, Färdig S, et al. Internet-based affect-focused psychodynamic therapy for social anxiety disorder: a randomized controlled trial with 2-year follow-up. Psychotherapy (2017) 54:351–60. doi: 10.1037/pst0000147
17. Donker T, Bennett K, Bennett A, Mackinnon A, van Straten A, Cuijpers P, et al. Internet-delivered interpersonal psychotherapy versus internet-delivered cognitive behavioral therapy for adults with depressive symptoms: randomized controlled noninferiority trial. J Med Internet Res (2013) 15:e82. doi: 10.2196/jmir.2307
18. Dahlin M, Andersson G, Magnusson K, Johansson T, Sjögren J, Håkansson A, et al. Internet-delivered acceptance-based behaviour therapy for generalized anxiety disorder: a randomized controlled trial. Behav Res Ther (2016) 77:86–95. doi: 10.1016/j.brat.2015.12.007
19. Boettcher J, Berger T, Renneberg B. Internet-based attention training for social anxiety: a randomized controlled trial. Cogn Ther Res (2012) 36:552–36. doi: 10.1007/s10608-011-9374-y
20. *Ström M, Uckelstam C-J, Andersson G, Hassmén P, Umefjord G, Carlbring P. Internet-delivered therapist-guided physical activity for mild to moderate depression: a randomized controlled trial. PeerJ (2013) 1:e178. doi: 10.7717/peerj.178
21. *Boettcher J, Åstrom V, Pahlsson D, Schenstrom O, Andersson G, Carlbring P. Internet-based mindfulness treatment for anxiety disorders: a randomized controlled trial. Behav Ther (2014) 45:241–53. doi: 10.1016/j.beth.2013.11.003
22. Carlbring P, Ekselius L, Andersson G. Treatment of panic disorder via the Internet: a randomized trial of CBT vs. applied relaxation. J Behav Ther Exp Psychiatry (2003) 34:129–40. doi: 10.1016/S0005-7916(03)00026-0
23. *Paxling B, Almlöv J, Dahlin M, Carlbring P, Breitholtz E, Eriksson T, et al. Guided internet-delivered cognitive behavior therapy for generalized anxiety disorder: a randomized controlled trial. Cogn Behav Ther (2011) 40:159–73. doi: 10.1080/16506073.2011.576699
24. Hedman E, Andersson E, Andersson G, Lindefors N, Lekander M, Rück C, et al. Mediators in Internet-based cognitive behavior therapy for severe health anxiety. PLoS One (2013) 8:e77752. doi: 10.1371/journal.pone.0077752
25. Bendelin N, Hesser H, Dahl J, Carlbring P, Zetterqvist Nelson K, Andersson G. Experiences of guided Internet-based cognitive-behavioural treatment for depression: a qualitative study. BMC Psychiatry (2011) 11:107. doi: 10.1186/1471-244X-11-107
26. Rozental A, Magnusson K, Boettcher J, Andersson G, Carlbring P. For better or worse: an individual patient data meta-analysis of deterioration among participants receiving Internet-based cognitive behavior therapy. J Consult Clin Psychol (2017) 85:160–77. doi: 10.1037/ccp0000158
27. Rozental A, Andersson G, Carlbring P. In the absence of effects: an individual patient data meta-analysis of non-response and its predictors in internet-based cognitive behavior therapy. Front Psychol (2019) 10:589. doi: 10.3389/fpsyg.2019.00589
28. Loerinc AG, Meuret AE, Twohig MP, Rosenfield D, Bluett EJ, Craske MG. Response rates for CBT for anxiety disorders: need for standardized criteria. Clin Psychol Rev (2015) 42:72–82. doi: 10.1016/j.cpr.2015.08.004
29. Jacobson NS, Truax P. Clinical significance: a statistical approach to defining meaningful change in psychotherapy research. J Consult Clin Psychol (1991) 59:12–9. doi: 10.1037/0022-006X.59.1.12
30. Karyotaki E, Ebert DD, Donkin L, Riper H, Twisk J, Burger S, et al. Do guided Internet-based interventions result in clinically relevant changes for patients with depression? An individual participant data meta-analysis. Clin Psychol Rev (2018) 63:80–92. doi: 10.1016/j.cpr.2018.06.007
31. Stewart L, Tierney JF. To IPD or not to IPD? Advantages and disadvantages of systematic reviews using individual patient data. Eval Health Prof (2002) 25(1):76–97. doi: 10.1177/0163278702025001006
32. Simmonds MC, Higgins JPT, Stewart LA, Tierney JF, Clarke MJ, Thompson SG. Meta-analysis of individual patient data from randomized trials: a review of methods used in practice. Clin Trials (2005) 2(3):209–17. doi: 10.1191/1740774505cn087oa
33. Bower P, Kontopantelis E, Sutton AP, Kendrick T, Richards D, Gilbody S, et al. Influence of initial severity of depression on effectiveness of low intensity interventions: meta-analysis of individual patient data. Br Med J (2013) 346:f540. doi: 10.1136/bmj.f540
34. Riper H, Hoogendoorn A, Cuijpers P, Karyotaki E, Boumparis N, Pastor AM, et al. Effectiveness and treatment moderators of internet interventions for adult problem drinking: an individual patient data meta-analysis of 19 randomised controlled trials. Plos Med (2018) 15:e1002714. doi: 10.1371/journal.pmed.1002714
35. Mukaka M, White SA, Terlouw DJ, Mwapasa V, Kalilani-Phiri L, Faragher EB. Is using multiple imputation better than complete case analysis for estimating a prevalence (risk) difference in randomized controlled trials when binary outcome observations are missing? Trials (2016) 17:341. doi: 10.1186/s13063-016-1473-3
36. Akl EA, Kahale LA, Agoritsas T, Brignardello-Petersen R, Busse JW, Carrasco-Labra A, et al. Handling trial participants with missing outcome data when conducting a meta-analysis: a systematic survey of proposed approaches. Syst Rev (2015) 4:98–8. doi: 10.1186/s13643-015-0083-6
37. Jacobson NS, Roberts LJ, Berns SB, McGlinchey JB. Methods for defining and determining the clinical significance of treatment effects: description, application, and alternatives. J Consult Clin Psychol (1999) 67:300–7. doi: 10.1037/0022-006X.67.3.300
38. Evans C, Margison F, Barkham M. The contribution of reliable and clinically significant change methods to evidence-based mental health. Evid-Based Ment Health (1998) 1(3):70–2. doi: 10.1136/ebmh.1.3.70
39. Saemundsson BR, Borsdottir F, Kristjansdottir H, Olason DB, Smari J, Sigurdsson JF. Psychometric properties of the Icelandic version of the Beck Anxiety Inventory in a clinical and a student population. Eur J Psychol Assess (2011) 27:133–41. doi: 10.1027/1015-5759/a000059
40. Heeren A, Maurange P, Rossignol M, Vanhaelen M, Peschard V, Eeckhout C, et al. Self-report version of the Liebowitz Social Anxiety Scale: psychometric properties of the French version. Can J Behav Sci (2012) 44:99–107. doi: 10.1037/a0026249
41. Lee EH, Kim JH, Yu BH. Reliability and validity of the self-report version of the Panic Disorder Severity Scale in Korea. Depress Anxiety (2009) 26:E120–3. doi: 10.1002/da.20461
42. Zuithoff NPA, Vergouwe Y, Kling M, Nazareth I, van Wezep MJ, Moons KGM, et al. The Patient Health Questionnaire-9 for detection of major depressive disorder in primary care: consequences of current thresholds in a crossectional study. BMC Fam Pract (2010) 11:98. doi: 10.1186/1471-2296-11-98
43. Rosen RC, Riley A, Wagner G, Osterloh IH, Kirkpatrick J, Mishra A. The international index of erectile function (IIEF): a multidimensional scale for assessment of erectile dysfunction. Urology (1997) 49:822–30. doi: 10.1016/S0090-4295(97)00238-0
44. Beck AT, Steer RA. Beck depression inventory. Manual, Svensk version (Swedish version). Fagernes: Psykologiförlaget AB (1996). doi: 10.1037/t00742-000
45. Sundin EC, Horowitz MJ. Impact of Event Scale: psychometric properties. Br J Psychiatry (2002) 180:205–9. doi: 10.1192/bjp.180.3.205
46. Spitzer RL, Kroenke K, Williams JB, Löwe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med (2006) 166:1092–7. doi: 10.1001/archinte.166.10.1092
47. Pallesen S, Nordhus IH, Carlstedt B, Thayer JF, Johnsen TB. A norwegian adaption of the penn state worry questionnaire: factor structure, reliability, validity and norms. Scand J Psychol (2006) 47:281–91. doi: 10.1111/j.1467-9450.2006.00518.x
48. Arrindell WA. The fear of fear concept: stability, retest artifact and predictive power. Behav Res Ther (1993) 31:139–48. doi: 10.1016/0005-7967(93)90065-3
49. Carey MP, Spector IP, Lantinga LJ, Krauss DJ. Reliability of the Dyadic Adjustment Scale. Psychol Assess (1993) 5:238–40. doi: 10.1037/1040-3590.5.2.238
50. Klorman R, Weerts TC, Hastings JE, Melamed BG, Lang PJ. Psychometric properties of some specific-fear questionnaires. Behav Ther (1974) 5:401–9. doi: 10.1016/S0005-7894(74)80008-0
51. Fantino B, Moore N. The self-reported Montgomery-Åsberg Depression rating scale is a useful evaluative tool in major depressive disorder. BMC Psychiatry (2009) 9:26. doi: 10.1186/1471-244X-9-26
52. Muris P, Merckelbach H. A comparison of two spider fear questionnaires. J Behav Ther Exp Psychiatry (1996) 27:241–4. doi: 10.1016/S0005-7916(96)00022-5
53. Gerstein DR, Volberg RA, Toce MT, Harwood H, Johnson RA, Buie T, et al. Gambling Impact and Behavior Study: Report to the National Gambling Impact Study Commission. Chicago, IL: NORC, at the University of Chicago (1999).
54. Hedman E, Andersson E, Ljótsson B, Andersson G, Andersson EM, Schalling M, et al. Clinical and genetic outcome determinants of Internet- and group-based cognitive behavior therapy for social anxiety disorder. Acta Psychiatr Scand (2012) 126:126–36. doi: 10.1111/j.1600-0447.2012.01834.x
55. jamovi project. jamovi (Version 0.9) [Computer Software] (2018). Retrieved from http://www.jamovi.org.
56. First MB, Gibbon M, Spitzer RL, Williams JB. Structured clinical interview for DSM-IV Axis I Disorders (SCID-I). Washington, D.C: American Psychiatric Press (1997).
57. Sheehan DV, Lecrubier Y, Sheehan KH, Amorim P, Janavs J, Weiller E, et al. The Mini-International Neuropsychiatric Interview (MINI): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. J Clin Psychiatry (1998) 59:22–33. doi: 10.1037/t18597-000
58. Blake DD, Weathers FW, Nagy LM, Kaloupek DG, Gusman FD, Charney DS, et al. The development of a clinician-administered PTSD scale. J Trauma Stress (1995) 8(1):75–90.
59. *Ivanova E, Lindner P, Ly KH, Dahlin M, Vernmark K, Andersson G, et al. Guided and unguided Acceptance and Commitment Therapy for social anxiety disorder and/or panic disorder provided via the Internet and a smartphone application: a randomized controlled trial. J Anxiety Disord (2016) 44:27–35. doi: 10.1016/j.janxdis.2016.09.012
60. Nyström MBT, Stenling A, Sjöström E, Neely G, Lindner P, Hassmén P, et al. Behavioral activation versus physical activity via the Internet: a randomized controlled trial. J Affect Disord (2017) 215:85–93. doi: 10.1016/j.jad.2017.03.018
61. *Andersson E, Walen C, Hallberg J, Paxling B, Dahlin M, Almlov J, et al. A randomized controlled trial of guided Internet-delivered cognitive behavioral therapy for erectile dysfunction. J Sex Med (2011) 8:2800–9. doi: 10.1111/j.1743-6109.2011.02391.x
62. *Boettcher J, Magnusson K, Marklund A, Berglund E, Blomdahl R, Braun U, et al. Adding a smartphone app to Internet-based self-help for social anxiety: a randomized controlled trial. Comput Hum Behav (2018) 87:98–108. doi: 10.1016/j.chb.2018.04.052
63. Andersson G, Hesser H, Veilord A, Svedling L, Andersson F, Sleman O, et al. Randomised controlled non-inferiority trial with 3-year follow-up of internet-delivered versus face-to-face group cognitive behavioural therapy for depression. J Affect Disord (2013a) 151:986–94. doi: 10.1016/j.jad.2013.08.022
64. *Carlbring P, Hägglund M, Luthström A, Dahlin M, Kadowaki Å, Vernmark K, et al. Internet-based behavioral activation and acceptance-based treatment for depression: a randomized controlled trial. J Affect Disord (2013) 148:331–7. doi: 10.1016/j.jad.2012.12.020
65. *Ivarsson D, Blom M, Hesser H, Carlbring P, Enderby P, Nordberg R, et al. Guided Internet-delivered cognitive behaviour therapy for post-traumatic stress disorder: a randomized controlled trial. Internet Interv (2014) 1:33–40. doi: 10.1016/j.invent.2014.03.002
66. *Vernmark K, Lenndin J, Bjärehed J, Carlsson M, Karlsson J, Öberg J, et al. Internet administered guided self-help versus individualized e-mail therapy: a randomized trial of two versions of CBT for major depression. Behav Res Ther (2010) 48(5):368–76. doi: 10.1016/j.brat.2010.01.005
67. *Carlbring P, Maurin L, Törngren C, Linna E, Eriksson T, Sparthan E, et al. Individually-tailored, Internet-based treatment for anxiety disorders: a randomized controlled trial. Behav Res Ther (2011) 49(1):18–24. doi: 10.1016/j.brat.2010.10.002
68. *Nordgren LB, Hedman E, Etienne J, Bodin J, Kadowaki A, Eriksson S, et al. Effectiveness and cost-effectiveness of individually tailored Internet-delivered cognitive behavior therapy for anxiety disorders in a primary care population: a randomized controlled trial. Behav Res Ther (2014) 59:1–11. doi: 10.1016/j.brat.2014.05.007
69. *Andersson G, Paxling B, Roch-Norlund P, Östman G, Norgren A, Almlöv J, et al. Internet-based psychodynamic versus cognitive behavioral guided self-help for generalized anxiety disorder: a randomized controlled trial. Psychother Psychosom (2012) 81:344–55. doi: 10.1159/000339371
70. *Carlbring P, Nilsson-Ihrfelt E, Waara J, Kollenstam C, Buhrman M, Kaldo V, et al. Treatment of panic disorder: live therapy vs. self-help via the Internet. Behav Res Ther (2005) 43(10):1321–33. doi: 10.1016/j.brat.2004.10.002
71. Andersson G, Waara J, Jonsson U, Malmaeus F, Carlbring P, Öst L-G. Internet-Based exposure treatment versus one-session exposure treatment of snake phobia: A randomized controlled trial. Cogn Behav Ther (2013b) 42:284–91. doi: 10.1080/16506073.2013.844202
72. *Boettcher J, Hasselrot J, Sund E, Andersson G, Carlbring P. Combining attention training with Internet-based cognitive-behavioural self-help for social anxiety: a randomised controlled trial. Cogn Behav Ther (2013) 43:34–48. doi: 10.1080/16506073.2013.809141
73. *Kuckertz JM, Gildebrant E, Liliequist B, Karlstrom P, Vappling C, Bodlund O, et al. Moderation and mediation of the effect of attention training in social anxiety disorder. Behav Res Ther (2014) 53:30–40. doi: 10.1016/j.brat.2013.12.003
74. *Dagöö J, Asplund RP, Bsenko HA, Hjerling S, Holmberg A, Westh S, et al. Cognitive behavior therapy versus interpersonal psychotherapy for social anxiety disorder delivered via smartphone and computer: a randomized controlled trial. J Anxiety Disord (2014) 28:410–7. doi: 10.1016/j.janxdis.2014.02.003
75. *Andersson G, Carlbring P, Holmstrom A, Sparthan E, Furmark T, Nilsson-Ihrfelt E, et al. Internet-based self-help with therapist feedback and in vivo group exposure for social phobia: a randomized controlled trial. J Consult Clin Psychol (2006) 74(4):677–86. doi: 10.1037/0022-006X.74.4.677
76. *Carlbring P, Gunnarsdóttir M, Hedensjö L, Andersson G, Ekselius L, Furmark T. Treatment of social phobia from a distance: a randomized trial of internet delivered cognitive behavior therapy (CBT) and telephone support. Br J Psychiatry (2007) 190:123–8. doi: 10.1192/bjp.bp.105.020107
77. *Tillfors M, Carlbring P, Furmark T, Lewenhaupt S, Spak M, Eriksson A, et al. Treating university students with social phobia and public speaking fears: Internet delivered self-help with or without live group exposure sessions. Depress Anxiety (2008) 25:708–17. doi: 10.1002/da.20416
78. *Furmark T, Carlbring P, Hedman E, Sonnenstein A, Clevberger P, Bohman B, et al. Bibliotherapy and Internet delivered cognitive behaviour therapy with additional guidance for social anxiety disorder: randomised controlled trial. Br J Psychiatry (2009) 195:440–7. doi: 10.1192/bjp.bp.108.060996
79. *Silfvernagel K, Westlinder A, Andersson S, Bergman K, Hernandez RD, Fallhagen L, et al. Individually tailored internet-based cognitive behaviour therapy for older adults with anxiety and depression: a randomised controlled trial. Cogn Behav Ther (2018) 47:286–300. doi: 10.1080/16506073.2017.1388276
80. *Andersson G, Waara J, Jonsson U, Malmaeus F, Carlbring P, Öst LG. Internet-based self-help versus one-session exposure in the treatment of spider phobia: a randomized controlled trial. Cogn Behav Ther (2009) 38:114–20. doi: 10.1080/16506070902931326
81. *Carlbring P, Degerman N, Jonsson J, Andersson G. Internet-based treatment of pathological gambling with a three-year follow-up. Cogn Behav Ther (2012) 41:321–34. doi: 10.1080/16506073.2012.689323
82. Gyani A, Shafran R, Layard R, Clark DM. Enhancing recovery rates: lessons from year one of IAPT. Behav Res Ther (2013) 51:597–606. doi: 10.1016/j.brat.2013.06.004
83. Springer KS, Levy HC, Tolin DF. Remission in CBT for adult anxiety disorders: a meta-analysis. Clin Psychol Rev (2018) 61:1–8. doi: 10.1016/j.cpr.2018.03.002
84. Titov N, Dear BF, Staples L, Bennett-Levy J, Klein B, Rapee RM, et al. The first 30 months of the MindSpot Clinic: Evaluation of a national e-mental health service against project objectives. Aust N Z J Psychiatry (2017) 51:1227–39. doi: 10.1177/0004867416671598
85. Karyotaki E, Kleiboer A, Smit F, Turner DT, Mira Pastor A, Andersson G, et al. Predictors of treatment dropout in self-guided web-based interventions for depression: an ‘‘individual patient data’’ meta-analysis. Psychol Med (2015) 45:2717–26. doi: 10.1017/S0033291715000665
86. Oxman AD, Clarke MJ, Stewart LA. From science to practice: meta-analyses using individual patient data are needed. JAMA (1995) 274(10):845–6. doi: 10.1001/jama.1995.03530100085040
87. Andersson G, Bergström J, Buhrman M, Carlbring P, Holländare F, Kaldo V, et al. Development of a new approach to guided self-help via the Internet. Swedish Exp J Technol Hum Serv (2008) 26:161–81. doi: 10.1080/15228830802094627
88. Frisch MB, Cornell J, Villanueva M, Retzlaff PJ. Clinical validation of the quality of life inventory: a measure of life satisfaction for use in treatment planning and outcome assessment. Psychol Assess (1992) 4:92–101. doi: 10.1037/1040-3590.4.1.92
Keywords: response rates, recovery, predictors, individual patient data meta-analysis, internet-based cognitive behavior therapy
Citation: Andersson G, Carlbring P and Rozental A (2019) Response and Remission Rates in Internet-Based Cognitive Behavior Therapy: An Individual Patient Data Meta-Analysis. Front. Psychiatry 10:749. doi: 10.3389/fpsyt.2019.00749
Received: 10 June 2019; Accepted: 18 September 2019;
Published: 25 October 2019.
Edited by:
Lina Gega, University of York, United KingdomCopyright © 2019 Andersson, Carlbring and Rozental. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Gerhard Andersson, R2VyaGFyZC5hbmRlcnNzb25AbGl1LnNl