Suicidal ideation following internet-delivered tailored CBT for depression – a secondary analysis of a factorial design trial

Introduction: Suicidal ideation is common in major depressive disorder (MDD) and a risk factor for suicidal behavior. Although it can be reduced with psychological treatments, the risks often make clinicians hesitant to offer treatment. This concerns remote treatment options, such as internet-delivered cognitive behavior therapy (ICBT), which may be considered unsafe. Although previous studies indicate that ICBT can reduce self-reported suicidal ideation both as primary and indirect treatment target, questions remain about under what circumstances ICBT can be offered as the primary treatment. In this secondary report, we investigated the importance of different treatment factors in reducing suicidal ideation via ICBT, including different kinds of therapist support.

Methods: We analyzed data from 197 participants from a factorial trial of ICBT for symptoms of MDD. Before inclusion all participants completed a structured clinical interview where obvious suicidal intent lead to exclusion. Suicidal ideation was assessed at pre- and posttreatment using one item of the PHQ-9 and one from BDI-II. The data were analyzed using generalized linear models.

Results: The pre- to posttreatment comparisons showed decreases in the reporting of suicidal ideation. Findings were consistent across the two measures that was used. There was no effect of support format and content tailoring.

Conclusions: The findings suggest that ICBT can help alleviate suicidal ideation even when it is not the focus of the treatment. This was the case regardless of mode of therapist support, who tailored the treatment content, and if case supervision was available or not.

1 Introduction

Suicidal ideation (SI) is a common symptom among those suffering from major depressive disorder (1), and serves as a risk factor for subsequent suicide attempts (2). Because of the risks linked to SI, it has been described as an essential intervention target (3).

Psychological treatments are an option for decreasing SI and suicidal behaviors, both as a direct target and indirectly via treating other forms of psychopathology (e.g., major depressive disorder). Méndez-Bustos et al. (4) noted that a range of different psychotherapeutic approaches have been used to reduce SI and other suicidal outcomes, including Cognitive Behavior Therapy (CBT), Dialectal Behavior Therapy (DBT), and Interpersonal Psychotherapy (IPT). CBT is based on altering behaviors and one’s stance towards thoughts and internal sensations as a means of treating psychopathology (5). In the treatment of SI, CBT relies on strategies such as challenging biased thinking and suicidal cognitions, as well as behavioral strategies such as relaxation techniques (6). A later umbrella review (7) reported a significant, albeit small, average reduction in SI following CBT with the authors mentioning scalability as an important advantage. However, some questions remain. For example, Mewton and Andrews (6) noted that there is still insufficient evidence that CBT reduces SI when it serves as an indirect target. Because of the inconclusive evidence up to this point, additional studies are needed.

Another consideration when treating SI is how the treatment can be delivered in an effective, safe, and accessible manner. Internet-delivered cognitive behavioral therapy (ICBT) can be useful for this purpose as it is more easily accessible than alternative contact modes, can be offered anonymously, and can reduce concerns about stigma associated with SI (8). ICBT is a form of self-help, with psychoeducation and practical strategies delivered via a web platform or an app (9). The treatment content is often delivered in a modular fashion, much like how regular CBT sessions cover different topics and assignments over the course of a treatment. It is frequently delivered with asynchronous guidance from a clinician. Among the advantages are that ICBT and internet interventions in general can reduce costs and bridge distances to specialist clinics. However, ICBT also comes with challenges, such as the lack of face-to-face contact with a clinician. Due to this, clinicians may be reluctant to offer ICBT in the presence of SI, despite the support for ICBT when treating several mental and somatic health problems (10). More specifically, internet-delivered cognitive behavioral therapy (ICBT) is effective for reducing symptoms of depression (11), and can be as effective as face-to-face CBT (12). In relation to parasuicidal outcomes, ICBT has been found to produce small but statistically significant reductions in suicidal ideation when it serves as the primary target for the treatment (13). Importantly, results of some trials investigating ICBT for depression have reported reductions in SI, suggesting that the treatment does not need to target SI directly (14, 15). Rather, in cases with less severe and persistent forms of SI, ICBT could be a viable treatment option that addresses both SI and other symptoms of depression. However, not all studies show secondary benefits of ICBT for depression on measures of SI. Helen et al. (16) did not find within-group reductions of SI following ICBT, regardless of whether the participants had received therapist support or not. Additional studies would be helpful to help clarify the effects of ICBT for SI, in particular when SI is not the primary treatment target.

Another consideration is whether there are factors related the treatment that can improve the effects and make it more suitable for certain populations. One example is the presence of weekly therapist support that has been linked to better treatment outcomes and better adherence compared to not having such support available on a weekly basis (17). Another example from an open study and two factorial design trials is the choice of treatment content and allowing participants to choose the content themselves compared against therapists making the choice (18–20). Investigations into factors such as these could be of importance from a decision-making standpoint as they offer information about what kind of treatment structure that offers the best chance of improvement for people seeking help.

The current study sought to expand on previous studies investigating the efficacy of internet-delivered CBT for depression in reducing suicidal ideation. More specifically, the primary aim of this secondary report was to investigate both the general effects of ICBT on SI, but also other factors relevant to the effects of ICBT for SI. These included the kind of therapist-guidance that was offered, decisions on treatment content, and the availability for case supervision for therapists guiding the participants through the treatment. Based on prior studies of similar treatments, it was hypothesized that suicidal ideation would decrease overall during the treatment. For the treatment factors, we wanted to explore whether the on-demand approach to guidance produced similar reductions in SI to the standard way of providing weekly guidance that is often used in regular care settings. Likewise, we were interested in whether participants receiving the option to tailor their own treatment had similar reductions in SI compared to the usual way of having the therapist tailor the content of the intervention. Lastly, we were interested in whether the availability of supervision for the therapists was important for the treatment’s ability to reduce SI. The analyses of treatment factors were conducted in an exploratory manner with no directional hypotheses regarding which condition that would perform better.

2 Materials and methods

The current study was a secondary analysis of a trial that has been described in detail in a previous article (19). The project received ethical approval from the regional ethics board. All participants provided informed consent before registering for the study.

2.1 Design

The trial was a randomized factorial trial in which the included participants were randomly assigned to one of eight conditions based on three different factors/independent variables. First, participant could receive either regular, once-weekly therapist support or have the option to request therapist support (referred to as on-demand support). Second, participants could either have the content of the intervention tailored for them by a therapist, or tailor it themselves. Third, the therapist assigned to the participant could either be eligible to receive supervision for the case or not. All participants began treatment at the same time but had different combinations of the independent variables in a balanced factorial design.

The treatment lasted for 10 weeks. During this time, participants received access to modules covering psychoeducation and strategies for dealing with depression and comorbid anxiety disorders (e.g., panic disorder), and transdiagnostic problems (e.g., perfectionism). The full content of the program is described in the original article (19). Treatment modules included exercises specific to depression (e.g., behavioral activation), more generic and transdiagnostic CBT principles (e.g., cognitive restructuring, gradual exposure, applied relaxation), and problem-specific treatment components (e.g., insomnia, stress). None of the modules dealt with suicidal ideation explicitly. Participants in the therapist-tailored condition received eight modules on average, while those with self-tailored content could select as many or as few modules as they wanted (but were instructed that eight modules was the average pace).

2.2 Recruitment, inclusion, and exclusion criteria

Detailed information about the recruitment procedure is presented elsewhere (19). Briefly, trial recruitment took place via multiple sources, including via social media posts and posters in primary care settings around Sweden. The study information specified that we were looking to recruit participants that experienced low mood and other symptoms of major depressive disorder. Prospective participants would register on the study website and fill in a screening. They would then complete a structured clinical interview (MINI 7.0; 21) before a decision on inclusion/exclusion was made. The inclusion criteria stated that the participant had to 1) be 18 years old or older, 2) meet the criteria for major depressive disorder or unspecified depressive disorder, 3) have elevated symptoms of major depressive disorder on the Patient Health Questionnaire-9 (PHQ-9; a sum score of at least 5) and/or Beck Depression Inventory, version two (BDI-II; a sum score of at least 10), 4) have access to the internet and a device with a web browser, 5) sufficient proficiency in Swedish, 6) no other ongoing psychotherapy and, if using psychotropic medication, a stable dose for the past three months. Participants were excluded if they a) had an active substance use problem, b) had severe psychiatric problems (e.g., anorexia nervosa) that could not be managed within the framework of the study, or c) expressed suicidal plans or preparations. The last criterion was assessed using the questions regarding suicidal ideation and suicide on the MINI 7.0. In total, seven participants were excluded for this reason. A total of 197 participants were included. All included participants were randomized by an independent party not involved in other parts of the project. A complete flowchart of the recruitment process can be seen in Figure 1 of the article reporting on the primary outcomes (19).

Figure 1

Figure 1. Proportion of Participants with Suicidal Ideation Before and After Treatment.

2.3 Outcome measures

Two items were used to assess the prevalence and frequency of suicidal ideation. First, we used item nine from the Patient Health Questionnaire (PHQ-9; 22) which asks the respondent to indicate how often they have been bothered by “thoughts that you would be better off dead or of hurting yourself in some way?”. Responses are given on a four-point scale with the alternatives being Not at all (scored as 0), Several days (1), More than half of the days (2), and Nearly every day (3). Second, we also used an item from the second iteration of the Beck depression inventory (BDI-II; 23) which asks the participant to indicate the prevalence and severity of suicidal ideation and suicidal intent. The responses given are either I do not have any thoughts of killing myself (scored as 0), I have thoughts of killing myself, but I would not carry them out (1), I would like to kill myself (2), and I would like to kill myself if I had the chance (3). Both measures were administered before and after the active treatment phase.

2.4 Statistical analysis

Statistical analyses were conducted using R (24). The intention-to-treat was used across all analyses. An alpha level of.05 was used for statistical inferences and 95% confidence intervals are reported. A Fisher’s exact test was used to test the relationship between the presence of suicidal ideation at the pre-treatment timepoint and the likelihood of having missing data at the posttreatment timepoint.

To assess the change in suicidal ideation during the treatment phase we specified two generalized linear models using the glm function in R. For each model, we were interested in the change over time (from pre-treatment to posttreatment), and the interaction between time and each of the three factors outlined above (who tailored the treatment, which kind of support that was provided, and if case supervision was allowed for the specific participant). The factors were coded as: -0.5 = therapist-tailored content, 0.5 = self-tailored content, -0.5 = scheduled therapist support, 0.5 = on-demand support, -0.5 = case supervision available, 0.5 = case supervision not available. The models also estimated the higher order interactions. With the non-normal distribution of responses in mind, we specified a logistical regression using the binomial family argument. For the responses on item 9 of PHQ-9, we recoded the few responses with a score of 2 and 3 as a score of 1, thus creating a dichotomous outcome for use in a logistical regression analysis. For item 9 of the BDI-II, the responses were already exclusively scored as 0 or 1, meaning that no recoding was necessary. Across both outcome measures, a score of 0 indicate no suicidal ideation and 1 indicate the presence of suicidal ideation. In addition to these main analyses, we also explored whether gender moderated the general treatment effect (via a time * gender interaction). For these analyses, gender was coded as -0.5 = woman, 0.5 = man.

3 Results

The demographics of the sample are described in Table 1. Overall outcomes from the trial have been summarized in Andersson et al. (19). Missing data at the posttreatment timepoint was not significantly related to the presence of suicidal ideation at pre-treatment, Fisher’s exact p = .211 for item 9 on the BDI-II and .626 for item 9 on the PHQ-9.

Table 1

Table 1. Demographic characteristics of the sample (n = 197).

3.1 Prevalence and change in suicidal ideation

The percentages of participant indicating that they experienced suicidal ideation according to the items of PHQ-9 and BDI-II can be seen in Table 2. Estimates from the generalized linear models are available in Table 3. The prevalence of suicidal ideation in the sample and the change over time is visualized in Figure 1. Both the observed percentages and the model coefficients indicated a reduction in suicidal ideation during the treatment phase. The odds ratios for both outcome measures showed a significantly reduced risk of experiencing suicidal ideation at the end of the treatment compared to before the start of the study. For both the BDI-II item and the PHQ-9 item, neither of the three factors predicted an increase or decrease in suicidal ideation. None of the higher order interactions were statistically significant.

Table 2

Table 2. Percentages of participants in the sample with suicidal ideation at the pre- and posttreatment treatment timepoints.

Table 3

Table 3. Coefficients and odds ratios for the generalized linear models.

3.2 Gender differences for changes in suicidal ideation

The results from the exploratory analyses indicated that the change over time in SI did not differ between women and men according to the BDI-II item, estimate = 0.91 [95% CI -0.34, 2.11], SE = 0.62, p = .145, OR = 2.47 [95% CI 0.73, 8.35]. The same was true for the PHQ-9 item, estimate = 0.95 [95% CI -0.35, 2.20], SE = 0.65, p = .141, OR = 2.58 [95% CI 0.73, 9.14].

4 Discussion

The results in this study are in line with previous trials indicating that suicidal ideation decrease following treatment with CBT more generally (25), and ICBT more specifically (14, 15). The findings from this factorial design trial support the idea that less severe SI can be treated within the context of a more generic version of ICBT for depression without the need for elements tailored specifically for SI. This finding is promising as ICBT has been suggested to fit some important needs of people burdened by SI, including the potential for anonymity and widespread availability (8). It also serves as an additional indication of the usefulness of non-SI specific treatments for reducing SI, which is an approach to this problem for which more evidence is needed (6). It should be noted that severe SI did serve as an exclusion criteria, which means that the potential of the treatment to help populations with more severe symptoms remains unknown. However, in populations with less severe depressive symptoms and SI, the findings support the use of ICBT.

We found no difference in the reductions between participants who received scheduled support versus those who had support on-demand. This is important as it shows that the efficacy of ICBT in reducing is SI may not be dependent a specific support format and that the treatment can be effective when delivered with less therapist-intense forms of guidance. The latter point is reminiscent of the findings reported by Watts et al. (15), who also reported decreases in SI while not utilizing weekly, regular therapist support. The findings from our present study could also be taken as an additional indication of the utility of the on-demand support format in general, which has also been found in a study on social anxiety disorder (26) and in transdiagnostic treatments for symptoms of anxiety and depression (27). Given the comparable efficacy and the additional time-efficiency relative to weekly, regular scheduled support, support on-demand could be of importance in future efforts of implementing and disseminating similar types of treatment.

The lack of a difference in the efficacy between self-tailored and therapist-tailored module content differs from the small but statistically significant effect reported in the original article on BDI-II ratings (19). Overall, this difference could not be expected to be relevant for SI as a module on SI was not included. The reduction of SI in our sample is comparable to other similar interventions that also primarily focus on reducing depressive symptoms, rather than focusing on SI (15). An aspect to investigate in future studies is whether the addition of a module focused specifically on SI could further improve the effects of the intervention.

4.1 Strengths and limitations

The strengths of the study include the use of two outcome measures for measuring suicidal ideation, which allows for more robust conclusions on the reductions in SI. However, the findings should also be interpreted with the methodological limitations in mind. First, although participants were randomized to different conditions, neither of these served as a no treatment/placebo control group. This limits the causal conclusions about the impact of the treatment. The lack of an untreated group to compare against means that we cannot rule out that the observed change in SI was due to factors that did not relate to the treatment, such as a natural decline over time during the treatment period. Other potential explanations include regression to the mean, where extreme values are less likely to occur with repeated sampling which could cause the illusion of a symptom decrease. Finally, the lack of an active control group that receives access to the general treatment procedure also makes it more difficult to conclude that it was the CBT content that reduced SI, and no other non-treatment factors such as contact with a therapist or the measurements procedures during the study. The fact that we did not observe a difference between those that that received weekly support and those that received support on-demand (which was asked for infrequently) do suggest that the support itself might not have been the important ingredient, but other factors could still have played a role. Second, the exclusion criteria also limit the generalizability of the findings as too severe suicidal ideation would have served as an exclusion criterion. Indeed, seven participants in the original trial were excluded due to suicidal intentions (details provided in 19), and it is unknown whether these individuals would have experienced the same reductions in SI during the treatment that we saw for the included participants. Because of this, the conclusions from the present study may not generalize to populations with more severe forms of SI, including plans for suicide.

5 Conclusions

Overall, the results of the trial are in line with previous studies that suggests that ICBT for depressive symptoms reduces the self-reported prevalence of suicidal ideation. This finding supports the notion that less frequent and severe forms of suicidal ideation can be treated within the context of ICBT and should not serve as an exclusion criterion for this kind of treatment. The study also adds to the field by providing an additional indication that this decrease in suicidal ideation is not contingent on scheduled therapist support, but that reductions also occur when the support is provided on-demand. This is encouraging from an implementation standpoint as this support format requires less time, thus potentially allowing for the rolling out of similar treatments even with a scarcity of trained therapists. Neither of the other treatment factors exhibited a significant difference, further supporting the potential for flexible applications of the treatment format.

Data availability statement

The original contributions presented in the study are included in the article/supplementary materials, further inquiries can be directed to the corresponding author/s.

Ethics statement

The studies involving humans were approved by the ethics committee at Linköping University. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.

Author contributions

AK: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Visualization, Writing – original draft, Writing – review & editing. GA: Conceptualization, Investigation, Writing – review & editing.

Funding

The author(s) declare financial support was received for the research, authorship, and/or publication of this article. The study was sponsored by the Swedish Research Council (Grant 2018-02900).

Acknowledgments

We thank the members of the ROBIN Team for their work with the trial. We also wish to thank our webmaster George Vlaescu for his help with the project.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

1. Cai H, Jin Y, Liu S, Zhang Q, Zhang L, Cheung T, et al. Prevalence of suicidal ideation and planning in patients with major depressive disorder: A meta-analysis of observation studies. J Affect Disord. (2021) 293:148–58. doi: 10.1016/j.jad.2021.05.115

PubMed Abstract | Crossref Full Text | Google Scholar

2. Kessler RC, Borges G, Walters EE. Prevalence of and risk factors for lifetime suicide attempts in the national comorbidity survey. Arch Gen Psychiatry. (1999) 56:617–26. doi: 10.1001/archpsyc.56.7.617

PubMed Abstract | Crossref Full Text | Google Scholar

3. Jobes DA, Joiner TE. Reflections on suicidal ideation. Crisis. (2019) 40:227–30. doi: 10.1027/0227-5910/a000615

PubMed Abstract | Crossref Full Text | Google Scholar

4. Méndez-Bustos P, Calati R, Rubio-Ramírez F, Olié E, Courtet P, Lopez-Castroman J. Effectiveness of psychotherapy on suicidal risk: A systematic review of observational studies. Front Psychol. (2019) 10. doi: 10.3389/fpsyg.2019.00277

PubMed Abstract | Crossref Full Text | Google Scholar

5. Dozois DJA, Dobson KS, Rnic K. Historical and philosophical bases of the cognitive-behavioral therapies. In: Dobson KS, Dozois DJA, editors. Handbook of cognitive-behavioral therapies, 4th ed. New York: Guilford Publications (2019).

Google Scholar

6. Mewton L, Andrews G. Cognitive behavioral therapy for suicidal behaviors: improving patient outcomes. Psychol Res Behav Manage. (2016) 9:21–9. doi: 10.2147/PRBM.S84589

Crossref Full Text | Google Scholar

7. Wu H, Lu L, Qian Y, Jin X-H, Yu H-R, Du L, et al. The significance of cognitive-behavioral therapy on suicide: An umbrella review. J Affect Disord. (2022) 317:142–8. doi: 10.1016/j.jad.2022.08.067

PubMed Abstract | Crossref Full Text | Google Scholar

8. Karyotaki E, van Ballegooijen W. Digital self-help interventions for suicidal ideation and behavior. Lancet Digital Health. (2020) 2:e4–5. doi: 10.1016/S2589-7500(19)30215-8

PubMed Abstract | Crossref Full Text | Google Scholar

9. Andersson G. Internet-delivered psychological treatments. Annu Rev Clin Psychol. (2016) 12:157–79. doi: 10.1146/annurev-clinpsy-021815-093006

PubMed Abstract | Crossref Full Text | Google Scholar

10. Andersson G, Titov N, Dear BF, Rozental A, Carlbring P. Internet-delivered psychological treatments: from innovation to implementation. World Psychiatry. (2019) 18:20–8. doi: 10.1002/wps.20610

PubMed Abstract | Crossref Full Text | Google Scholar

11. Karyotaki E, Efthimiou O, Miguel C, Bermpohl FM, Furukawa TA, Cuijpers P, et al. Internet-based cognitive behavioral therapy for depression: A systematic review and individual patient data network meta-analysis. JAMA Psychiatry. (2021) 78:361–71. doi: 10.1001/jamapsychiatry.2020.4364

PubMed Abstract | Crossref Full Text | Google Scholar

12. Hedman-Lagerlöf E, Carlbring P, Svärdman F, Riper H, Cuijpers P, Andersson G. Therapist-supported Internet-based cognitive behavior therapy yields similar effects as face-to-face therapy for psychiatric and somatic disorders: an updated systematic review and meta-analysis. World Psychiatry. (2023) 22:305–14. doi: 10.1002/wps.21088

PubMed Abstract | Crossref Full Text | Google Scholar

13. Büscher R, Beisemann M, Doebler P, Micklitz HM, Kerkhof A, Cuijpers P, et al. Digital cognitive–behavioral therapy to reduce suicidal ideation and behaviors: a systematic review and meta-analysis of individual participant data. Evidence-Based Ment Health. (2022) 25:e8–e17. doi: 10.1136/ebmental-2022-300540

Crossref Full Text | Google Scholar

14. Newby JM, Mewton L, Williams AD, Andrews G. Effectiveness of transdiagnostic internet cognitive behavioral treatment for mixed anxiety and depression in primary care. J Affect Disord. (2014) 165:45–52. doi: 10.1016/j.jad.2014.04.037

PubMed Abstract | Crossref Full Text | Google Scholar

15. Watts S, Newby JM, Mewton L, Andrews G. A clinical audit of changes in suicide ideas with internet treatment for depression. BMJ Open. (2012) 2:e001558. doi: 10.1136/bmjopen-2012-001558

PubMed Abstract | Crossref Full Text | Google Scholar

16. Helen C, Louise F, Philip JB, Andrew M, Kathleen MG, Tara D. The effect of a web-based depression intervention on suicide ideation: secondary outcome from a randomized controlled trial in a helpline. BMJ Open. (2013) 3:e002886. doi: 10.1136/bmjopen-2013-002886

PubMed Abstract | Crossref Full Text | Google Scholar

17. Koelen JA, Vonk A, Klein A, de Koning L, Vonk P, de Vet S, et al. Man vs. machine: A meta-analysis on the added value of human support in text-based internet treatments (“e-therapy”) for mental disorders. Clin Psychol Rev. (2022) 96:102179. doi: 10.1016/j.cpr.2022.102179

PubMed Abstract | Crossref Full Text | Google Scholar

18. Andersson G, Estling F, Jakobsson E, Cuijpers P, Carlbring P. Can the patient decide which modules to endorse? An open trial of tailored internet treatment of anxiety disorders. Cogn Behav Ther. (2011) 40:57–64. doi: 10.1080/16506073.2010.529457

PubMed Abstract | Crossref Full Text | Google Scholar

19. Andersson G, Käll A, Juhlin S, Wahlström C, de Fine Licht E, Färdeman S, et al. Free choice of treatment content, support on demand and supervision in internet-delivered CBT for adults with depression: A randomized factorial design trial. Behav Res Ther. (2023) 162:104265. doi: 10.1016/j.brat.2023.104265

PubMed Abstract | Crossref Full Text | Google Scholar

20. Dahlin M, Johansson A, Romare H, Carlbring P, Andersson G. Worry-specific versus self-tailored internet-based treatments for generalized anxiety disorder, with scheduled support or support on demand: A pilot factorial design trial. Internet Interventions. (2022) 28:100531. doi: 10.1016/j.invent.2022.100531

PubMed Abstract | Crossref Full Text | Google Scholar

21. Sheehan DV, Lecrubier Y, Sheehan KH, Amorim P, Janavs J, Weiller E, et al. The Mini-International Neuropsychiatric Interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. J Clin Psychiatry. (1998) 59 Suppl 20:22–33;quiz 34-57. http://europepmc.org/abstract/MED/9881538.

PubMed Abstract | Google Scholar

22. Kroenke K, Spitzer RL, Williams JBW. The PHQ-9: Validity of a brief depression severity measure. J Gen Internal Med. (2001) 16:606–13. doi: 10.1046/j.1525-1497.2001.016009606.x

Crossref Full Text | Google Scholar

23. Beck AT, Steer RA, Carbin MG. Psychometric properties of the Beck Depression Inventory: Twenty-five years of evaluation. Clin Psychol Rev. (1988) 8:77–100. doi: 10.1016/0272-7358(88)90050-5

Crossref Full Text | Google Scholar

24. R Core Team. R: A language and environment for statistical computing. R Foundation for Statistical Computing (2023). Available at: https://www.R-project.org/.

Google Scholar

25. Weitz E, Hollon SD, Kerkhof A, Cuijpers P. Do depression treatments reduce suicidal ideation? The effects of CBT, IPT, pharmacotherapy, and placebo on suicidality. J Affect Disord. (2014) 167:98–103. doi: 10.1016/j.jad.2014.05.036

PubMed Abstract | Crossref Full Text | Google Scholar

26. Käll A, Olsson Lynch C, Sundling K, Furmark T, Carlbring P, Andersson G. Scheduled support versus support on demand in Internet-delivered cognitive behavioral therapy for social anxiety disorder: Randomized controlled trial. Clin Psychol Europe. (2023) 5:1–22. doi: 10.32872/cpe.11379

Crossref Full Text | Google Scholar

27. Hadjistavropoulos HD, Schneider LH, Mehta S, Karin E, Dear BF, Titov N. Preference trial of internet-delivered cognitive behavior therapy comparing standard weekly versus optional weekly therapist support. J Anxiety Disord. (2019) 63:51–60. doi: 10.1016/j.janxdis.2019.02.002

PubMed Abstract | Crossref Full Text | Google Scholar