Internet-Based Cognitive Behavioral Therapy for Psychological Distress Associated With the COVID-19 Pandemic: A Pilot Randomized Controlled Trial

Background: The COVID-19 pandemic has been associated with various negative psychological consequences. This is a challenge for the society as regular psychological services cannot be offered to the same extent as before the pandemic. In addition to the requirement of social distancing, there is a need to adjust psychological treatment components like exposure to avoid increasing the spread of the infection. Internet-delivered cognitive behavior therapy (ICBT) has an established evidence base for a range of psychiatric problems and has been suggested as one possible approach to deal with the situation. This study aimed to conduct a randomized controlled pilot trial during the summer of 2020 with a broad focus on psychological distress and a treatment approach that tailors the intervention based on symptom profile and preferences.

Methods: Following the advertisement and interview, we included 52 participants with elevated levels of psychological distress. They were randomly allocated to either a 7-week-long individually tailored ICBT (n = 26) or a wait-list control condition (n = 26). Measures of depression and quality of life were used as primary outcomes. We also included secondary outcome measures of anxiety, insomnia, trauma, stress, anger, and alcohol use. For screening, we used the CoRonavIruS Health Impact Survey (CRISIS).

Results: Overall moderate to large between-group effects were found at post-treatment in favor of the treatment on measures of both depression [Beck Depression Inventory (BDI); Cohens d = 0.63; Patient Health Questionnaire (PHQ-9): d = 0.62] and anxiety [Generalized Anxiety Disorder-7-item scale (GAD-7); d = 0.82]. This was also observed for stress symptoms [Perceived Stress Scale (PSS-14); d = 1.04]. No effects were seen on measures of quality of life, insomnia, symptoms of post-traumatic stress, and anger. There was an effect on alcohol use [Alcohol Use Disorder Identification Test (AUDIT); d = 0.54], which was not of clinical relevance.

Conclusion: Individually tailored ICBT shows initial promise as a way to reduce psychological problems in association with the COVID-19 pandemic. A possible limitation was that the trial was conducted when the effects of the pandemic were decreasing and when fewer people were affected by the restrictions (e.g., the summer of 2020).

Introduction

In March 2020, the WHO declared the spread of SARS-CoV-2 as a pandemic (Arden and Chicot, 2020). The related disease, COVID-19, had by then infected more than 1,18,000 people in over 110 countries (Shah et al., 2020). The virus causes respiratory diseases, from a mild to severe degree, and is spread between people (Onyeaka et al., 2020).

A pandemic of an infectious disease affects not only physical but also mental health (Xiao et al., 2020). A global pandemic is likely to lead to fear and worry and influence psychological well-being (Shah et al., 2020). In particular, as a result of being very sudden and highly contagious, the virus causes anxiety, depression, and stress (Horesh and Brown, 2020; Wang et al., 2021). Most of the research on the pandemic has focused on understanding the virus, infection patterns, and physiological symptoms, and in particular to rapidly develop a vaccine. The psychological aspects and suffering have also been focused on, but to a lesser extent (Shah et al., 2020). Given the large costs for society, it has been argued that the indirect negative effects of the pandemic should also be focused on (Wang et al., 2021).

Fear, anxiety, anger, and post-traumatic stress disorder are symptoms that can occur when there is a direct threat of being infected or an indirect threat in response to restrictions in life (Onyeaka et al., 2020). The psychological impact can remain for a long time after the infection, and the spread of the virus is under control (Onyeaka et al., 2020). During the ongoing pandemic, people are urged to keep their distance from each other, isolate themselves, and stay in quarantine. The most fundamental method of dealing with crises, mutual social support, is thereby in some ways not possible (Horesh and Brown, 2020). Quarantine and isolation have been associated with fear (Rubin and Wessley, 2020), acute stress syndrome, depression, post-traumatic stress syndrome, insomnia, irritability, anger, and emotional exhaustion (Jung and Jun, 2020). In the absence of in-person interaction, depression and anxiety tend to occur to a greater extent and even worsen (Xiao, 2020). In turn, stress and anxiety are also associated with reduced sleep quality (Xiao et al., 2020). Quarantine and/or isolation may also lead to boredom, loneliness (Shah et al., 2020; Xiao et al., 2020), and a feeling of losing control (Kessler et al., 2007). Lack of physical activity, in addition to the lack of social interaction, can also lead to increased stress levels (Xiao et al., 2020). In summary, even if individuals who are in quarantine or have isolated themselves have a lower risk for infection and largely maintain their physical well-being, negative psychological effects can emerge (Brooks et al., 2020; Xiao et al., 2020). Moreover, psychological effects seem to be milder when people voluntarily follow the restrictions in comparison with when the restrictions are imposed by authorities (Brooks et al., 2020).

The pandemic may worsen mental health and lead to stress-related problems (Horesh and Brown, 2020). In a study by Wang et al. (2020), half of the participants rated the psychological impact of the virus as moderate to severe, and a third reported moderate to severe anxiety. In a review by Brooks et al. (2020), the authors noted that symptoms of post-traumatic stress disorder, confusion, and anger were the most frequently reported negative effects of quarantine. The impact of quarantine and isolation on a psychological level appears to be extensive and long-lasting, even extending after the quarantine period (Brooks et al., 2020). An associated expected economic downturn has also increased fear and stress (Onyeaka et al., 2020). Many have already faced, or will face, unemployment and economic stress when industries and community services close, which contributes to the negative emotions (Azim et al., 2020). The physical symptoms of the virus, such as cough and fever, can aggravate cognitive suffering and anxiety because of fear of the virus (Shah et al., 2020), and even though a majority of the population may not be affected, media reporting and the experienced risk of becoming infected in the future cause stress and anxiety. Consequently, it is unclear whether the psychological effects are due mostly to medical, social, or economic factors (Horesh and Brown, 2020).

While the WHO and other authorities have focused on the biological and physical aspects (Azim et al., 2020), interventions for mental health in relation to the current situation have been less focused (Shah et al., 2020; Wang et al., 2020; Xiang et al., 2020), even if there have been calls for action (Wind et al., 2020). For both the ongoing work with handling the pandemic and for the parallel work of restoring society following the pandemic, stable mental health is a key factor (Azim et al., 2020; Holmes et al., 2020). Requirements for alternative ways to adapt and work around the crisis come as a result of the need for social distancing (Horesh and Brown, 2020). Internet-delivered cognitive behavior therapy (ICBT) has been investigated and has been found to be effective for a range of psychiatric problems and somatic health problems (Andersson, 2016). ICBT guided by a therapist is as effective as face-to-face treatment (Carlbring et al., 2018) and tends to have larger effects than unguided ICBT-programs (Baumeister et al., 2014). There are several advantages with ICBT in comparison to face-to-face treatment, such as increased availability and cost-effectiveness (Andersson and Titov, 2014).

Individually tailored ICBT has been developed as a way to handle the comorbidity and preferences of the client (Carlbring et al., 2010). Tailoring means that the individual receives different treatment modules depending on symptom presentation, current situation, and preferences (Carlbring et al., 2010; Păsărelu et al., 2017). This means that different clients within a treatment protocol can receive and work with different modules in their treatment. This is in contrast to standardized treatment programs, in which all clients receive the same modules with the same content. Individually tailored ICBT seems to be as effective as standardized internet treatments for depression (Johansson et al., 2012; Kraepelien et al., 2018) and anxiety (Carlbring et al., 2010; Berger et al., 2014; Bergman Nordgren et al., 2014). Choice of treatment modules can be by the therapist in collaboration with the client but can also work when the client makes the selection based on descriptions of the modules (Andersson et al., 2011). It is found that comorbidity between depression and anxiety symptoms is very common (Kessler et al., 2007) and also seems to be the case during the corona pandemic (Wang et al., 2021), and therefore, tailored ICBT treatment could be a way to address this. Psychological interventions, such as CBT, have to be modified to suit the temporary needs of the population during the pandemic. For example, to reduce the risk of infection spreading, CBT should preferably be provided through the internet or telephone. ICBT does not require the presence of a therapist (Wang et al., 2020) and has been proposed as a treatment option during the pandemic (Onyeaka et al., 2020; Wang et al., 2020; Xiang et al., 2020).

The aim of this pilot phase I randomized controlled trial (RCT) was to investigate the effects of individually tailored ICBT with support from a therapist with a focus on psychological symptoms that had occurred or worsened because of the pandemic. Phase I trials are aiming to establish the safety of trials, adverse effects, and information on outcomes by involving small numbers of participants (Umscheid et al., 2011). We addressed symptoms that had developed, or worsened, because of the corona pandemic itself and its consequences, such as social distancing. Except for a trial that found promising effects of a brief ICBT treatment for corona-related worry (Wahlund et al., 2021), this is one of the first trials regarding the effects of ICBT on psychological symptoms linked to the corona pandemic and its consequences.

Materials and Methods

Study Design

The study was an RCT in which participants were randomly allocated either a 7-week-long ICBT or a wait-list control condition. Primary and secondary measures (described more in detail below) were administered in conjunction with the recruitment (pre-treatment) and post-treatment. Participants in the control group received the same treatment a few weeks after post-treatment measurement for the treatment group. The Swedish National Ethics Committee approved the study protocol (Dnr 2020-02313).

Participants and Recruitment

The recruitment began in June 2020. We aimed to recruit at least 60 participants for this pilot phase I investigation to inform a subsequent larger trial, which was planned and registered in clinicaltrials.org. Given the relative ease of conducting online studies and generating a sufficiently powered sample size, we tested the effects of the intervention, which is usually not possible or recommended in smaller pilot studies (Leon et al., 2011). The study was advertised on social media platforms and was also announced in a national newspaper. Interested individuals were directed to the study web page, www.coronacope.se, where information about the study and treatment principles was provided. The registered individuals signed an informed consent sheet online. They were then instructed to complete the screening measures, including sociodemographic questions and the following questionnaires: Beck Depression Inventory (BDI-II), Brunnsviken Brief Quality of Life Scale (BBQ), Patient Health Questionnaire (PHQ-9), Generalized Anxiety Disorder-7-item scale (GAD-7), Alcohol Use Disorder Identification Test (AUDIT), Insomnia Severity Index (ISI), Impact of Events Scale-Revised (IES-R), Perceived Stress Scale (PSS), Dimensions of Anger Reactions (DAR-5), and The CoRonavIruS Health Impact Survey (CRISIS; as shown below for details). By the time the screening was completed, individuals who met the inclusion criteria (as shown below) were called for a clinical telephone interview. The purpose of the interview was to find areas of relevance for each individual to facilitate tailoring of the treatment, as well as to detect potential obstacles for participation (e.g., ongoing substance abuse or being actively suicidal). In case of suicidality or self-harm behavior, the individual was called and advised on how to seek suitable alternative care (in Sweden, health care is tax-funded and does not require special insurance). Other excluded participants were provided with an individual email with reasons for exclusion (mainly that the treatment offered was not suitable for their problems).

In total, 60 individuals completed the screening and were called for a clinical interview. All interviewed cases were discussed during case management meetings with the interviewers (clinical psychologists), a psychiatrist, and the principal investigator, who had the main clinical responsibility. Inclusion and exclusion criteria were discussed, and a decision was made to either include or exclude each participant. Following the exclusion of six persons after the case management meeting, we included 52 participants. These were randomized to either ICBT for 7 weeks (n = 26) or to a wait-list control condition (n = 26). Randomization was administered through an online random number generator and was performed by a person who was not involved in the research project.¹ A flowchart of participant progression throughout the study is shown in Figure 1. Descriptive statistics of included participants is shown in Table 1.

FIGURE 1

Figure 1. Flowchart of the progress from registration to post-measurement.

TABLE 1

Table 1. Demographic characteristics of the participants at pre-treatment.

Inclusion Criteria

To be included in the study, participants had to: (1) experience mental health problems and issues which occurred or worsened in conjunction with the pandemic of the coronavirus and/or its consequences, (2) speak, read and write Swedish fluently, (3) have access to a computer, tablet, or a telephone with the internet, and (4) be 18 years or older.

Exclusion Criteria

If any of the following exclusion criteria were met, the individual was excluded: (1) having a severe mental or somatic illness, which would complicate participation or make participation impossible (including ongoing COVID-19 infection), (2) ongoing alcohol or substance abuse, (3) acute suicidality, (4) ongoing psychological treatment, and (5) having drug treatment with an unstable dosage or a dose, which was planned to change within the time of treatment. The dosage was regarded as stable if it had been the same for the last 3 months. An overall assessment was made on whether acute suicidality was present, including the pre-treatment measures on self-report measures and the information obtained during the clinical telephone interview. Thoughts about death and suicide were, in most cases, not regarded as acute suicidality, but if any plans or more concrete thoughts about course of action were formulated, the individual was encouraged and helped to seek help elsewhere and was excluded from the study.

Treatment and Therapist

The treatment consisted of seven selected modules out of 16 possible. The participants gained access to one module every week, and the modules were specifically selected based on the answers of the participants on the screening and clinical interview. They also had the opportunity to wish which modules they wanted by the end of the first module. All participants were assigned the first module, Introduction, and the last module, Conclusion, and the maintenance plan. The other five modules were given individually, which dealt with behavioral activation, cognitive techniques, acceptance, emotion regulation, anxiety and exposure, anxiety and worry, anxiety and panic, social anxiety, sleep strategies, perfectionism, stress management, relaxation, problem-solving, and imaginary exposure regarding difficult memories. The modules were derived from a selection of earlier studies (e.g., Carlbring et al., 2010; Johansson et al., 2012) on major depression and anxiety disorder. Modules were adjusted to suit the target group including the mental health problems that may have been influenced by the pandemic and recommendations and restrictions by the Swedish Public Health Agency regarding the spread of the coronavirus. Each module consisted of texts for the participants to read and exercises to work with and write about in the modules. Each participant had a therapist who gave online support and feedback on the work with the modules and exercises and motivated the participant to continue to work with the treatment (Andersson, 2015). Participants were asked to complete the PHQ-9 every week with the purpose to monitor possibly worsening during the treatment. The control group had the opportunity to contact the principal investigator, if needed, and participants were asked to fill in the PHQ-9 every week during the waiting period. The control group was offered an equal treatment of the same procedures as soon as the treatment group had finished (including the post-waiting period for the control group).

The ICBT included support from a therapist, and all communication between the therapists and the participants, except for the clinical interview for inclusion, was conducted through a secure platform. Through this secure platform, self-report measures and the distribution of modules were administered and conducted (Vlaescu et al., 2016). Once a week, the therapists gave feedback on the work of the participants with the modules through text-message option on the platform. At the same time, the therapists gave access to the next coming module. The participants did also have the opportunity to contact their therapist with messages through the treatment platform. If contact was initiated this way, the participant could expect the therapist to answer via message within 24 h. The clinicians working with the treatment were one licensed clinical psychologist and three recently graduated clinical psychologists who were experienced in both planning and providing ICBT. The recently graduated psychologists were supervised by the licensed clinical psychologist. The principal investigator was also available for supervision and advice. In addition, a psychiatrist was available during the entire research project and contributed with medical competence and advice.

Measures

All measures were administered at pre-treatment and post-treatment, except for the CRISIS. The CRISIS is a self-report measure and was developed for covering key domains relevant to mental distress and resilience during the COVID-19 pandemic (Nikolaidis et al., 2020). CRISIS has demonstrated good psychometric attributes, including good construct validity and excellent test–retest reliability, with intra-class correlations between 0.79 and 0.89 in American and British samples (Nikolaidis et al., 2020). The CRISIS was exclusively included in the pre-treatment measure and was the only form of the questionnaires described above that the participants did not receive in the post-treatment measure. The applicants were also asked if they had tested positive for the coronavirus.

Primary Outcome Measures

The primary outcome measures were the severity of depressive symptoms, as measured with the BDI-II (Beck et al., 2005), and quality of life, which was measured with the BBQ (Lindner et al., 2016). The BDI-II is one of the most frequently used self-report measures to investigate depressive symptoms (Beck et al., 2005). It consists of 21 depression symptom-related statements, and the individuals estimate how well the statements match their experience. Higher values reflect more severe symptoms. Depressive symptoms are interpreted as minimal (0–13), mild (14–19), moderate (20–28), or severe (29–63; Beck et al., 2005). The BDI-II has good test–retest reliability (r = 0.93) and excellent internal consistency (α = 0.92) in a clinical population (Beck et al., 2005). The psychometric properties are maintained even when administered on the internet (Holländare et al., 2010).

To examine the quality of life, the BBQ was used, which has 12 statements and the total score ranged from 0 to 96 (Lindner et al., 2016). A higher score indicates a higher experienced quality of life. An optimal cutoff between a clinical and non-clinical group is 52.5, with a sensitivity of 0.79 and a specificity of 0.71 (Frykheden, 2014). The internal and test–retest reliability, as well as the concurrent and convergent validity, are high (Lindner et al., 2016). Lindner et al. (2016) concluded that the BBQ is sensitive to changes and can differentiate between clinical and healthy groups.

Secondary Outcome Measures

The PHQ-9, as the BDI-II, intends to measure the degree of depression (Kroenke et al., 2001), and higher values indicate symptoms in the higher extent. It is scored on a range from 0 to 36 based on the answers of the individual on nine items. In addition, the questionnaire includes one question regarding to which extend the depressive symptoms make it harder to handle work, the home, or getting along with other people. Values are assessed as no or minimal depression (0–4), mild depression (5–9), moderate depression (10–14), moderate to severe (15–19), or severe depression (20–27; Kroenke and Spitzer, 2002). Except for having good validity and compliance with clinical assessments (Spitzer et al., 1999), the PHQ-9, by Löwe et al. (2004), has been shown to have excellent internal consistency (α = 0.89) and, by Kroenke et al. (2001), also has been shown to have excellent test–retest reliability.

For measuring symptoms of anxiety and worry, the questionnaire GAD-7 was used. It is scored from 0 to 28 and has demonstrated high validity in comparison with clinical assessments and other measures, both in research (Spitzer et al., 2006) and primary care samples (Löwe et al., 2008). Higher scores reflect higher anxiety and worry symptoms. The estimate of an individual on this questionnaire is interpreted as mild (5–10), moderate (11–15), or severe (15 and above). A recommended cutoff when screening for anxiety disorders is 10 (Spitzer et al., 2006). The questionnaire has shown excellent intern consistency (α = 0.92) and good test–retest reliability (Spitzer et al., 2006).

The AUDIT consists of 10 items regarding alcohol use, its harmful effects, and dependency symptoms (Berman et al., 2012). When investigated, the AUDIT has consistently shown good validity and reliability. The AUDIT was included as a measure for detecting the level of possible alcohol use and was not primarily analyzed as a separate outcome measure. Higher scores represent a more frequent use of alcohol. Cutoff values for adventurous or harmful use of alcohol are eight for men and six for women (Berman et al., 2012).

The ISI is a questionnaire designed for measuring and investigating symptoms of insomnia, consisting of seven questions and results in a total score ranging between 0 and 28 (Bastien et al., 2001). Higher scores indicate more experienced problems with sleep. These scores are assessed as non-significant insomnia symptoms (0–7), some insomnia symptoms (8–14), moderate insomnia and clinically significant (15–21), or severe and clinically significant insomnia (22–28; Bastien et al., 2001). The instrument has been shown to be a valid and reliable instrument for quantifying perceived insomnia, with a moderate internal consistency (α = 0.74; Bastien et al., 2001).

The IES-R is a self-report measure developed to explore symptoms of post-traumatic stress disorder (Arnberg et al., 2014). The questionnaire consists of 22 questions addressing symptoms of re-experiencing, avoidance, and hypervigilance. Weiss (2004) recommends not using cutoff scores for IES-R, but higher scores indicate more severe symptoms of post-traumatic stress disorder. According to Weiss and Marmar (1997), the IES-R has demonstrated high test–retest reliability as well as high internal consistency.

The 14-item self-report measure PSS-14 is designed to detect and measure symptoms of stress and has been shown by Cohen et al. (1983) to have solid psychometric attributes in terms of validity and reliability. The Swedish version of the 14-item version of this questionnaire has not been evaluated in an equally detailed manner, but PSS-10 has shown to have good internal consistency (α = 0.84; Nordin and Nordin, 2013). No official cutoff values for clinical significance have been defined.

The brief 5-item instrument DAR-5 is designed to assess the experience of anger (Goulart et al., 2020). Each item is scored on a 5-point Likert scale, which varies from one (none or almost none of the time) to five (all or almost all of the time), with a total score ranging from 5 to 25. Higher values imply the experience of higher anger, and the suggested cutoff value for reflecting psychological distress and functional impairment is 12 (Forbes et al., 2014). According to Goulart et al. (2020), the DAR-5 demonstrates good internal reliability (ranging from α = 0.80 to 0.90) and construct validity in general. In their sample, DAR-5 had an acceptable internal consistency (α = 0.73).

Data Analytic Approach

Statistical analyses were performed using IBM Statistics SPSS version 27 (SPSS 27). We used both complete case analysis (CCA) and Intention-to-Treat-analysis (ITT). Multiple regression analysis was used to analyze treatment effects. There are several ways to enter the variables in a regression model. A forced entry was used as recommended by Studenmund and Cassidy (1987). With forced entry, all predictors are forced into the regression model simultaneously. Before multiple regression was performed, assumptions of the linear model were investigated, involving additivity and linearity, independent residuals, homoscedasticity, and normally distributed errors. The assumption of no perfect multicollinearity was added. For all tests of significance, a two-sided approach and the limit, α = 0.05, were used.

A broadly recommended method to handle missing data according to Van Ginkel (2020) is multiple imputations (Rubin, 1987). Using this procedure, and by performing 20 imputations, all of the available data were used in the analysis through full information maximum likelihood estimation, and thus, as mentioned above, an ITT approach was applied. This kind of analysis is based on the assumption that data are Missing At Random (MAR). In other words, the probability of missing data is allowed to be dependent on any observed variable but not on the would-be value of the missing data point (Schafer and Graham, 2002).

To calculate effect sizes Cohen’s d was used, using adjusted means of post-treatment measures were controlled for pre-treatment measures and observed SDs were controlled for post-treatment measures. Cohen’s d is interpreted as 0.20 (small), 0.50 (medium), and >0.80 (large) based on the guidelines by Cohen (1988).

We also calculated reliable change index (RCI), which is a method to compare the values at pre- and post-treatment measures, where the change must achieve a certain level to be considered as reliable when considering the reliability of the measure (Jacobson and Truax, 1991).

Results

Sample Characteristics

Descriptive statistics are presented in Table 1. Most of the included participants were female (71.2%), and the mean age was 42.7 years (SD = 17.4). Of the 52 participants included in the study, none declared that they had been tested positive for the COVID-19 virus. Using independent t-tests, no differences were found between the treatment and control groups regarding any continuous baseline variable (all p’s > 0.05) with two exceptions. The treatment group had higher ratings on BDI-II [t(50) = −2.06, p = 0.044] and PHQ-9 [t(50) = 2.12, p = 0.039]. Because the pre-treatment measures were used as predictors to explain the variance of the post-treatment measures, this was not considered as a major issue but should be and is taken into account when interpreting the results. The two groups did neither differ regarding age [t (50) = 0.24, p = 0.81] nor any other demographic characteristics investigated with χ²-tests.

Treatment Dropout and Missing Data

Those who did not complete the post-treatment measures were defined as dropouts. As the flowchart in Figure 1 illustrates, data loss at the post-treatment was nine (17%) in total, with six (23%) from the treatment group and three (11.5%) from the control group. Accordingly, 43 (83%) individuals completed the post-treatment measures. Using independent samples t-tests, there were no differences on the pre-treatment measures (all p’s > 0.05), with the exceptions of GAD [t(50) = 2.86, p = 0.006] and PSS [t(50) = 2.26, p = 0.028], with higher scores in the non-completers group. No differences between completers and non-completers were observed regarding demographic characteristics (χ²-tests). Participants who did not complete the post-treatment and thus were defined as dropouts completed 1.67 (SD = 1.37) modules on average. Little’s Missing Completely at Random (MCAR) test was not statistically significant, χ²(9) = 10.412, p = 0.318, which indicates there was no apparent pattern explaining missing data.

Treatment Adherence and Therapist Time

Treatment adherence was defined when the participant had responded in the treatment platform regarding exercises in the modules or sent a message to the therapist, indicating that the participant had understood the main purpose of the module. On average, participants completed 4.31 (61.6%) out of the seven modules (SD = 2.57). Of all 26 participants who were randomized to the treatment group, 10 (38.5%) completed all assigned modules. The module Introduction, which was included in the treatment of all participants, was the most completed module among all other modules with 25 participants out of 26 completing it. After Introduction, the module focusing on cognitive techniques was the second commonly completed module, which was completed 13 times, and the module addressing behavioral activation was the third commonly completed module with 12 participants completing that. The module least completed by the participants was the one about social anxiety, which was completed by zero participants. Then, the three modules about anxiety and exposure, anxiety and panic, and problem-solving were completed by one participant each. In parenthesis, the number of participants who completed each module are as follows: acceptance (9), anxiety and worry (8), stress management (8), emotion regulation (8), sleep strategies (6), relaxation (4), perfectionism (4), and imaginary exposure regarding difficult memories (2). Except for the change in scores of IES-R, pre-treatment and post-treatment scores, and the number of modules completed (r = −0.49, p = 0.026), treatment adherence was not correlated with change scores of any of the other outcome measures (all p’s > 0.05).

The therapists spent on average 111.3 min (SD = 67.8) on each participant in total, and the average time spent per week was 15.90 min (SD = 9.89). The minimum time spent on a participant was 0 min and the highest was 358 min.

Analysis of Treatment Effects

Main Outcome Measures

Table 2 shows descriptive statistics for each condition at each assessment point with the imputed data. B- and beta-values from the regression analysis are presented, with group condition as a predictor controlling for pre-treatment scores. Standardized regression coefficients and corresponding effect size for each outcome measure at post-treatment measure are reported in Table 3.

TABLE 2

Table 2. Imputed means, SDs, and the number of participants for each measure divided by condition and assessment point.

TABLE 3

Table 3. Regression model of the impact of group condition on primary and secondary outcome measures with variance explained by pre-treatment measure included.

When controlling for pre-treatment scores, the results from the regression model revealed a significant unstandardized regression coefficient for group condition on BDI-II with b = −5.48 95% CI [−0.43, −10.53], t = −2.130, p = 0.034. This shows that the treatment group, on average, had increased or decreased their estimates on BDI-II at post-treatment measurement, with 5.48 points more than the control group. The between-group effect size was Cohen’s d = 0.63 [95% CI: 0.07, 1.18], which is a large effect size. For the BBQ, no significant unstandardized regression coefficient for group condition was found, b = 2.86, 95% CI [−5.29, 10.94], t = 0.68, p = 0.49. The between-group effect size was Cohen’s d = 0.15 [95% CI: 0.39, −0.70].

Secondary Outcome Measures

No significant effect was found for the group condition as a predictor of the post-treatment outcome on the PHQ-9 when controlling for pre-treatment scores, b = −2.63, 95% CI [−5.53, 0.28], t = −1.78, p = 0.076. The between-group effect size was Cohen’s d = 0.62 [95% CI: 0.05, 1.16]. For GAD-7, a significant effect for the group condition was found, b = −3.55, 95% CI [−6.11, −1.00], t = −2.73, p = 0.006, with a large between-group effect size Cohen’s d = 0.82 [95% CI: 0.24, 1.37]. For the AUDIT, a significant group effect was found, b = −1.29 95% CI [−2.53, −0.04], t = −2.035, p = 0.043. The between-group effect size was Cohen’s d = 0.54 [95% CI: −1.09, 0.02]. For the ISI, no significant effect was found, b = 0.13, 95% CI [−2.92, 3.17], t = 0.081, p = 0.935. The between-group effect size was Cohen’s d = 0.07 [95% CI: −0.48, 0.61]. There was no significant effect on the IES-R, b = −6.29, 95% CI [−14.89, 2.31], t = −1.44, p = 0.15. The between-group effect size was Cohen’s d = 0.45 [95% CI: −0.10, 1.00]. For the PSS-14, a significant group effect was found, b = −6.36, 95% CI [−10.26, −2.45], t = −3.20, p = 0.001. The between-group effect size was Cohen’s d = 1.04 [95% CI: 0.45, 1.61]. For the DAR-5, no significant effect was found for the group condition, b = −0.72 95% CI [−2.02, 0.58], t = −1.09, p = 0.275. The between group effect size was Cohen’s d = 0.27 [95% CI: −0.28, 0.81].

The same analyses were calculated with CCA, which gave very similar results as the ITT analyses with imputed data. Data and calculations with CCA are available on request.

Reliable Change and Negative Effects

Reliable change index was calculated for the BDI-II, GAD-7, and PSS-14. BDI-II and BBQ were the main outcome measures in the study, but since BBQ did not show any significant effects, we calculated RCI for the GAD-7 and PSS-14, which showed statistically significant treatment effects. To calculate RCIs, we used the means of pre-treatment and post-treatment measures of BDI-II, GAD-7, and PSS-14, and SDs for the three scales at pre-treatment measures, using the observed data. For Cronbach’s alpha, values from Beck et al. (2005) for BDI-II (α = 0.92) and Spitzer et al. (2006) for GAD-7 (α = 0.92) were used. Values from Nordin and Nordin (2013) for Cronbach’s alpha for PSS-10 (α = 0.84) were used as this was the most adequate value to use for this calculation. Calculated RCI for BDI-II was 6.09, for GAD-7 was 3.70, and for PSS-14 was 7.06. Missing cases were defined as unchanged.

For the BDI-II, 13 (50%) participants in the treatment group had a reliable change. One (4%) participant showed a reliable deterioration when using the RCI criteria in the other direction. Six (23%) participants did not reach RCI either way, and six (23%) did not complete the post-treatment measure, leaving 12 (46%) participant categorized as unchanged. Regarding the control group, eight (30%) participant reached RCI, and one (4%) participant had reliable deterioration. About 14 (54%) participants were unchanged and 3 (12%) participants did not answer the post-treatment measure, with a total of 17 (66%) participant categorized as unchanged.

For the GAD-7, 11 (42%) individuals in the treatment group reached RCI. None of the individuals showed a reliable deterioration, and nine (35%) did not reach RCI. Together with the six (23%) participants who did not answer the post-treatment measure, 15 (58%) were considered unchanged. About eight (30%) participants in the control group reached RCI, while five (19%) participants were shown to have a reliable deterioration. About 10 (38%) participants did not reach either RCI or deterioration and, as with BDI-II and the other measures, three (12%) participants within the control group did not complete the post-treatment measures, leaving 13 (50%) participants categorized as unchanged.

For the PSS-14, 12 (47%) participants in the treatment group reached RCI. Eight (30%) participants did not reach RCI and none of the participants showed reliable deterioration. Together with the six (23%) dropouts, 14 (53%) participants were considered unchanged. Regarding the control group, five (19%) participants reached RCI, while 18 (69%) participants did not reach RCI. Together with the three (12%) participants who did not answer the post-treatment measure, 21 (81%) participants categorized as unchanged. As in the treatment group, none of the participants showed reliable deterioration.

Discussion

The present study was designed to investigate the effects of a transdiagnostic individually tailored ICBT for different psychological symptoms, such as depressive symptoms and anxiety, which had occurred and/or worsened because of the corona pandemic and/or its consequences. We prepared the study and report it as a phase I pilot investigation with focus on effects. The treatment modules were adapted to suit the current situation, and overall, the approach worked. We found similar results as reported in previous ICBT studies (Andersson et al., 2019). The results were also in line with a previous ICBT study on corona-related worry (Wahlund et al., 2021). Given the need for safe, effective, and cost-effective internet interventions (Wind et al., 2020), the results are promising and should be followed by a larger trial. Based on the primary outcome measure BDI-II, the intervention resulted in a significantly larger reduction of depressive symptoms compared to the control condition (d = 0.63). This result was also observed on the secondary depression outcome measure, the PHQ-9 (d = 0.62). The treatment did not show a significant effect on the quality of life, as indicated by the second primary outcome measure, the BBQ (d = 0.15). Apart from the PHQ-9, two of the secondary outcomes, the GAD-7 (d = 0.82) and the PSS-14 (d = 1.04), were statistically significant, indicating that the treatment decreased anxiety symptoms and stress symptoms. We do not regard the effect on the AUDIT (d = 0.54) as clinically relevant due to floor effects and differences at baseline. The other three secondary outcome measures, ISI (d = 0.07), IES-R (d = 0.45), and DAR-5 (d = 0.27), did not indicate any significant effects on insomnia, symptoms of post-traumatic stress disorder, and anger.

Tailored ICBT, including support by a therapist, was thus shown to be effective for depressive symptoms with moderate effect size, in line with earlier studies (Sztein et al., 2017; Kraepelien et al., 2018). This was observed on both the primary outcome measure, BDI-II, and the secondary outcome measure, PHQ-9. With regard to reliable change (RCI), 50% of the participants in the treatment group showed a reliable change on the BDI-II. The modules which addressed depressive symptoms contained behavior activation and cognitive restructuring, which are commonly used interventions within CBT for depression. The findings suggest that these components were useful in this setting as well, where the focus was on the pandemic and its consequences. The specific changes that were made in the modules to make them suitable for the pandemic situation were mostly implemented in the examples given in this study. One example is the reinforcing activities in behavioral activation that one can engage themselves with when considering the risk of infection.

We found no effects on the second primary outcome, BBQ, which intends to measure the experienced quality of life (Lindner et al., 2016). The effect of ICBT on quality of life is varied across studies, and not only could the lack of change at least partly depend on the outcome measure BBQ but also that the treatment did not directly focus on the areas that are measured in the BBQ (e.g., friendship, spare time). More specifically, the modules included in the treatment did potentially not address the topics that the participants needed to result in a higher experienced quality of life. The therapists and the participants had 16 modules to choose from and to wish for, whereas other topics, such as loneliness, mindfulness, or working with self-esteem, were not included and may have been more important. Further, experienced quality of life can involve many things, including employment and the private economy, which also could have changed because of the pandemic and its consequences. Finally, it is also possible that the quality of life is a construct that shows lower response to ICBT than to face-to-face treatment (Hofmann et al., 2014).

In contrast to the experienced quality of life, anxiety symptoms as measured with the GAD-7 decreased in the treatment group in comparison to the control group, with a large between-group effect. This was in line with results from earlier studies on ICBT for anxiety disorders (Richards et al., 2015; Andersson et al., 2017, n.d.). In terms of reliable change, 42% of the participants reached these criteria according to the RCI. The anxiety modules essentially contained commonly used CBT strategies for anxiety, such as exposure and cognitive techniques. As with the depression modules, we implemented small adjustments of the previous material in relation to the pandemic and the risk of spread of the virus. For example, an individual with social anxiety was given the example to expose himself or herself to the trigger to call someone instead of going to a shopping mall, which before the pandemic would have been encouraged. In retrospect, anxiety could have been a primary outcome, but this was not what we planned and expected. However, the modules seem to have worked.

The effect on the secondary outcome measures, AUDIT and PSS-14, was also found, both with moderate-to-large effect sizes. The finding that perceived stress, as measured with PSS-14, was reduced mirrors the effects reported in other ICBT trials for stress (Haber et al., 2017). Unlike symptoms of stress, which could be seen as being targeted by the modules in the treatment (e.g., stress management and relaxation), a module specifically dealing with alcohol use was not included. Treatment of anxiety, stress, and depression may have an indirect effect on alcohol use (Riper et al., 2014), but we need to consider floor effects that may not be of clinical relevance.

We did not find significant treatment effects on the ISI, the IES-R, or the DAR-5. The treatment involved modules specifically dealing with insomnia, difficult memories, and emotion regulation, but only a proportion of participants worked with these modules. As the treatment was individually tailored, all participants had their treatment plan with individually combined modules. This explained why insomnia, difficult memories, and feelings of anger did not decrease more. It may also be a question of power and sample size. Earlier studies have reported several negative effects of the pandemic and its consequences on psychological well-being, such as fear (Rubin and Wessley, 2020; Shah et al., 2020), anger (Jung and Jun, 2020), anxiety, depression, and other stress responses (Horesh and Brown, 2020; Wang et al., 2021). These symptoms are not isolated, and comorbidity between depression and anxiety symptoms for example is high (Wang et al., 2021). The choice of an individually tailored ICBT-treatment was therefore motivated but has consequences for the secondary outcome measures and for problems that were not common across participants.

Two main strengths of this study were the use of a controlled design and the adaption of treatment material for the current pandemic. There are also limitations. One limitation is the use of a waiting list as the control condition, which does not control for non-specific factors and result in higher between-group effect sizes in comparison to active controls (Gold et al., 2017). A second limitation is a fairly poor adherence, as the participants in the treatment group completed only 4.31 (61.6%) out of 7 modules. This is somewhat lower than that found in other ICBT studies (Van Ballegooijen et al., 2014). One explanation could be that only a limited number of the modules felt relevant for the participants. The ICBT modules were based on an earlier study and were modified and adapted to the situation with the pandemic, but as discussed in relation to the BBQ, it is possible that other topics and modules would have been needed to address experienced psychological problems in regard to the pandemic. Even though 14 different modules were available, we did not include modules on problems, such as loneliness, that could have been more relevant and important to address. However, our impression was that we could give relevant modules to each individual and that it was an advantage to have different modules to select. Another approach would have been to have a transdiagnostic treatment with the same modules given to all participants (Titov et al., 2011). A third possible limitation was that we, unlike many other studies, conducted the trial during the summer of 2020. During the summer, people are usually less troubled by mild to moderate psychological problems compared to other periods of the year (Ayers et al., 2013). During the summer of 2020, infection rates also went down, and many people in Sweden hoped that the pandemic and its consequences would disappear in the autumn of 2020. Now in 2021, we know that this did not happen and thus we are in the process to conduct the larger trial with adjustments of the modules to make them even more adapted to the pandemic. A fourth possible limitation is that we recruited from the general public via advertisements. The participants are likely to differ from patients seen in clinics. On the other hand, a clinic-based study would have been difficult to conduct because of restrictions, and there are indications that patients who receive internet treatment may be representative of the general public rather than regular clinic patients (Titov et al., 2010). The treatment was also developed to be beneficial for individuals from the general public, irrespective of psychiatric status.

A final limitation was that we only included self-report measures. With the form CRISIS, typical physiological symptoms of the COVID-19 virus were investigated, and together with demographic questions, individuals were asked if they had been tested positive for the virus, but no health economic measures were included, such as sick leave, which would have been interesting to investigate in relation to both psychological symptoms and effects of the treatment. None of the participants had, at the time of the pre-treatment, tested positive for COVID-19. Nevertheless, we cannot be certain that none of these participants had had the virus since some infected people do not get any symptoms. Moreover, we cannot be certain that no one was infected during the study. No adverse events and no particular statements about getting sick with symptoms, indicating infection of the coronavirus, was reported.

Conclusion

The current pilot RCT study provides an initial indication of the possibilities of using an ICBT approach for dealing with the various psychological symptoms associated with the corona pandemic (Holmes et al., 2020). The treatment resulted in moderate to large decrease in measures of depression, anxiety, and stress symptoms. Future studies are needed to investigate the long-term effects and to further adjust treatments for the pandemic situation. The study was implemented in Sweden, where there is high access and use of the internet. Sweden has had one approach regarding restrictions and quarantine, while other countries have had other approaches. The pandemic, by definition, impacts the population all over the world and the effects of tailored ICBT would, therefore, be of relevance to investigate in other countries and contexts, with different levels of both internet access and restrictions given by authorities.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors on request, without undue reservation.

Ethics Statement

The studies involving human participants were reviewed and approved by Etikprövningsmyndigheten Sverige. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

GA initiated the study and planned it together with MB. VA, MS, ES, and MB conducted the interviews, adapted the treatment for COVID-19, and served as therapists in the trial. ML served as a medical consultant and also participated in intake interview meetings and supervision. VA wrote the manuscript and conducted the analyses together with GA. All authors contributed to the article and approved the submitted version.

Funding

This study was sponsored in part by a university grant to GA and by grants from the Swedish Research Council.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

We thank George Vlaescu for technical support with the treatment platform.

Footnotes

1. ^www.random.org

References

Andersson, G. (2015). The Internet and CBT: A Clinical Guide. Boca Raton: CRC Press.

Google Scholar

Andersson, G. (2016). Internet-delivered psychological treatments. Annu. Rev. Clin. Psychol. 12, 157–179. doi: 10.1146/annurev-clinpsy-021815-093006

PubMed Abstract | CrossRef Full Text | Google Scholar

Andersson, G., Carlbring, P., Titov, N., and Lindefors, N. (2019). Internet interventions for adults with anxiety and mood disorders: a narrative umbrella review of recent meta-analyses. Can. J. Psychiatr. 64, 465–470. doi: 10.1177/0706743719839381

PubMed Abstract | CrossRef Full Text | Google Scholar

Andersson, G., Estling, F., Jakobsson, E., Cuijpers, P., and Carlbring, P. (2011). Can the patient decide which modules to endore? An open trial of tailored internet treatment of anxiety disorders. Cogn. Behav. Ther. 40, 57–64. doi: 10.1080/16506073.2010.529457

PubMed Abstract | CrossRef Full Text | Google Scholar

Andersson, E., Hedman, E., Wadström, O., Boberg, J., Yaroslav Andersson, E., Axelsson, E., et al. (2017). Internet-based extinction therapy for worry: a randomized controlled trial. Behav. Ther. 48, 391–402. doi: 10.1016/j.beth.2016.07.003

PubMed Abstract | CrossRef Full Text | Google Scholar

Andersson, E., Ljótsson, B., Hedman-Lagerlöf, M., Nygren, L., Persson, M., Rosengren, K., et al. (n.d.). Targeting excessive worry with internet-based extinction therapy: a randomized controlled trial with meditation analysis and economical evaluation. Psychol. Med. (in press).

Google Scholar

Andersson, G., and Titov, N. (2014). Advantages and limitations of internet-based interventions for common mental disorders. World Psychiatry 13, 4–11. doi: 10.1002/wps.20083

PubMed Abstract | CrossRef Full Text | Google Scholar

Arden, M. A., and Chicot, J. (2020). Health psychology and the coronavirus (COVID-19) global pandemic: a call for research. Br. J. Health Psychol. 25, 231–232. doi: 10.1111/bjhp.12414

PubMed Abstract | CrossRef Full Text | Google Scholar

Arnberg, F., Michel, P., and Bergh Johannesson, K. (2014). Properties of Swedish posttraumatic stress measures after a disaster. J. Anxiety Disord. 28, 402–409. doi: 10.1016/j.janxdis.2014.02.005

PubMed Abstract | CrossRef Full Text | Google Scholar

Ayers, J. W., Althouse, B. M., Allem, J. P., Rosenquist, J. N., and Ford, D. E. (2013). Seasonality in seeking mental health information on google. Am. J. Prev. Med. 44, 520–525. doi: 10.1016/j.amepre.2013.01.012

PubMed Abstract | CrossRef Full Text | Google Scholar

Azim, D., Kumar, S., Nasim, S., Arif, T. B., and Nanjiani, D. (2020). COVID-19 as a psychological contagion: a new Pandora’s box to close? Infect. Control Hosp. Epidemiol. 41, 989–990. doi: 10.1017/ice.2020.127

PubMed Abstract | CrossRef Full Text | Google Scholar

Bastien, C. H., Valliéres, A., and Morin, C. M. (2001). Validation of the insomnia severity index as an outcome measure for insomnia research. Sleep Med. 2, 297–307. doi: 10.1016/S1389-9457(00)00065-4

PubMed Abstract | CrossRef Full Text | Google Scholar

Baumeister, H., Reichler, L., Munzinger, M., and Lin, J. (2014). The impact of guidance on internet-based mental health interventions - a systematic review. Internet Interv. 1, 205–215. doi: 10.1016/j.invent.2014.08.003

CrossRef Full Text | Google Scholar

Beck, A. T., Steer, R. A., and Brown, G. K. (2005). BDI-II. Beck Depression Inventory Second Edition: Manual, Svensk Version (Swedish Version). Stockholm: Psykologiförlaget AB.

Google Scholar

Berger, T., Boettcher, J., and Caspar, F. (2014). Internet-based guided self-help for several anxiery disorders: a randomized controlled trial comparing a tailored with a standardized disorder-specific approach. Psychotherapy 51, 207–219. doi: 10.1037/a0032527

PubMed Abstract | CrossRef Full Text | Google Scholar

Bergman Nordgren, L., Hedman, E., Etienne, J., Bodin, J., Kadowaki, Å., Eriksson, S., et al. (2014). Effectiveness and cost-effectiveness of tailored internet-delivered cognitive behavior therapy for anxiety disorders in a primary care population: a randomized controlled trial. Behav. Res. Ther. 59, 1–11. doi: 10.1016/j.brat.2014.05.007

PubMed Abstract | CrossRef Full Text | Google Scholar

Berman, A. H., Wennberg, P., and Källmén, H. (2012). AUDIT och DUDIT – Identifiera problem med alkohol och droger. Stockholm: Gothia förlag.

Google Scholar

Brooks, S. K., Webster, R. K., Smith, L. E., Woodland, L., Wessely, S., Greenberg, N., et al. (2020). The psychological impact of quarantine and how to reduce it: rapid review of the evidence. Lancet 395, 912–920. doi: 10.1016/S0140-6736(20)30460-8

PubMed Abstract | CrossRef Full Text | Google Scholar

Carlbring, P., Andersson, G., Cuijpers, P., Riper, H., and Hedman-Lagerlöf, E. (2018). Internet-based vs. face-to-face cognitive behavior therapy for psychiatric and somatic disorders: an updated systematic review and meta-analysis. Cogn. Behav. Ther. 47, 1–18. doi: 10.1080/16506073.2017.1401115

PubMed Abstract | CrossRef Full Text | Google Scholar

Carlbring, P., Maurin, L., Törngren, C., Linna, E., Eriksson, T., Sparthan, E., et al. (2010). Individually tailored internet-based treatment for anxiety disorders: a randomized controlled trial. Behav. Res. Ther. 49, 18–24. doi: 10.1016/j.brat.2010.10.002

PubMed Abstract | CrossRef Full Text | Google Scholar

Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences. Hillsdale: L. Erlbaum Associates.

Google Scholar

Cohen, S., Kamarck, T., and Mermelstein, R. (1983). A global measure of perceived stress. J. Health Soc. Behav. 24, 385–396.

PubMed Abstract | Google Scholar

Forbes, D., Alkemade, N., Mitchell, D., Elhai, J. D., McHugh, T., Bates, G., et al. (2014). Utility of the deminsions of anger reactions-5 (DAR-5) scale as a brief anger measure. Depress. Anxiety 31, 166–173. doi: 10.1002/da.22148

PubMed Abstract | CrossRef Full Text | Google Scholar

Frykheden, O. (2014). Effektiv mätning av livskvalitet – Brunnsviken brief quality of life inventory (BBQ): En psykometrisk utvärdering [Effective measurement of quality of life. Brunnsviken brief quality of life instrument (BBQ): a psychometric evaluation. [unpublished master of science thesis]. [Stockholm]: Stockholm university.

Google Scholar

Gold, S. M., Hasselmann, H., Friede, T., Hegerl, U., Mohr, D. C., and Otte, C. (2017). Control conditions for randomised trials of behavioural interventions in psychiatry: a decision framework. Lancet 4, 725–732. doi: 10.1016/S2215-0366(17)30153-0

PubMed Abstract | CrossRef Full Text | Google Scholar

Goulart, A. C., Bismarchi, D., Rienzo, M., Haddad Syllos, D., and Wang, Y.-P. (2020). Dimensions of anger reactions (DAR-5): a useful screening tool for anger in the general population. Int. J. Psychiatry Clin. Pract. doi: 10.1080/13651501.2020.1821893 (in press).

PubMed Abstract | CrossRef Full Text | Google Scholar

Haber, E., Ebert, D. D., Lehr, D., Cuijpers, P., Berking, M., Nobis, S., et al. (2017). The benefit of web- and computer-based interventions for stress: a systematic review and meta-analysis. J. Med. Internet Res. 19:e32. doi: 10.2196/jmir.5774

PubMed Abstract | CrossRef Full Text | Google Scholar

Hofmann, S. G., Wu, J. Q., and Boettcher, H. (2014). Effect of cognitive-behavioral therapy for anxiety disorders on quality of life: a meta-analysis. J. Consult. Clin. Psychol. 82, 375–391. doi: 10.1037/a0035491

PubMed Abstract | CrossRef Full Text | Google Scholar

Holländare, F., Andersson, G., and Engström, I. (2010). A comparison of psychometric properties between internet and paper versions of two depression instruments (BDI-II and MADRS-S) administered to clinic patients. J. Med. Internet Res. 12:e45. doi: 10.2196/jmir.1392

PubMed Abstract | CrossRef Full Text | Google Scholar

Holmes, E. A., O’Connor, R. C., Perry, V. H., Tracey, I., Wessely, S., Arseneault, L., et al. (2020). Multidisciplinary research priorities for the COVID-19 pandemic: a call for action for mental health science. Lancet Psychiatry 7, 547–560. doi: 10.1016/S2215-0366(20)30168-1

PubMed Abstract | CrossRef Full Text | Google Scholar

Horesh, D., and Brown, A. D. (2020). Traumatic stress in the age of COVID-19: a call to close critical gaps and adapt to new realities. Psychol. Trauma 12, 331–335. doi: 10.1037/tra0000592

PubMed Abstract | CrossRef Full Text | Google Scholar

Jacobson, N. S., and Truax, P. (1991). Clinical significance: a statistical approach to defining meaningful change in psychotherapy research. J. Consult. Clin. Psychol. 59, 12–19. doi: 10.1037/0022-006X.59.1.12

PubMed Abstract | CrossRef Full Text | Google Scholar

Johansson, R., Sjöberg, E., Sjögren, M., Johnsson, E., Carlbring, P., Andersson, T., et al. (2012). Tailored vs. standardized internet-based cognitive behavior therapy for depression and comorbid symptoms: a randomized controlled trail. PLoS One 7:e36905. doi: 10.1371/journal.pone.0036905

PubMed Abstract | CrossRef Full Text | Google Scholar

Jung, S. J., and Jun, J. Y. (2020). Mental health and psychological intervention amid COVID-19 outbreak: perspectives from South Korea. Yonsei Med. J. 61, 271–272. doi: 10.3349/ymj.2020.61.4.271

PubMed Abstract | CrossRef Full Text | Google Scholar

Kessler, R. C., Merikangas, K. R., and Wang, P. S. (2007). Prevalance, comorbidity, and service utilization for mood disorders in the United States at the beginning of the twenty-first century. Annu. Rev. Clin. Psychol. 3, 137–158. doi: 10.1146/annurev.clinpsy.3.022806.091444

PubMed Abstract | CrossRef Full Text | Google Scholar

Kraepelien, M., Forsell, E., Karin, E., Johansson, R., Lindefors, N., and Kaldo, V. (2018). Comparing individually tailored to disorder-specific internet-based cognitive-behavioural therapy: benchmarking study. Br. J. Psychiatry 4, 282–284. doi: 10.1192/bjo.2018.41

PubMed Abstract | CrossRef Full Text | Google Scholar

Kroenke, K., and Spitzer, R. L. (2002). The PHQ-9: a new depression diagnostic and severity measure. Psychiatr. Ann. 32, 509–521. doi: 10.3928/0048-5713-20020901-06

CrossRef Full Text | Google Scholar

Kroenke, K., Spitzer, R. L., and Williams, J. B. W. (2001). The PHQ-9: validity of a brief depression severity measure. J. Gen. Intern. Med. 16, 606–613. doi: 10.1046/j.1525-1497.2001.016009606.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Leon, A. C., Davis, L. L., and Kraemer, H. C. (2011). The role and interpretation of pilot studies in clinical research. J. Psychiatr. Res. 45, 626–629. doi: 10.1016/j.jpsychires.2010.10.008

PubMed Abstract | CrossRef Full Text | Google Scholar

Lindner, P., Frykheden, O., Forsström, D., Andersson, E., Ljótsson, B., Hedman, E., et al. (2016). The Brunnsviken brief quality of life scale (BBQ): development and psychometric evaluation. Cogn. Behav. Ther. 45, 182–195. doi: 10.1080/16506073.2016.1143526

PubMed Abstract | CrossRef Full Text | Google Scholar

Löwe, B., Decker, O., Müller, S., Brähler, E., Schellberg, D., Herzog, W., et al. (2008). Validation and standardization of the Generalized Anxiety Disorder Screener (GAD-7) in the general population. Med. Care 46, 266–274. doi: 10.1097/MLR.0b013e318160d093

PubMed Abstract | CrossRef Full Text | Google Scholar

Löwe, B., Kroenke, K., Herzog, W., and Gräfe, K. (2004). Measuring depression outcome with a brief self-report instrument: sensitivity to change of the patient health questionnaire (PHQ-9). J. Affect. Disord. 81, 61–66. doi: 10.1016/S0165-0327(03)00198-8

PubMed Abstract | CrossRef Full Text | Google Scholar

Nikolaidis, A., Paksarian, D., Alexander, L., DeRosa, J., Dunn, J., Nielson, D. M., et al. (2020). The Coronavirus Health and Impact Survey (CRISIS) reveals reproducible correlates of pandemic-related mood states across the Atlantic. medRxiv. [Preprint]. doi: 10.1101/2020.08.24.20181123

PubMed Abstract | CrossRef Full Text | Google Scholar

Nordin, M., and Nordin, S. (2013). Psychometric evaluation and normative data of the Swedich version of the 10-item perceived stress scale. Scand. J. Psychol. 54, 502–507. doi: 10.1111/sjop.12071

PubMed Abstract | CrossRef Full Text | Google Scholar

Onyeaka, H., Zahid, S., and Patel, R. S. (2020). The unaddressed behavioral health aspect during the coronavirus pandemic. Cureus 12:e7351. doi: 10.7759/cureus.7351

PubMed Abstract | CrossRef Full Text | Google Scholar

Păsărelu, C. R., Andersson, G., Bergman, N. L., and Dobrean, A. (2017). Internet-delivered transdiagnostic and tailored cognitive behavioral therapy for anxiety and depression: a systematic review and meta-analysis of randomized controlled trials. Cogn. Behav. Ther. 46, 1–28. doi: 10.1080/16506073.2016.1231219

PubMed Abstract | CrossRef Full Text | Google Scholar

Richards, D., Richardson, T., Timulak, L., and McElvaney, J. (2015). The efficacy of internet-delivered treatment for generalized anxiety disorder: a systematic review and meta-analysis. Internet Interv. 2, 272–282. doi: 10.1016/j.invent.2015.07.003

CrossRef Full Text | Google Scholar

Riper, H., Andersson, G., Hunter, S. B., de Wit, J., Berking, M., and Cuijpers, P. (2014). Treatment of comorbid alcohol use disorders and depression with cognitive-behavioural therapy and motivational interviewing: a meta-analysis. Addiction 109, 394–406. doi: 10.1111/add.12441

PubMed Abstract | CrossRef Full Text | Google Scholar

Rubin, D. B. (1987). Multiple Imputation for Nonresponse in Surveys. New York, NY: Wiley.

Google Scholar

Rubin, G. J., and Wessely, S. (2020). The psychological effects of quarantining a city. Br. Med. J. 368:m313. doi: 10.1136/bmj.m313

PubMed Abstract | CrossRef Full Text | Google Scholar

Schafer, J. L., and Graham, J. W. (2002). Missing data: our view of the state of the art. Psychol. Methods 7, 147–177. doi: 10.1037/1082-989X.7.2.147

PubMed Abstract | CrossRef Full Text | Google Scholar

Shah, K., Kamrai, D., Mekala, H., Mann, B., Desai, K., and Patel, R. S. (2020). Focus on mental health during the coronavirus (COVID-19) pandemic: applying learnings from the past outbreaks. Cureus 12:e7405. doi: 10.7759/cureus.7405

PubMed Abstract | CrossRef Full Text | Google Scholar

Spitzer, R. L., Kroenke, K., and Williams, J. B. (1999). Validation and utility of a self-report version of PRIME-MD: the PHQ primary care study. Primary care evaluation of mental disorders. Patient health questionnaire. J. Am. Med. Assoc. 282, 1737–1744.

Google Scholar

Spitzer, R. L., Kroenke, K., Williams, J. W., and Löwe, B. (2006). A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch. Intern. Med. 166, 1092–1097. doi: 10.1001/archinte.166.10.1092

PubMed Abstract | CrossRef Full Text | Google Scholar

Studenmund, A. H., and Cassidy, H. J. (1987). Using econometrics: A practical guide. Boston: Little, Brown.

PubMed Abstract | Google Scholar

Sztein, D. M., Koransky, C. E., Fegan, L., and Himelhoch, S. (2017). Efficacy of cognitive behavioural therapy delivered over the internet for depressive symptoms: a systematic review and meta-analysis. J. Telemed. Telecare 24, 527–539. doi: 10.1177/1357633X17717402

PubMed Abstract | CrossRef Full Text | Google Scholar

Titov, N., Andrews, G., Kemp, A., and Robinson, E. (2010). Characteristics of adults with anxiety or depression treated at an internet clinic: comparison with a national survey and an outpatient clinic. PLoS One 5:e10885. doi: 10.1371/journal.pone.0010885

PubMed Abstract | CrossRef Full Text | Google Scholar

Titov, N., Dear, B. F., Schwencke, G., Andrews, G., Johnston, L., Craske, M. G., et al. (2011). Transdiagnostic internet treatment for anxiety and depression: a randomised controlled trial. Behav. Res. Ther. 49, 441–452. doi: 10.1016/j.brat.2011.03.007

PubMed Abstract | CrossRef Full Text | Google Scholar

Umscheid, C. A., Margolis, D. J., and Grossman, C. E. (2011). Key concepts of clinical trials: a narrative review. Postgrad. Med. 123, 194–204. doi: 10.3810/pgm.2011.09.2475

PubMed Abstract | CrossRef Full Text | Google Scholar

Van Ballegooijen, W., Cuijpers, P., Van Straten, A., Karyotaki, E., Andersson, G., Smit, J., et al. (2014). Adherence to internet-based and face-to-face cognitive behavioural therapy for depression: a meta-analysis. PLoS One 9:e100674. doi: 10.1371/journal.pone.0100674

PubMed Abstract | CrossRef Full Text | Google Scholar

Van Ginkel, J. R. (2020). Standardized regression coefficients and newly proposed estimators for R² in multiply imputed data. Psychometrika 85, 185–205. doi: 10.1007/s11336-020-09696-4

PubMed Abstract | CrossRef Full Text | Google Scholar

Vlaescu, G., Alasjö, A., Miloff, M., Carlbring, P., and Andersson, G. (2016). Features and functionality of the iterapi platform for internet-based psychological treatment. Internet Interv. 6, 107–114. doi: 10.1016/j.invent.2016.09.006

PubMed Abstract | CrossRef Full Text | Google Scholar

Wahlund, T., Mataix-Cols, D., Lauri, K. O., de Schipper, E., Ljótsson, B., Aspvall, A., et al. (2021). Brief online cognitive behavioural intervention for dysfunctional worry related to the COVID-19 pandemic: a randomised controlled trial. Psychother. Psychosom. 90, 191–199. doi: 10.1159/000512843

PubMed Abstract | CrossRef Full Text | Google Scholar

Wang, Y., Di, Y., Ye, J., and Wei, W. (2021). Study on the public psychological states and its related factors during the outbreak of coronavirus disease 2019 (COVID-19) in some regions of China. Psychol. Health Med. 26, 13–22. doi: 10.1080/13548506.2020.1746817

PubMed Abstract | CrossRef Full Text | Google Scholar

Wang, C., Pan, R., Wan, X., Tan, Y., Xu, L., Ho, C. S., et al. (2020). Immediate psychological responses and associated factors during the initial stage of the 2019 coronavirus disease (COVID-19) epidemic among the general population in China. Int. J. Environ. Res. Public Health 17, 1729–1754. doi: 10.3390/ijerph17051729

PubMed Abstract | CrossRef Full Text | Google Scholar

Weiss, D. S. (2004). “The impact of event scale – revised,” in Assessing Psychological Trauma and PTSD. 2nd Edn. eds. J. P. Wilson and T. M. Keane (New York: Guliford Press), 168–189.

Google Scholar

Weiss, D. S., and Marmar, C. R. (1997). “The Impact of Event Scale—Revised” in Assessing Psychological Trauma and PTSD. eds. J. P. Wilson and T. M. Keane (The Guilford Press), 399–411.

Google Scholar

Wind, T. R., Rijkeboer, M., Andersson, G., and Riper, H. (2020). The COVID-19 pandemic: the “black swan” for mental health care and a turning point for e-health. Internet Interv. 20:100317. doi: 10.1016/j.invent.2020.100317

PubMed Abstract | CrossRef Full Text | Google Scholar

Xiang, Y. T., Yang, Y., Li, W., Zhang, L., Zhang, Q., Cheung, T., et al. (2020). Timely mental health care for the 2019 novel coronavirus outbreak is urgently needed. Lancet Psychiatry 7, 228–229. doi: 10.1016/S2215-0366(20)30046-8

PubMed Abstract | CrossRef Full Text | Google Scholar

Xiao, C. (2020). A novel approach of consultation on 2019 novel coronavirus (COVID-19) - related psychological and mental problems: structured letter therapy. Psychiatry Investig. 17, 175–176. doi: 10.30773/pi.2020.0047

PubMed Abstract | CrossRef Full Text | Google Scholar

Xiao, H., Zhang, Y., Kong, D., Li, S., and Yang, N. (2020). Social capital and sleep quality in individuals who self-isolated for 14 days during the coronavirus disease 2019 (COVID-19) outbreak in january 2020 in China. Med. Sci. Monit. 26:e923921. doi: 10.12659/MSM.923921

PubMed Abstract | CrossRef Full Text | Google Scholar