Adherence Predictors in Internet-Delivered Self-Help Intervention for Life Stressors-Related Adjustment Disorder

Background: There is a growing body of evidence to show that low-intensity self-help internet-delivered interventions are effective in the treatment of mental disorders. Despite the promising effectiveness of internet-delivered interventions, there is still a challenge for mental health services to implement internet-delivered interventions in routine health care. The aim of this study was to analyze the predictors of adherence to a self-help internet-delivered intervention for adjustment disorder.

Methods: This was a secondary report of data, including unpublished data, from a randomized controlled trial of an internet-delivered self-help intervention for adjustment disorder. The study included 1,077 participants who had completed online baseline assessments. All participants had experienced significant life stressors over the last 2 years and had high levels of adjustment disorder symptoms. We analyzed the role of sociodemographic variables, pre-treatment adjustment disorder symptoms, outcome expectations, and perceived barriers to mental health services on the use of the intervention.

Results: We found that usage of internet-delivered self-help intervention and higher adherence was associated with female gender, greater age, higher pre-intervention outcome expectations, exposure to other forms of psychological therapy in addition to the internet-intervention at the time of the study, and reported perceived barriers to mental health services by the study participants.

Conclusions: The findings of the study indicated the importance of non-specific therapeutic factors on adherence during internet-delivered intervention. Perceived barriers to mental health services were associated with higher adherence to self-help intervention, which indicated that communities with restricted access to mental health services could benefit from low-intensity internet-delivered interventions.

Introduction

Adjustment disorder is among the most commonly diagnosed mental disorders worldwide (1, 2). Until very recently, adjustment disorder received little attention in research (3) and was often viewed as a sub-threshold or mild disorder within the broader spectrum of psychopathology (4). However, adjustment disorder may have a serious effect on a person's functioning following a significant life stressor and is associated with an increased suicide risk (5). The new definition of adjustment disorder given in the 11th edition of the International Classification of Diseases (ICD-11) released by the World Health Organization (WHO) (6) has facilitated research into adjustment disorder (7), including studies on interventions. The ICD-11 defines adjustment disorder as a disorder associated with stress that could be diagnosed as a response to an identifiable stressor with two specific symptoms: (1) preoccupation with the stressor and (2) failure to adapt (6).

It has been recently suggested that low-intensity internet-delivered interventions, including internet-delivered self-guided interventions, could be well-suited for the treatment of adjustment disorder (8). Adjustment disorder is associated with an identifiable stressor, and low-intensity intervention, delivered via the internet or a mobile application, could therefore provide patients with specific coping and stress-management skills targeted at dealing with the stressor-related symptoms. Recent controlled studies have demonstrated that self-guided low-intensity interventions can help to reduce symptoms of adjustment disorder (9–11). Findings from research on adjustment disorder internet intervention are in line with the study outcomes of internet interventions for other mental disorders. There is a considerable body of evidence that shows that internet-delivered interventions (12, 13), as well as internet-delivered self-help interventions, are effective in treating various mental disorders, such as depression, panic attacks, social anxiety, and posttraumatic stress disorder (14–17). However, despite these promising findings, mental health services still find it challenging to implement innovative internet-delivered interventions in routine health care (18).

Exploration of factors contributing to the effectiveness—and understanding the usage patterns—of internet-delivered interventions is important for the development and dissemination of internet-delivered self-help interventions in public health care systems (19). In real-life situations, patients in mental health care services often have comorbid conditions, diverse psychosocial backgrounds, different levels of previous exposure to life stressors, and various symptom levels, and they often come from diverse socioeconomic status groups. Furthermore, users of these interventions have their own expectations regarding the delivery and outcomes of treatment, which may impact their usage of interventions (15). Internet-delivered interventions can reduce barriers to the treatments of mental disorders (20). However, such relatively easier access to interventions via the internet could also be associated with higher attrition rates (21, 22). In contrast to traditional face-to-face treatments where patients need to invest more time and resources into setting up an appointment with a therapist or traveling to a therapy session, internet-delivered interventions can be accessed easier, and patients can more easily select treatment components to engage with or opt out of.

There are many different terms used to refer to participants' activity and their use of internet interventions, such as adherence, attrition, non-usage, and engagement. Several theoretical formulations of adherence that are applicable to internet interventions have been proposed recently (21). However, there is still no common and widely accepted definition or understanding of adherence in internet interventions (23). Despite these ongoing theoretical discussions, recent meta-analytical studies on adherence (24–26) have already identified a number of important predictors of adherence, such as female gender, education level, age, treatment expectations, and baseline symptom severity. The majority of internet-delivered intervention studies have focused on depression or anxiety disorders (12), and, consequently, we know significantly more about adherence predictors from intervention studies of these disorders. Surprisingly, there is a rather small number of studies on internet-delivered interventions for trauma- and stress-related disorders (14, 20, 27). Furthermore, there are only a few internet-delivered intervention studies on adjustment disorder (11, 28–30).

The aim of this exploratory study was to analyze the predictors of adherence and usage activity in self-help internet-delivered intervention for adjustment disorder. We aimed to estimate the role of variables known from previous adherence studies on internet-delivered low-intensity interventions, such as sociodemographic variables, including gender, education level, age, and symptom levels of adjustment disorder. We also evaluated the effects of pre-intervention patient outcome expectations on usage and adherence during an internet-delivered low-intensity self-help intervention, and this was based on previous findings that revealed the importance of expectations in therapy. Furthermore, internet-delivered interventions, especially self-guided interventions, are often regarded as effective low-cost solutions for overcoming barriers to mental disorder treatments, and we explored how perceived barriers to mental health services and previous exposure to traditional face-to-face treatment affects adherence to self-help intervention for adjustment disorder.

Methods

Intervention

The intervention was a brief modular internet-delivered CBT self-help intervention for adjustment disorder, and it is described in more detail in the study protocol (28). Access to the intervention website was provided via a computer or a mobile device with an internet connection. The intervention included four modules covering three intervention tasks each, thus totaling 12 tasks. The four modules were body relaxation, mindfulness, time management, and resolving conflicts in interpersonal relationships. Participant activities in the intervention included written tasks, relaxation, and mindfulness practice. The time needed to complete each task ranged from ~5 to 15 min. There was no predetermined sequence of intervention modules, and users of the intervention could choose any task from the list in all modules after logging in to the intervention website (28). After completing the intervention task, participants were asked to evaluate their current stress level, which allowed them to obtain the individualized interactive online chart of stress level changes over time for each participant. Participants received automatic weekly email reminders about participation in intervention after their inclusion in the study.

Data on the efficacy of the intervention revealed medium between-group effect sizes for the primary measures of adjustment disorder symptoms (d = 0.57) and the secondary measures of psychological well-being (d = 0.51) to the waiting list condition at one-month post-intervention follow-up (9). No additional benefit on primary or secondary outcomes was found in the RCT that evaluated the effects of additional therapist support on the self-help condition in the same intervention (10). Intervention effects d ranged from 0.51 to 0.67 in the self-help and self-help with therapist support on-demand group at the one-month follow-up.

Participants and Procedure

Secondary analyses were conducted using data from a previous controlled trial on a self-help intervention for adjustment disorder (10). Procedures of participant recruitment have been described in greater detail in the trial paper (10). In short, self-referred participants were invited to register for the self-help intervention via social networks and media advertisements. Information about the intervention was made available online via the intervention website. Potential participants were asked to register and provide informed consent before completing the initial online assessment procedure. Inclusion criteria were: (1) ≥18-year-old adults, (2) high levels of adjustment disorder symptoms, (3) exposure to a significant life stressor over the past 2 years, (4) access to a computer or any other device with screen and internet connection, and (5) sufficient Lithuanian language literacy. The study was approved by the Vilnius University Ethics Committee for Psychological Research and was registered in the Australian and New Zealand Clinical Trials Registry with the registration number ACTRN12616000883415.

Out of 1,607 participants who completed the baseline assessment, 530 were excluded for not having met the inclusion criteria—mainly the cut-off score for adjustment disorder symptoms. In total, 1,077 participants who met the inclusion criteria were randomized to two intervention groups: (1) self-help group and (2) self-help group with additional therapist support targeted at aiding the use of the program. The results did not show any significant differences between the two intervention groups as reported in our trial (10). Therefore, we pooled all participants into a single group for further analysis of the usage of intervention. As previously reported, significant intervention effects were achieved after 1 month (9, 10), and baseline and 1-month follow-up data were therefore used in the present analyses.

The demographic characteristics of all study participants are presented in Table 1. Mean age of participants was 35.25 years (SD = 11.70), and 81.6% of the participants were female (n = 879). Participants were exposed to an average of 4.05 (SD = 1.87) life stressors over the last 2 years, mostly in the form of work-related stressors 61% (n = 656), conflicts within the family 51% (n = 547), and financial difficulties 45% (n = 487).

TABLE 1

Table 1. Characteristics of study participants at baseline.

Measures

Demographic Data and the Use of Psychological Treatments

Gender, university degree, and living in a big city vs. small town/rural area were the main demographic variables used as predictors. Additionally, we asked two questions about participants' experiences of psychosocial treatments. We asked binary questions that required a yes/no response to find out if participants experienced barriers to mental health services and if they were using traditional face-to-face psychological treatment at the start of the internet-delivered intervention.

Life-Stressors and Adjustment Disorder

Adjustment disorder symptoms were measured using the brief version of the Adjustment Disorder New Model (ADNM-8) scale (31). The ADNM-8 is comprised of two parts: (1) a 17-item list of life-stressors and (2) an eight-item list of ICD-11 adjustment disorder symptoms. In the first part of the ADNM-8, participants were asked to indicate significant stressors that they had experienced in the last 2 years. In the second part of ADNM-8 participants were asked to rate each of the symptom items on a 4-point Likert type scale (1 = never; 2 = rarely; 3 = sometimes; and 4 = often). The total score of the ADNM-8 is the sum of all item responses. Internal consistency of the symptom part of the ADNM-8 scale was sufficient with Cronbach's alpha = 0.64.

Psychological Well-Being

The well-being of participants was measured with the WHO-5 Well-being Index (WHO-5) (32). The WHO-5 is a widely used measure of psychological well-being and is comprised of five items that reflect the presence and absence of positive well-being related to the quality of life (33). Items of WHO-5 are scored on a six-point Likert-type scale ranging from 0 = at no time to 5 = all the time. The sum of responses is converted to a scale ranging from 0 to 100 by multiplying the raw score by four. A score of 0 indicates the worst well-being, and a score of 100 represents the best possible well-being. Internal consistency for the WHO-5, measured by Cronbach's alpha, was 0.83 in this study.

Intervention Outcome Expectations

We used a 10-point Likert-type Subjective Units of Distress Scale (SUD) for the assessment of pre-intervention outcome expectations, following a similar approach as in a recent treatment outcome expectations study (34). Participants were asked to rate their current state at baseline using a single item “Please, indicate how do you feel today before using BADI program” on a 10-point SUD scale of natural numbers ranging from 1 = very bad to 10 = very good presented below the question. Participants were also asked to indicate their expected condition 1 month after the use of the self-help intervention on the same 10-point SUD scale using the item “Indicate how do you expect to feel after 1 month of usage of the BADI program”. Participants' outcome expectations were estimated by subtracting the expected SUD score from the current SUD score. A higher score indicated more optimistic outcome expectations, and the scores ranged from −5 to 9 in this study.

Data Analysis

A multivariate binary logistic analysis with all study variables included in the model was used to predict the usage of intervention to deal with the potential overfitting of the model (35). Data were analyzed using a two-step data analysis plan. Firstly, we searched for predictors of the completion of at least one module of intervention. This was necessary because a significant proportion of randomized participants did not use the self-help intervention after the enrollment. Secondly, we searched for the predictors of engaged participants who completed more than a median (≥4) of intervention tasks of the self-help intervention among those who started to use the intervention after randomization. Additionally, we estimated the predictors of usage activity by adjusting odds ratios (OR) to changes of adjustment disorder symptoms and psychological well-being measured at the one-month follow-up. All statistical analyses were performed using Statistical Package for the Social Sciences IBM SPSS version 24.0.

Results

One-third of all recruited participants 34.0% (n = 366) became intervention users and started using the intervention after the enrollment and randomization. Intervention usage was coded if a participant completed at least one intervention task in one of the modules during the first month since the inclusion in the study. The average number of tasks completed in 1 month of intervention per user was 4.85 (SD = 6.87; range 1–57) median = 3. Around a third of users, 34.2% (n = 125), completed ≥4 tasks of intervention and were considered high users of intervention. Post-intervention assessment data were available from 105 active users who completed 1-month follow-up assessments.

Descriptive statistics of study variables and correlations among these variables are presented in Table 1. Around one-third of participants (29.2%) reported perceived barriers to mental health services in a community. In total, 16.7 (n = 180) participants received other psychosocial treatment at the time of intervention.

Predictors of Intervention Usage vs. Non-usage

Binary logistic regression analysis (R² _Nagelkerke = 0.044) revealed that female gender (OR = 1.43, p = 0.046), greater age (OR = 1.03, p = < 0.001), exposure to traditional face-to-face therapy at baseline (OR = 1.65, p = 0.004), and higher intervention outcome expectations (OR = 1.12, p = 0.011) were significant predictors of non-usage vs. usage of the internet intervention measured by at least one completed module (see Table 2). Life stressors, adjustment disorder symptoms or well-being, did not predict the use of the intervention. Moreover, additional therapist support was not a significant predictor of non-usage vs. usage of the intervention after randomization.

TABLE 2

Table 2. Multivariate binary logistic analysis of self-help intervention adherence predictors.

Gender had a significant effect on usage vs. non-usage, χ²(2) = 4.51, p = 0.034. A higher percentage of women (35.5%, n = 312) started using intervention in contrast to that of men (27.3%, n = 54). Participants who actively started using intervention were older (M = 37.58, SD = 12.62) in comparison to non-users (M = 34.05, SD = 11.01), t_(655.741) = 4.74, p < 0.001. Intervention outcome expectations among users (M = 2.41, SD = 1.53) were higher than the non-users (M = 2.10, SD = 1.59), t₍₁₀₇₅₎ = 3.09, p = 0.002. Being in another therapy at the start of intervention had a significant effect on the use of intervention, χ²(2) = 7.93, p = 0.004. A higher percentage of the users of intervention were also undergoing another treatment (21.3%, n = 78) than that of the non-users (14.3%, n = 102).

Predictors of Low-Usage vs. High-Usage of the Intervention

Furthermore, we explored the predictors of activity in the intervention as measured by higher engagement in the program and completion of ≤ 4 modules. Binary logistic regression analysis (R² _Nagelkerke = 0.141) revealed that significant predictors were age (OR = 1.05, p < 0.001), barriers to mental services (OR = 1.76, p = 0.034), and being in another therapy (OR = 1.83, p = 0.036). Logistic regression (R² _Nagelkerke = 0.141) with adjusted OR's to therapeutic changes of the primary and secondary outcomes at one-month follow-up revealed such significant predictors as age (OR = 1.07, p = 0.008), education (OR = 0.25, p = 0.039), baseline adjustment disorder symptoms (OR = 0.78, p = 0.019), and pre-intervention outcome expectations (OR = 1.69, p = 0.016) (see Table 2).

Discussion

Adherence and attrition have often been identified as important issues in internet interventions (21–23, 36–38). This study contributed to this line of research with the analysis of the predictors of adherence in low-intensity self-help intervention for adjustment disorder. Adherence in this intervention was similar to other large-scale self-help interventions tested in naturalistic settings, which found high non-usage and low adherence rates, [e.g., (36)]. Furthermore, adherence issues were reported in the other recent study of internet self-help adjustment disorder intervention conducted in Switzerland with 48% drop-out rates (11).

Despite low adherence and high non-usage, the large sample size of the study allowed us to conduct an analysis of the intervention usage predictors, which revealed that sociodemographic variables, access to other forms of treatments, perceived barriers to mental health services, and higher outcome expectations were significant predictors of use of the intervention. Similar to the previous studies (24–26) we found that age and gender were important predictors of use of the intervention. Greater age was associated with high use of the intervention also after controlling for symptom change. Female gender was associated with usage vs. non-usage, but it did not predict high usage of the intervention in our study. Furthermore, we found no significant effect of education level on adherence in contrast to several other studies, [e.g., (26, 36)]. However, a large proportion of participants of this intervention had a university degree, which could have an impact on our findings.

The role of pre-treatment symptoms on adherence was ambiguous. Adjustment disorder symptoms at baseline were initially not associated with usage vs. non-usage of intervention. However, after adjusting for symptom change, we found that higher adherence was associated with lower levels of adjustment disorder symptoms at baseline. Furthermore, we found that participants who had already received traditional face-to-face therapy were more adherent users of the intervention. This study did not exclude participants engaged in other forms of therapy, and a surprisingly large proportion of participants at baseline “self-blended” participation in the internet-delivered self-help program with their ongoing other therapy. This exposure to another treatment contributed to higher adherence and was in line with the finding of other studies. Models of blended treatments are expected to provide the advantages of both internet-delivered and face-to-face interventions (18). However, a therapy support condition did not have any effect on adherence in this study.

A significant predictor of higher adherence was perceived barriers to mental health services of study participants. eHealth interventions provided via technology using the internet or mobile applications can help overcome treatment barriers (20) and may contribute to the development of mental health services Around a third of participants reported that they registered for this intervention because they could not access psychological treatments in their community. Participants who reported barriers to mental health services were more adherent users of this low-intensity self-help intervention for adjustment disorder. This finding indicates that usage of internet interventions can be associated with a lack of available mental health services in a community, and people might engage more in internet interventions if they are offered in such developing communities.

Another significant predictor was pre-treatment outcome expectations. Non-usage of participants was predicted by significantly lower pre-intervention outcome expectations. Participants who expected their condition to improve used the intervention and were high users after adjusting findings for intervention outcomes. In line with the previous studies, we found that expectations predicted adherence to the internet-delivered intervention (15, 25, 39, 40). Our findings suggest that outcome expectations might be a relevant and useful predictor of drop-out rates in unguided self-help internet interventions. Participants' with more optimistic pre-treatment outcome expectations were more engaged in the intervention, and high usage was associated with better therapeutic outcomes in this intervention (10). The study also indicated that greater outcome expectations, as a non-specific therapeutic factor, is an important predictor of higher adherence in low-intensity self-help internet-delivered interventions, and it should be addressed in research and implementation of internet interventions in healthcare.

Several limitations of this study should be mentioned. The study sample was large, which enabled us to search for various predictors of adherence in this intervention. However, we included only sociodemographic and psychosocial variables in the analysis of adherence. It is possible that a variety of other barriers and facilitators, such as technological usability, perceived utility, and design of intervention, could contribute to adherence in eHealth interventions (38); thus, the role of these factors could be tested in future adherence studies. Moreover, there are ongoing debates about how to define and measure adherence in internet interventions (21, 37). We analyzed the non-usage, low-usage, and high-usage of intervention in our adherence analysis using the self-reported amount of completed tasks in the intervention to register the usage after access to the intervention had been provided to participants. While this approach is often used in internet intervention research, other ways of registering participant online activity in an intervention using such methods as advanced software solutions or using eye movement tracking could give additional information in further studies.

This study did not reach out to participants characterized by non-usage or low-usage. Interviews with the study participants who refused to use intervention or used it only very little could contribute valuable information on subject of the predictors of adherence. Furthermore, our approach to the assessment of expectations in this study had limited psychometric properties. Further studies are needed with more elaborate methods to measure patient outcome expectations in internet-delivered self-help interventions. Finally, this intervention was offered in Lithuania, a country with limited access to mental health services, and this could have also impacted the participants' usage patterns in the study.

Despite these limitations, the study contributed to the growing knowledge about predictors of adherence in low-intensity self-guided internet interventions with findings about the predictors of intervention on adjustment disorder in a new cultural context. In line with previous studies, we demonstrate that female gender, greater age, and higher expectations are important predictors of adherence across various mental disorders in internet-based treatments, including adjustment disorder. Furthermore, this study points out that perceived barriers to mental health services can increase adherence to self-help internet-interventions, and communities with restricted access to mental health services could benefit from such low-intensity internet-interventions.

Data Availability Statement

The datasets generated for this study are available on request to the corresponding author.

Ethics Statement

The study was approved by the Vilnius University Research Ethics Committee for Psychology Research. The study was conducted in accordance with the principles of the Declaration of Helsinki as well as the national Lithuanian and international ethical regulations for research with human subjects. All participants provided written informed consent for participation in the study on a secure online website specially designed for this study.

Author Contributions

EK: principal investigator, study design, data analysis, the concept of the article, and writing the first draft. JE: data collection, contribution to study planning, and significant revisions of the manuscript. MO: interpretation of results and significant revisions of the manuscript. PZ: data collection and significant revisions of the manuscript. GA: interpretation of results and critical revisions of the manuscript.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The handling Editor declared a past co-authorship with one of the authors EK.

References

1. Evans SC, Reed GM, Roberts MC, Esparza P, Watts AD, Correia JM, et al. Psychologists' perspectives on the diagnostic classification of mental disorders: results from the WHO-IUPsyS global survey. Int J Psychol. (2013) 48:177–93. doi: 10.1080/00207594.2013.804189

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Reed GM, Mendonça Correia J, Esparza P, Saxena S, Maj M. The WPA-WHO global survey of psychiatrists' attitudes towards mental disorders classification. World Psychiatry. (2011) 10:118–31. doi: 10.1002/j.2051-5545.2011.tb00034.x

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Casey P. Adjustment disorder: new developments. Curr Psychiatry Rep. (2014) 16:451. doi: 10.1007/s11920-014-0451-2

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Bachem R, Casey P. Adjustment disorder: a diagnosis whose time has come. J Affect Disord. (2017) 227:243–53. doi: 10.1016/j.jad.2017.10.034

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Casey P, Jabbar F, O'Leary E, Doherty AM. Suicidal behaviours in adjustment disorder and depressive episode. J Affect Disord. (2015) 174:441–6. doi: 10.1016/j.jad.2014.12.003

PubMed Abstract | CrossRef Full Text | Google Scholar

6. World Health Organization. International classification of diseases for mortality and morbidity statistics (11th Revision). (2018). Available online at: https://icd.who.int/browse11/l-m/en

7. Kazlauskas E, Zelviene P, Lorenz L, Quero S, Maercker A. A scoping review of ICD-11 adjustment disorder research. Eur J Psychotraumatol. (2017) 8(Suppl. 7):1421819. doi: 10.1080/20008198.2017.1421819

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Maercker A, Bachem RC, Lorenz L, Moser CT, Berger T. Adjustment disorders are uniquely suited for eHealth interventions: concept and case study. JMIR Ment Heal. (2015) 2:e15. doi: 10.2196/mental.4157

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Eimontas J, Rimsaite Z, Gegieckaite G, Zelviene P, Kazlauskas E. Internet-based self-help intervention for ICD-11 adjustment disorder: preliminary findings. Psychiatr Q. (2018) 89:451–60. doi: 10.1007/s11126-017-9547-2

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Eimontas J, Gegieckaite G, Dovydaitiene M, Mazulyte E, Rimsaite Z, Skruibis P, et al. The role of therapist support on effectiveness of an internet-based modular self-help intervention for adjustment disorder: a randomized controlled trial. Anxiety Stress Coping. (2018) 31:146–58. doi: 10.1080/10615806.2017.1385065

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Moser C, Bachem R, Berger T. ZIEL : internet-based self-help for adjustment problems : results of a randomized controlled trial. J Clin Med. (2019) 8:1655. doi: 10.3390/jcm8101655

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Carlbring P, Andersson G, Cuijpers P, Riper H, Hedman-Lagerlöf E, Castro A, et al. Internet-based vs. face-to-face cognitive behavior therapy for psychiatric and somatic disorders: an updated systematic review and meta-analysis. Cogn Behav Ther. (2018) 47:1–18. doi: 10.1080/16506073.2017.1401115

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Cuijpers P, Hedman E, Riper H, Carlbring P, Andersson G. Guided internet-based vs. face-to-face cognitive behavior therapy for psychiatric and somatic disorders: a systematic review and meta-analysis. World Psychiatry. (2014) 13:288–95. doi: 10.1002/wps.20151

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Lewis CE, Farewell D, Groves V, Kitchiner NJ, Roberts NP, Vick T, et al. Internet-based guided self-help for posttraumatic stress disorder (PTSD): randomized controlled trial. Depress Anxiety. (2017) 34:555–65. doi: 10.1002/da.22645

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Boettcher J, Renneberg B, Berger T. Patient expectations in internet-based self-help for social anxiety. Cogn Behav Ther. (2013) 42:203–14. doi: 10.1080/16506073.2012.759615

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Karyotaki E, Riper H, Twisk J, Hoogendoorn A, Kleiboer A, Mira A, et al. Efficacy of self-guided internet-based cognitive behavioral therapy in the treatment of depressive symptoms. JAMA Psychiatry. (2017) 74:351. doi: 10.1001/jamapsychiatry.2017.0044

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Berger T, Hämmerli K, Gubser N, Caspar F. Internet-based treatment of depression: a randomized controlled trial comparing guided with unguided self-help. Cogn Behav Ther. (2011) 40:4:251–66. doi: 10.1080/16506073.2011.616531

CrossRef Full Text | Google Scholar

18. Andersson G, Titov N, Dear BF, Rozental A, Carlbring P. Internet-delivered psychological treatments: from innovation to implementation. World Psychiatry. (2019) 18:20–8. doi: 10.1002/wps.20610

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Frewen P, Schmahl C, Olff M. Interdisciplinary approaches to understand traumatic stress as a public health problem. Eur J Psychotraumatol. (2017) 8(Suppl. 5):1441582. doi: 10.1080/20008198.2018.1441582

CrossRef Full Text | Google Scholar

20. Olff M. Mobile mental health: a challenging research agenda. Eur J Psychotraumatol. (2015) 6:27882. doi: 10.3402/ejpt.v6.27882

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Ryan C, Bergin M, Wells JS. Theoretical perspectives of adherence to web-based interventions: a scoping review. Int J Behav Med. (2018) 25:17–29. doi: 10.1007/s12529-017-9678-8

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Eysenbach G. The law of attrition. J Med Internet Res. (2005) 7:e11. doi: 10.2196/jmir.7.1.e11

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Sieverink F, Kelders SM, Gemert-Pijnen V. Clarifying the concept of adherence to ehealth technology: systematic review on when usage becomes adherence. J Med Internet Res. (2017) 19:e402. doi: 10.2196/jmir.8578

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Karyotaki E, Kleiboer A, Smit F, Turner DT, Pastor AM, Andersson G, et al. Predictors of treatment dropout in self-guided web-based interventions for depression: an “individual patient data” meta-analysis. Psychol Med. (2015) 45:2717–26. doi: 10.1017/S0033291715000665

CrossRef Full Text | Google Scholar

25. Beatty L, Binnion C. A systematic review of predictors of, and reasons for, adherence to online psychological interventions. Int J Behav Med. (2016) 23:776–94. doi: 10.1007/s12529-016-9556-9

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Castro A, López-del-Hoyo Y, Peake C, Mayoral F, Botella C, García-Campayo J, et al. Adherence predictors in an internet-based intervention program for depression. Cogn Behav Ther. (2018) 47:246–61. doi: 10.1080/16506073.2017.1366546

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Wang J-P, Maercker A. Web-based interventions for traumatized people in mainland China. Eur J Psychotraumatol. (2014) 5:26519. doi: 10.3402/ejpt.v5.26519

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Skruibis P, Eimontas J, Dovydaitiene M, Mazulyte E, Zelviene P, Kazlauskas E. Internet-based modular program BADI for adjustment disorder: protocol of a randomized controlled trial. BMC Psychiatry. (2016) 16:264. doi: 10.1186/s12888-016-0980-9

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Rachyla I, Pérez-Ara M, Molés M, Campos D, Mira A, Botella C, et al. An internet-based intervention for adjustment disorder (TAO): study protocol for a randomized controlled trial. BMC Psychiatry. (2018) 18:161. doi: 10.1186/s12888-018-1751-6

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Quero S, Rachyla I, Mol M, Mor S, Tur C, Cuijpers P. Can between-session homework be delivered digitally? A pilot randomized clinical trial of CBT for adjustment disorders. Int J Enviromental Res Public Heal. (2019) 16:3824. doi: 10.3390/ijerph16203842

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Kazlauskas E, Gegieckaite G, Eimontas J, Zelviene P, Maercker A. A brief measure of the international classification of diseases-11 adjustment disorder: investigation of psychometric properties in an adult help-seeking sample. Psychopathology. (2018) 51:10–5. doi: 10.1159/000484415

PubMed Abstract | CrossRef Full Text | Google Scholar

32. World Health Organization. Info Package: Mastering Depression in Primary Care, Version 2.2. Frederiksborg: WHO, Regional Office for Europe, Psychiatric Research Unit (1998).

33. Topp CW, Østergaard SD, Søndergaard S, Bech P. The WHO-5 well-being index: a systematic review of the literature. Psychother Psychosom. (2015) 84:167–76. doi: 10.1159/000376585

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Kazlauskas E, Jovarauskaite L, Mazulyte E, Skruibis P, Dovydaitiene M, Eimontas J, et al. ‘It will get even better’: preliminary findings from a trauma-focused psychotherapy effectiveness study reveal false positive patients ' long-term outcome expectations after the treatment. Nord J Psychiatry. (2017). 71:277–81. doi: 10.1080/08039488.2016.1276623

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Baybak M. What you see may not be what you get: a brief, nontechnical introduction to overfitting in regression-type models. Psychosom Med. (2004) 66:411–21. doi: 10.1097/01.psy.0000127692.23278.a9

CrossRef Full Text | Google Scholar

36. Murray E, White IR, Varagunam M, Godfrey C, Khadjesari Z, McCambridge J. Attrition revisited: adherence and retention in a web-based alcohol trial. J Med Internet Res. (2013) 15:e162. doi: 10.2196/jmir.2336

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Short CE, DeSmet A, Woods C, Williams SL, Maher C, Middelweerd A, et al. Measuring engagement in eHealth and mHealth behavior change interventions: viewpoint of methodologies. J Med Internet Res. (2018) 20:e292. doi: 10.2196/jmir.9397

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Simblett S, Greer B, Matcham F, Curtis H, Polhemus A, Ferrão J, et al. Barriers to and facilitators of engagement with remote measurement technology for managing health: systematic review and content analysis of findings. J Med Internet Res. (2018) 20:e10480. doi: 10.2196/10480

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Watson HJ, Levine MD, Zerwas SC, Hamer RM, Crosby RD, Sprecher CS, et al. Predictors of dropout in face-to-face and internet-based cognitive-behavioral therapy for bulimia nervosa in a randomized controlled trial. Int J Eat Disord. (2017) 50:569–77. doi: 10.1002/eat.22644

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Kok RN, Beekman ATF, Cuijpers P, van Straten A. Adherence to a web-based pre-treatment for phobias in outpatient clinics. Internet Interv. (2017) 9:38–45. doi: 10.1016/j.invent.2017.05.004

PubMed Abstract | CrossRef Full Text | Google Scholar