Online Indicated Preventive Mental Health Interventions for Youth: A Scoping Review

Objective: Between the ages of 12 and 25 the onset of mental disorders typically occurs, and the burden of mental health problems is greatest for this group. Indicated preventive interventions to target individuals with subclinical symptoms to prevent the transition to clinical levels of disorders have gained considerable traction. However, the threshold to seek help appears to be high even when help is needed. Online interventions could offer a solution, especially during the COVID-19 pandemic. This scoping review will present an overview of the recent research of indicated online preventive interventions for youth (12–25 years) experiencing the early stages of mental health complaints with the aim of identifying the nature and extent of the research evidence.

Methods: The 5-stage framework by Arksey and O'Malley was used. Academic literature published from 2013 onwards in printed or electronic format was included from Scopus, PsychINFO, and Ovid MEDLINE(R) ALL.

Results: The search yielded 11,122 results, with the final selection resulting in inclusion of 30 articles for this review. In total, the articles included 4,950 participants. 26.7% of the selected articles focused on youth between 12 and 25 years. Of the articles 60% did not screen for, nor exclude participants with clinical levels of symptoms. Most studies used a common evidence-based therapy for the disorder-category targeted. More than half of the online interventions included some form of human support. Adherence levels ranged between 27.9 and 98%. The results indicate general effectiveness, usability and acceptability of online indicated preventive interventions. The most commonly used approach was CBT (n = 12 studies). Studies varied in their size, rigor of study, effectiveness and outcome measures. Online interventions with a combination of clinical and peer moderation (n = 3 studies) appear to result in the most stable and highest effect sizes.

Conclusion: Online indicated preventive mental health interventions for youth with emerging mental health issues show promise in reducing various mental health complaints, and increasing positive mental health indicators such as well-being and resilience. Additionally, high levels of usability and acceptability were found. However, the included studies show important methodological shortcomings. Also, the research has mainly focused on specific diagnostic categories, meaning there is a lack of transdiagnostic approaches. Finally, clear definitions of- as well as instruments to measure- emerging or subclinical mental health symptoms in youth remain are missing.

Introduction

Mental health can be defined as “a state of well-being in which every individual realizes his or her own potential, can cope with the normal stresses of life, can work productively and fruitfully, and is able to make a contribution to her or his community” (1). Nonetheless, this state is disrupted in half (2) to almost three quarters (3) of all people living in the western world at some point in their life, and in 1 in 4 in any given year (4). The onset of mental disorders typically occurs in childhood and adolescence (2, 5), with 75% of mental disorders beginning before the age of 25 (5, 6). The waiting lists to receive care accordingly are continually growing (7), and costs associated with mental illness are substantial [e.g., (8)], and growing (9). Moreover, the burden of mental health problems is substantial for these individuals and is indicated by negative effects upon quality of life (10), life expectancy (11) social functioning, ability to work (10), and (self-)stigmatization (10, 12, 13). This burden has been found to be the greatest in young people aged between 15 to 25 years (6).

Increased attention has been paid at interventions aimed at youth with emerging symptoms to treat them as early as possible in the development of a mental disorder, for example, during the peak period of risk for onset, with a focus on both symptomatic as well as functional recovery (14). Prevention and early intervention are recognized as key elements for minimizing the psychosocial and economic impacts of any potentially serious health condition (15, 16). Previous research has shown the effectiveness of face-to-face psychosocial preventive interventions for youth. Improvements in behavioral and social outcomes were observed as well as a decrease in the proportion of participants transitioning from mental health complaints to mental disorders (17, 18). Unfortunately, the gap between needing help and seeking help is substantial. Only one in three young people seek help for their mental health problems (19, 20), and most individuals present to services at a much later stage (21–23). Subsequently these individuals present with more developed and severe problems that are more difficult to treat, and have more functional and social consequences since the mental illness strikes in a critical developmental period where social, vocational and educational milestones were to be achieved (15, 24). In other words, even though help for mental health problems is needed in adolescence or young adulthood, the threshold to seek it appears high. Perceived barriers for help-seeking in young people include negative attitudes toward seeking help (e.g., internalized stigma or shame), practical concerns (e.g., costs and transportation), believing they have to manage the problem on their own, downplaying their problems, doubts concerning the effectiveness of treatment, the unavailability of help (19, 25) and perceived public mental-health stigma (26).

Online interventions might offer a solution to the perceived barriers. Advantages of online interventions are the possibility to receive help anonymously, and increased convenience because individuals can choose when and where they access help (27–29). Moreover, online interventions have the potential to reach people who are unwilling or unable to receive face-to-face help, for example, those who live in remote areas or those with decreased mobility (30). Online interventions may be especially appealing to young people as most youth are familiar and competent with using digital technology. This is illustrated by data that indicate that 96% of European youth (aged 16–24) use the internet regularly (31). Furthermore, research shows that young people report using the internet to find information pertaining to mental health (32), they have positive perceptions about using the internet for mental health related-issues (33), and clinicians hold positive attitudes toward using technology for treatment too (34). Moreover, online interventions hold the potential to decrease costs for the individual and the healthcare system. Lastly, online interventions offer mental health care from home during the current COVID-19 pandemic (35, 36) which may be especially important for individuals with emerging complaints who are prone to developing more severe mental health issues (37).

Online indicated preventive interventions for individuals with an indicated need for care, that is, youth with emerging complaints, offer a promising approach to address this unmet need. From a resource perspective, it may be more feasible to target individuals with subclinical symptoms than non-symptomatic individuals who may not have a need for an intervention (38). The clinical staging model (39, 40) illustrates the differentiation between subclinical symptom clusters (stage 1a or 1b) and the onset of more discrete syndromes or clinical entities (stage 2, 3, and 4). Previous meta-analyses investigating face-to-face preventive interventions in youth have also shown that indicated preventive interventions have larger intervention effects than universal preventive interventions (38, 41).

The effectiveness of online indicated mental health preventive interventions for young people has been addressed in four systematic reviews and two meta-analyses between 2014 and 2016. In these reviews “youth” is defined as between the ages of 12 and 25, in concordance with most international definitions of youth as well as governmental and youth mental health institutions (42, 43). Interestingly, the majority of participants included in these reviews are youth with subclinical symptoms (stage 1a or 1b), however some participants might be in a later stage since a clinical diagnosis was not an exclusion criterion in most included studies. Also, the transition to clinical disorders, which is the established primary outcome of indicated preventive treatment trials, was generally not measured. Lastly, the reviews varied quite substantially in their scope. Rice et al. (44) conducted a systematic review including studies focusing on online and social networking as indicated preventive interventions for the treatment of depressive symptomatology in youth (12–25 years). The overall finding was that online interventions appear to be promising in reducing depression symptomatology in young people. The systematic review of Ali et al. (45) included six studies targeting online peer-to-peer support for young people (12–25 years) with emerging mental health problems. Two out of six studies found support for the effectiveness of online peer-to-peer support although an overall lack of quality of the studies was found, and the type of moderation used in the studies was poorly reported. In 2015, Pennant et al. (46) included 27 studies in their systematic review and meta-analysis researching both indicated and universal preventive computerized therapies for anxiety and depression in children and young people (12–25 years). It was found that indicated and general preventive intervention had positive effects for reducing symptoms of anxiety and depression. However, follow-up data about long-term effects was scarce, and the authors stated that the magnitude of the effects needed to be interpreted cautiously due to the heterogeneity associated with a number of outcomes and predominantly low quality of the evidence. In all three systematic reviews it was not specified whether the indicated prevention had an effect on the rate of transition to clinical disorders, since outcome measures included solely measures of symptom severity.

O'Dea et al. (47) reviewed the evidence for online interventions for universal and indicated prevention targeting depression and anxiety symptoms and disorders in youth (12–18 years). They included six studies, and found positive effects on symptoms in all but one trial. They concluded that there are a number of gaps in the literature, for example a lack of cost-effectiveness data, and heterogeneity in sample sizes, randomization procedures, and outcome measures, making it difficult to compare trial results. There was only one study that measured the effect of indicated prevention on the development of clinical levels of depression; it was found that Cognitive Behavioral Therapy (CBT) lowered this risk. Ebert et al. (48) conducted a meta-analysis including internet and computer-based cognitive behavioral therapy for anxiety and depression in children and youth (< 25 years). They included 13 Randomized Control Trials (RCTs) and found an overall effect size of g = 0.72, reflecting a decrease in symptom severity. Again, the authors reported high heterogeneity and long-term effects of the studies. In the most recent meta-analysis of Conley et al. (49) the impact of universal and indicated preventive technology-delivered interventions for higher education students (age not specified) was investigated. They included 22 universal and 26 indicated prevention studies, and found larger positive treatment effects for indicated preventive interventions than universal preventive interventions. The authors reported important limitations on the experimental rigor and recommended that future research should for example provide more details on participant characteristics, and intervention content; and collect follow-up data.

While Rice et al. (44), Ali et al. (45), Pennant et al. (46), O'Dea etal. (47), Ebert et al. (48), and Conley et al. (49) included online indicated preventive interventions in their reviews, universal preventive interventions were included as well. To our knowledge no more recent reviews have been published. Moreover, there have been no reviews specifically of studies focusing on the effect of indicated preventive interventions provided online for youth. This scoping review will present an overview of the recent research of indicated online preventive interventions for youth experiencing the early stages of mental health complaints with the aim of identifying the nature and extent of the research evidence.

Methods

Framework

We utilized the 5-stage framework by Arksey and O'Malley (50) developed for reporting a scoping review. This framework entails the following stages: (1) identifying the research question, (2) identifying relevant studies, (3) selection of studies, (4) charting the data, and (5) summarizing and reporting the results.

Research Question

The focus of this review was to present an overview of indicated online preventive interventions for emerging mental health symptoms in youth and aimed to identify the nature and extent of the research evidence. This led to the following guiding question: What is known in the literature about the use of indicated online preventive interventions for youth with emerging mental health problems?

Search Strategy

A search was conducted together with the university librarian with experience in conducting reviews (JD). Appendix 1 displays the used search terms. Academic literature published in printed or electronic format was included from the following sources: Scopus, PsychINFO, and Ovid MEDLINE(R) ALL. Articles written in the English and Dutch language were retrieved. In order to ascertain recent findings, articles from 2013 onwards were included in this review. Study designs were limited to randomized controlled trials, quasi-experimental study designs and experimental studies without a comparison group. See Supplementary Material for the search criteria.

Eligibility Criteria

The eligibility criteria were determined to find all articles relevant to the research question. A highlighted summary of the main inclusion/exclusion criteria covered here is provided in Table 1.

TABLE 1

Table 1. Eligibility criteria.

Population

Studies eligible for inclusion were those containing a sample of youth, defined in concordance to most international, governmental and youth institutional definitions (42, 43) as participants aged 12–25, who have signs or symptoms of a mental disorder that are either self-reported or assessed via a screening process. Studies were included if they included participants below age 12 or above age 25 as long as the majority of the participants was between ages 12–25. This was assessed by the mean age of the participants, the standard deviation of age, and the proportion of participants within this age range (studies were included only when more than half of the participants fell within the range). Articles were excluded when the age range of the included participants was outside 12–25 and the mean age was not reported.

Studies were also excluded if they had selected participants with a known mental disorder (either self-reported or diagnosed by a clinician). However, since most indicated preventive interventions did not screen for the presence of a mental disorder, studies were not excluded if they did not screen for mental disorders. The risk of reducing specificity by including these articles was deemed essential in order to have a broader scope to best summarize the relevant research.

Intervention

Interventions needed to be delivered primarily in an online (digital) setting (defined as: online-, Internet, Web-, or mobile-based) and focused on indicated prevention of mental disorders in youth aged 12–25 with signs or symptoms of a mental disorder. Only online indicated preventive interventions were included [defined as: preventive online interventions which target individuals who are showing early symptoms and signs of a disorder to prevent progression from clinical stages 1a and 1b to stages 2–4; (40)].

Interventions that are primarily face-to-face with some additional online content were also excluded. Lastly, process evaluation studies were excluded (although important implementation findings may be highlighted in the identified studies).

Comparison and Study Type

Studies comparing online prevention programs, websites, apps, games, social media or smartwatch interventions compared to no intervention, waiting list, or face-to-face interventions were included. No exclusion criteria were applied to the type of comparator used. RCTs, quasi-experimental study designs and experimental studies without a comparison group were eligible for inclusion in this review. No minimum follow-up time period was specified. Observational studies or protocols were excluded. There were no restrictions by timing or type of setting. Only articles written in English or Dutch were included.

Outcomes

The following a priori determined outcome measures were included: negative mental health indicators, for example, depression, anxiety, psychological distress and suicidal behavior, and transition to symptom levels above clinical diagnostic threshold; positive mental health indicators, for example, self-efficacy, coping skills, resilience, emotional well-being, self-esteem; and well-being indicators, for example, social participation, quality of life, social functioning, empowerment, communication, social support. Outcomes that were not indicative of mental health and well-being were excluded.

Charting the Data

Data were extracted by two out of five reviewers independently and in duplicate with the use of standardized data extraction forms. Any disagreements were resolved through discussion and by a third reviewer. To ensure accuracy, the extracted data was reviewed by experts in the field based on relevance. Data that was extracted included: intervention characteristics, methodology, program outcomes and information about program completion, engagement rate, and inclusion of human support.

The following data items were extracted: (1) intervention name, author, country where research was conducted, year of publication, (2) intervention characteristics (type of intervention, duration, target group), (3) method (study design, sample, selection biases, confounders, blinding, data collection, analysis, intervention integrity), (4) program outcomes, and (5) adherence (non-completion/dropout rates). In case of deficient or missing outcome data, authors were contacted and data were requested.

Summarizing and Reporting Results

To determine the extent and nature of the studies, a numerical analysis was conducted, using tables and chart mappings. Using conventional content analysis the descriptive data was analyzed. The user-centered design framework was followed, which states that two reviewers have to examine the data and identify codes relative to the findings. The codes were grouped according to themes to summarize the literature and answer the research question.

Results

The search conducted by the university librarian (JD) yielded 11,122 results. After the screening process, the remaining 77 articles were assessed for relevance using the eligibility criteria in Table 1 by six experts (DN, MAJ, JG, TvA, AP, and CM), and an additional two articles were provided by experts, resulting in the final inclusion of 30 articles for this scoping review. For the study selection procedure, see the PRISMA diagram in Figure 1.

FIGURE 1

Figure 1. PRISMA diagram.

For all study characteristics, see Table 2.

TABLE 2

Table 2. Study characteristics.

Timeline

Six studies were published in 2016. The number of studies regarding preventative online measures for youth saw a slight decline in the years 2017 (n = 5) and 2018 (n = 4), followed by a spike in publications occurring in 2019 (n = 13). At the present moment, there have been an additional two studies published in 2020.

Geographic Location

The included studies were predominantly conducted in Australia (n = 9) and the United States of America (n = 7), accounting for more than half (53%) of the contributions in this scoping. The remaining studies were conducted in the Netherlands (n = 5), Canada (n = 1), Finland (n = 1), Germany (n = 1), Japan (n = 1), Sweden (n = 1), Switzerland (n = 1), and the United Kingdom (n = 1). See Table 2.

Study Design

The included studies consisted primarily of RCTs (n = 23). Of these 23 RCTs, two were stratified and one was clustered. The remaining were experimental designs without comparison groups (n = 6). Another study that was included based on the preliminary inclusion criteria was identified as a mixed design (n = 1). Almost half (46%) of all included studies made use of a follow-up procedure, either within 3 months (n = 4), between 3 to 6 months (n = 6), or after 7 months and beyond (n = 4). See Table 3.

TABLE 3

Table 3. Study outcome measures and results.

Sample Size and Study Population

See Table 2 for sample characteristics. In total, the 30 articles included 4,950 participants in their studies, (165 participants per study on average), ranging from 14 to 536 participants. In total, 26,7% of the selected articles focused on young people between the ages of 12 and 25, 30% focused solely on youth from the ages 11 to 19, and 43,3% focused on adolescents of 17 years and older. The weighted mean age of participants over all articles was 18,9 years. In all articles but 1 (77), the majority of participants were female. 33,3% of the articles focused on depression symptoms or disorders, 10% on anxiety related symptoms or disorders, and 20% on either depression or anxiety symptoms or disorders. 6,7% of the articles focused on symptoms of psychosis, 3,3% on suicide, and 3,3% on depression or psychosis symptoms. 23,3% of the articles focused on elevated stress and had a transdiagnostic approach.

Of the included articles 40% focused solely on indicated prevention and excluded participants who met criteria for a mental disorder. The other 60% did not screen for, or exclude participants with presence of a mental disorder. Therefore, these studies were not strictly indicated prevention studies despite using the terms “prevention” or “indicated” in the publications. The measures used to establish whether a participant had emerging complaints vs. a known mental disorder varied substantially over the studies, ranging from self-report (e.g. “Have you ever been diagnosed with a mental disorder?”) to a structured DSM-5 interview with clear cut-offs for clinical levels of mental disorders. A specific and validated clinician-rated instrument was used only for identifying the subclinical complaints of psychosis (UHR-state) (51). Lastly, the studies used different at-risk definitions, which indicates that a clear consensus on definitions is also missing.

Intervention Type and Duration

Most of the studies used a common evidence-based therapy for the disorder-category targeted. The most commonly used approach was CBT (n = 17), of which several studies (n = 5) combined this approach with another, for example Interpersonal Therapy (58), Motivational Interviewing (MI) (57), third wave techniques (61), mindfulness (59), and strength-based interventions such as mindfulness, self-compassion and positive psychology (52). One study researched Cognitive Training (CT) (71), and another mindfulness and strength training (51). Three studies investigated Acceptance and Commitment Therapy (ACT) (54, 67, 68). One study used cognitive bias modification (65), and two attentional bias modification (70, 80). Less common approaches were used by Takahashi et al. (77) using a motion picture producing app; Anttilaet al. (53) who used self-determination as a framework and allowed participants chose relevant subjects to discuss; and Hides et al. (63) using Music Therapy. In the study by Traber-Walker et al. (79), an app was used as an adjunct to face-to-face therapy; for example, containing information and registration forms. Lastly, three studies offered a decision aid to help find the right treatment, and find information (e.g., lifestyle advice, psychoeducation); and did not provide further treatment on their platforms (62, 74, 75). More than half of the online interventions included some form of human support, ranging from sending reminders, to group or individual sessions with clinicians. In most studies the treatments were based on specific theoretical bases for the disorder-category being targeted, for example, CBT for depressive symptoms, with standard modules and options to tailor the treatment to the individual's needs. See Table 3.

The range of duration of the online intervention was 50 min to 16 weeks. See Table 3. The three decision aid programs had the shortest duration, namely only one session (50 or 90 min; and not specified). Not taking these three studies into account, the online treatment programs varied in duration from 2 weeks to 16 weeks.

Adherence

The included studies showed a varied range of adherence to the programs, see Table 2. The adherence percentages were either adopted directly from the reported number provided by the authors of the included papers, or calculated based on the percentage of participants who either completed at least half of the program in the experimental condition, or dropped out during the experimental phase. Adherence levels ranged from 27.9% of participants (73) to 98% of participants (69) with a mean adherence percentage of 63.81%. However, caution should be exercised in the depiction of these numbers due to the lack of consensus in measuring adherence.

Outcome Measures

Outcome measures consisted/included factors such as: depression, anxiety, social anxiety, distress, eating pattern disturbances, excessive drinking, suicidal ideation, mindfulness, self-efficacy, self-esteem, cognitive functioning, psychological inflexibility, social dysfunctioning, quality of life, rumination, emotional regulation and various other factors. Noticeable in the selected articles is that many different questionnaires (with varying validity) were used to measure a single psychological construct such as depression. For a complete overview, see Table 3.

Key Findings

For a full overview of the overall findings, including the outcome variables, the study design, the number of participants in each study and whether human support was used in the intervention, see Table 3.

The quality of the selected articles differed considerably. Firstly, the number of participants included in the selected studies showed a wide range (from n = 14 to n = 536). Moreover, various articles did not provide data on effect sizes of significant effects (n = 7). In addition, some articles deconstructed or created questionnaires without reporting their psychometric properties. Lastly, some studies did not have a control group to compare the effects of the interventions to (n = 6). Therefore, the results are to be interpreted with caution, and the authors refrain from making conclusive comparisons between studies.

The results of the selected articles show that online preventive interventions are generally effective in reducing negative outcome measures such as depressive symptoms (n = 16), anxiety (n = 5) and stress (n = 6). As for the positive outcome measures, the majority of articles measuring positive health indicators showed that online preventive interventions significantly improves positive mental health factors such as well-being (n = 4) and social functioning (n = 2). However, a large part of the selected articles measuring positive health indicators also showed non-significant improvement intervention on factors such as self-efficacy (n = 4) and self-esteem (n = 4).

For studies using CBT as the intervention, small to large effect sizes were found (Cohen's d (d) between 0.36 and 1.25). Studies that combined CBT with other approaches reported small to large effect sizes (d = −0.17 to 0.99). CT was found to have a small effect size (d = 0.45). ACT interventions reported a medium effect size (d = 0.62 to 0.78). Mindfulness and strength-based interventions found medium to large effect sizes (d = 0.66 to 1.83). Cognitive bias modification found a large effect size (d = 0.86) and attentional bias modification a medium effect size (d = 0.57). The motion picture producing app found a large effect size (d = 0.94). For an app used as adjunct to face-to-face therapy a small to medium effect size was reported (d = 0.25 to 0.50). For music therapy and the decision aids the Cohen's d was not reported. Online interventions without human support resulted in small to large effect sizes (d = −0.09 to 0.99). Online interventions with robot support yielded small effect sizes (d = 0.37 to.44). Studies that included clinical moderation found small to large effect sizes (d = 0.33 to 1.25). Finally, online interventions with the combination of clinical and peer moderation found small to large effect sizes (d = 0.25 to 1.83).Overall, studies varied in their size, rigor of study, effectiveness and outcome measures; the effect sizes were highest for the mindfulness and strength-based intervention (1 study, n = 14; social functioning d = 1.83), CBT (n = 12 studies; d = 0.36 to 1.25), and the motion picture app (1 study, n = 22, no control; d = 0.94 depressive symptoms). Online interventions with a combination of clinical and peer moderation (n = 3 studies; d = 0.25 to 1.83) appear to result in the most stable and highest effect sizes. See Table 4 for an overview.

TABLE 4

Table 4. Number of studies and effect sizes per intervention type and form of support.

Even though the scope of this review was indicated prevention, 60% of the articles did not exclude participants who met criteria for a mental disorder. However, clinical stages 1a and 1b are not synonym with the absence of a mental disorder as assessed with the DSM/ICD (ref). But to give a complete overview, we summarize the 12 studies that excluded participants with a mental disorder below and in Table 5. These studies show varying results. Overall, no effects to large effect sizes were found (d = 0 to 1.83). Studies using CBT as the intervention found small to large effect sizes (d = 0.36 to >1.0). One study using an ACT intervention found no significant treatment effects, and one found significant effects (p = 0.027, no effect size reported). The motion picture app found a large effect size (d = 0.94). Mindfulness and strength-based interventions found medium to large effect sizes (d = 0.66 to 1.83). Online interventions without human support resulted in no effect to large effect sizes (d = 0 to 0.94). Studies that included clinical moderation found small to large effect sizes (d = 0.36 to 89). Finally, online interventions with the combination of clinical and peer moderation found medium to large effect sizes (d = 0.5 to 1.83). Overall, studies using mindfulness and strength-based interventions, a motion picture app or CBT found the highest effect sizes. The studies using ACT as the intervention type show varying results and no effect sizes are known for one study, making it difficult to draw conclusions on the effectiveness of ACT. Moreover, online interventions with the combination of clinical and peer moderation appear to result in the most stable (smaller range of effectiveness findings) and highest effect sizes.

TABLE 5

Table 5. Study characteristics for studies excluding participants with clinical levels of symptoms.

The most robust data for the effectiveness of online preventive interventions are from the following three articles due to their high number of participants (n-range of studies from 413 to 536) and their RCT design. These studies provide general support for the effectiveness of online preventive interventions for youth. (1) Radomski et al. (73) used CBT as the intervention type, and did not find significant differences between conditions but user experience was significantly more positive in the intervention group compared to the control group. (2) Sanci et al. (74) researched a decision aid, and found a significantly stronger reduction in negative affect in the intervention group compared to the control group post-intervention and at the 1-month follow-up. In addition, a significant increase in help-seeking behavior was measured in the intervention group compared to the control group at the 3-month follow-up. (3) Staples et al. (76), also used CBT as the intervention type, and found significant reduction in symptoms of depression, anxiety and non-specific psychological distress with large within-group effect size (d >1.0), high levels of treatment satisfaction and no significant differences between the online intervention group and the routine care group.

Overall, young people commonly reported high satisfaction and usability of online interventions. For example, high levels of treatment satisfaction were reported (76). Moreover, safety, reliability (53) and positive user experience (73) of online platforms were found.

Discussion

The focus of this review was to present an overview of indicated online preventive interventions for emerging mental health symptoms in youth (12–25 years). We aimed to identify the nature and extent of the relevant research evidence from treatment studies. This led to the following guiding question: What is known about the use of indicated online preventive interventions for youth with emerging mental health problems?

The findings of the included articles of the scoping review indicate the overall importance of online indicated preventive intervention. The results show that online preventive interventions are generally effective to reduce subclinical symptoms of various mental illnesses and improve several outcome measures such as quality of life and mindfulness. In addition, young people commonly reported good satisfaction, acceptability and usability of online interventions.

However, the included studies pose several limitations and therefore conclusions should be made with caution. Also the research published to date has focused predominantly on specific diagnostic categories, suggesting there is a lack of studies that have targeted transdiagnostic mechanisms. Finally, clear definitions of- as well as instruments to measure- emerging or subclinical mental health symptoms are missing. In the next section, the found gaps in the research, and the limitations of this scoping review will be discussed, as well as recommendations for future research.

Gaps in the Literature

Overall, the included articles show that online indicated preventive mental health interventions for youth with emerging mental health issues show promise in reducing various mental health complaints, and increasing positive mental health indicators such as well-being and resilience. From the 30 articles selected for our scoping, the vast majority of the included studies were RCTs with adequate use of control groups (n = 24). Nonetheless, the included studies showed important shortcomings. For example, effect sizes were often not reported, psychometric qualities of used instruments were not investigated, and control groups were missing. Moreover, it remains unclear how long these positive effects last. The majority of articles had no follow-up data exceeding 3 months, and only four articles had follow-up data exceeding 7 months. The limited availability of long-term data is an issue, since it does not provide adequate insight whether online indicated preventive interventions for youth with emerging mental health issues delays the onset of a consequent mental illness, or whether it prevents the onset altogether. To provide an answer to this issue, future research regarding online indicated preventive interventions needs to investigate the long-term effects.

We note that there is an emerging consensus among researchers of the potential importance of indicated preventive interventions for young people. However, clear definitions of subclinical mental health complaints vs. clinical mental health disorders are missing, as well as instruments to measure these different stages. For example, in a considerable number of studies participants were only asked whether they were ever formally diagnosed with a mental illness. Thus, the external validity of available indicated prevention research is limited, and the findings of the studies should be interpreted with caution. The clinical staging model of McGorry et al. (40) might offer a way to differentiate subclinical mental health complaints from clinical mental health disorders using different stages of mental health disorders. The model provides clear descriptions and cut-offs [e.g., (81)]. To the best of our knowledge a clinical instrument to operationalize these stages has not yet been developed. When looked at online indicated prevention research, it becomes clear that the clinical staging model has not been fully implemented and that there is critical fundamental work still to be undertaken. Additionally, indicated prevention in terms of the clinical staging model entails the prevention of severe mental health conditions. Light mental disorders in the affective spectrum would fall in stage 1b (40). In other words, the clinical staging model does not have the same cut offs as the DSM/ICD categories. This also shows that it is relatively difficult to apply the clinical staging model at the current moment. An interesting finding from studies in at-risk populations is that emerging mental health complaints are often diffuse and non-diagnosis specific. Also, emerging complaints have divergent trajectories, potentially leading to different mental disorders as well as remission or recovery (40). Within the included research of this scoping review, however, the focus lies almost exclusively on specific disorder categories, for example, youth with depressive symptoms. Moreover, the interventions used were disorder-oriented and less individually tailored. Since emerging complaints are often diffuse, have divergent trajectories, and are underpinned by overlapping mechanisms, a transdiagnostic approach would potentially make indicated preventive interventions more useful.

Only a small proportion of the found articles focused on youth within the age of 12–25. Most studies either looked at children younger than 18 years old, or at adults above 18 years old. This finding highlights a common obstacle in the modern day psychiatry, namely the gap that exists between child and adult psychiatry. The transition from child to adult psychiatry holds a risk for disruption in continuity of care (82, 83). Despite this, the onset of disorders (5, 6), as well as the strongest health burden (6) and multilevel life transitions (82) lie within this period. The group of youth between the ages of 12–25 years old is traditionally being divided in two groups based on age, labeled ‘child' and “adult,” while the characteristics and complaints of these individuals might suggest treating this group as a whole. More and more this need is emphasized, and currently being implemented in for example the Dutch health care system (84). The mean age of the sample of this scoping review was 18 years old, right at this cut, which also emphasizes the need to lift this boundary in scientific research and clinical practice.

To date a range of different platforms, websites and apps for online selective preventive interventions have been developed. Most studies used evidence-based therapies or frameworks for these programs. There was a great variance in the inclusion of additional human support to these interventions; ranging from sending reminders to weekly therapy sessions with a clinician. One study found no beneficial effect of inclusion of human support (76); however there is extensive research that adding human support enhances clinical effectiveness of online interventions (85, 86). The additional value of different types of human support should be investigated more extensively to be able to draw firm conclusions.

The mean adherence to the included programs varied substantially among the included studies. However, caution in interpreting this data is advised for various reasons. The biggest issue with interpreting the adherence rates in the present scoping review stems from the fact that the included papers were not using a standard method to describe the adherence to their program: therefore, there is no clear convention or determining “adherence.” For example, adherence could be described by using the program in the experimental condition “at least once (59),” or “completing at least one module (out of a total of 4 modules (57).” Consequently, rates may seem artificially high due to the unclear demarcation of “adherence.” Beintner et al. (87) found that out of a total of 216 publications that measured adherence in their analysis, 23 (10.6%) used one metric, 46 (21.3%) used two, 56 (25.9%) used three, and 63 (29.2%) included the use of four or more metrics. Indeed, it is a challenge to compare adherence rates in different studies with each other due to a missing common standard, as concluded by Beintner et al. (87). A possible solution to this methodological challenge is to introduce a-priori measurements to generate more meaningful data in future studies measuring adherence to and use of online programs, in accordance with the reasoning of Alvarez-Jiménez et al. (88). Additionally, adherence rates may be artificially inflated due to recruitment setting or participation incentive as described in the study of Mccall et al. (69) (98% adherence rate) where student participants received extra course credit contingent on the amount of modules they completed and must be taken into account when interpreting data.

The majority of studies measuring usability and acceptability (defined by how intuitive and easy a program was to use and whether the program was satisfactory and acceptable, respectively) reported acceptable levels [e.g., (57, 60, 61, 66, 68)]. However, as it was critically noted by Deady et al. (57), the largely unguided nature of online preventative interventions might negatively impact adherence. About half of the studies included in this scoping review made use of some form of human support. However, it is difficult to make inferences about the possible impact of human support on adherence since the inclusion of human support as well as the measurements of adherence varied substantially among the included studies. It is important to carefully examine the advantages of providing largely unguided interventions with low adherence vs. interventions that require more guidance but yield higher adherence rates. Moreover, it could be valuable to investigate new ways to increase adherence in unguided settings.

Limitations and Strengths

Several limitations and strengths to this review should be highlighted. First of all, this study is a scoping review as opposed to a systematic review. Although this review has an ambitious breadth, it is not meant to be exhaustive in nature. For example, only three databases were searched. However, the advantage of a scoping review is that it aims to map key concepts, main sources, types of evidence available, volume, nature and characteristics rapidly, especially in areas where less research has been conducted. In contrast to most systematic reviews, the scoping review included not only RCTs, but different methods and study designs, implicating that the literature is potentially described more broadly (50). In addition, a scoping review is descriptive in nature (50), and no quality assessment of studies has been done. Therefore, further mechanisms and quality of evidence could not be provided.

Another limitation is that studies with certain disorder categories were excluded, for example eating disorders. One could argue this disorder would justify inclusion. Moreover, even though clear inclusion and exclusion criteria were drafted a priori, it was difficult to apply these criteria with a high degree of precision. Existing studies were often not explicitly based on concepts of clinical staging. Furthermore, the majority of the studies treated youth between the ages of 12 and 25 as two separate groups, contrary to our conceptualization of treating this age group as a whole. As a result, the identified articles varied in nature, population, methods, definitions and outcome measures, making it difficult to draw conclusions about online indicated preventive interventions for youth.

A strength of this review is that we used a transparent methodological framework to find key trends in the literature, which potentially gives a preliminary basis for future systematic reviews. Further, the review identified important gaps in the existing literature. Lastly, to our knowledge, this review is the first in the past 5 years to shed light on indicated preventive mental health intervention for youth.

Future Research

Overall, high-quality investigations of the effectiveness of online indicated preventive interventions with follow-up data exceeding a few months for youth are missing. Further good quality research is needed to assess the effectiveness of the different online interventions using different therapeutic approaches. We suggest that researchers develop standardized definitions and instruments concerning subclinical symptoms in addition to clear definitions of “adherence.” Moreover, a gap in transdiagnostic approaches is evident; as well as research that has specifically targeted the adolescent population in the age range from 12 to 25 years, which crosses the divide between the child and adult mental healthcare systems. Future research should include clinical trials of indicated preventive interventions for youth between the ages of 12 and 25 based on the clinical staging model with a focus on transdiagnostic mechanisms.

Recommendations

In order for effective online interventions to be implemented in large numbers of youth with emerging mental health issues, it is the authors' opinion that representatives of this youth should be involved in the development process of the interventions and the online platform (e.g., co-creation). The platform should be adaptive and improve continuously in response to feedback, thereby enabling idiosyncratic or personalized support. The review also shows that there are many different platforms and online interventions. Uniformity could prevent reinventing the wheel and contribute to the improvement of quality over time, both of the interventions and platforms, through research and the sharing of experiences. Lastly, it is of importance to that these services are financially compensated on a structural basis, for example from governments to enable ongoing innovation and development and keep up with the fast pace of development of technology. This requires commitment of governments and participation of “offline” (in-person) care parties to improve blended online and offline care adjusted to the needs of young people at different time points during their development.

Author Contributions

MD, DN, ME, and KA completed initial study design. MD, ME, LN, KA, and ER assessed the provided articles. MD, ME, and LN made the final inclusion selection. DN, JDa, CM, AP, and TvA provided an expertise assessment and contributed extra articles to the scoping. The manuscript was written by MD, ME, and LN. All authors read and approved the final manuscript.

Funding

This study was supported by the The Netherlands Organization for Health Research and Development (ZonMW) to the ENgage YOung peoply earlY (ENYOY) study (No. 63631 0008). MA-J was supported by an Investigator Grant (APP1177235) from the National Health and Medical Research Council.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyt.2021.580843/full#supplementary-material

References

1. Organization WH. Promoting mental health: Concepts,emerging evidence, practice (Summary Report). Geneva: World Health Organization (2004).

Google Scholar

2. Kessler RC, Berglund P, Demler O, Jin R, Merikangas KR, Walters EE. Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the National Comorbidity Survey Replication. Arch Gen Psychiatry. (2005) 62:593–602. doi: 10.1001/archpsyc.62.6.593

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Angst J, Paksarian D, Cui L, Merikangas KR, Hengartner MP, Ajdacic-Gross V, et al. The epidemiology of common mental disorders from age 20 to 50: results from the prospective Zurich cohort Study. Epidemiol Psychiatr Sci. (2016) 25:24–32. doi: 10.1017/S204579601500027X

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Kessler RC, Chiu WT, Demler O, Merikangas KR, Walters EE. Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the National Comorbidity Survey Replication. Arch Gen Psychiatry. (2005) 62:617–27. doi: 10.1001/archpsyc.62.6.617

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Jones PB. Adult mental health disorders and their age at onset. Br J Psychiatry Suppl. (2013) 54:s5–10. doi: 10.1192/bjp.bp.112.119164

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Kessler RC, Amminger GP, Aguilar-Gaxiola S, Alonso J, Lee S, Ustun TB. Age of onset of mental disorders: a review of recent literature. Curr Opin Psychiatry. (2007) 20:359–64. doi: 10.1097/YCO.0b013e32816ebc8c

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Magnée T, de Beurs D, Verhaak P. Wachttijden voor de poh-ggz. Huisarts en wetenschap. (2017) 60:205. doi: 10.1007/s12445-017-0131-7

CrossRef Full Text | Google Scholar

8. Vigo D, Thornicroft G, Atun R. Estimating the true global burden of mental illness. Lancet Psychiatry. (2016) 3:171–8. doi: 10.1016/s2215-0366(15)00505-2

PubMed Abstract | CrossRef Full Text | Google Scholar

9. The-carter-center. report: mental illness will cost the world $16 trillion (USD) by 2030. Mental Health Weekly. (2018) 28:7–8. doi: 10.1002/mhw.31630

CrossRef Full Text

10. Rüsch N, Corrigan P, Todd A, Bodenhausen G. Implicit self-stigma in people with mental illness. J Nerv Ment Dis. (2010) 198:150–3. doi: 10.1097/NMD.0b013e3181cc43b5

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Mathers C. Global burden of disease. Int Encyclopedia Public Health. (2017) 3:256–67. doi: 10.1016/b978-0-12-803678-5.00175-2

CrossRef Full Text

12. Corrigan P. How stigma interferes with mental health care. Am Psychol. (2004) 59:614–25. doi: 10.1037/0003-066X.59.7.614

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Muntaner C, Ng E, Vanroelen C, Christ S, Eaton WW. Social stratification, social closure and social class as determinants of mental health disparities. In: Aneshensel CS, Phelan JC, Bierman A, editors. Handbook of the Sociology of Mental Health. 2 ed. Dordrecht: Springer (2013). p. 205–28.

Google Scholar

14. Iorfino F, Scott EM, Carpenter JS, Cross SP, Hermens DF, Killedar M, et al. Clinical stage transitions in persons aged 12 to 25 years presenting to early intervention mental health services with anxiety, mood, and psychotic disorders. JAMA Psychiatry. (2019) 76:1167–75. doi: 10.1001/jamapsychiatry.2019.2360

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Fox S, Southwell A, Stafford N, Goodhue R, Jackson D, Smith C. Better Systems, Better Chances: A Review of Research and Practice for Prevention and Early Intervention. Canberra, ACT: Australian Research Alliance for Children and Youth (ARACY) (2015). Available online at: https://www.aracy.org.au/publications-resources/command/download_file/id/274/filename/Better-systems-better-chances.pdf

16. Munoz RF, Cuijpers P, Smit F, Barrera AZ, Leykin Y. Prevention of major depression. Ann Rev Clin Psychol. (2010) 6:181–212. doi: 10.1146/annurev-clinpsy-033109-132040

CrossRef Full Text | Google Scholar

17. Weare K, Nind M. Mental health promotion and problem prevention in schools: what does the evidence say? Health Promot Int. (2011) 26(Suppl 1):i29–69. doi: 10.1093/heapro/dar075

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Tennant R, Goens C, Barlow J, Day C, Stewart-Brown S. A systematic review of reviews of interventions to promote mental health and prevent mental health problems in children and young people. J Public Ment Health. (2007) 6:25–32. doi: 10.1108/17465729200700005

CrossRef Full Text | Google Scholar

19. Gulliver A, Griffiths KM, Christensen H. Perceived barriers and facilitators to mental health help-seeking in young people: a systematic review. BMC Psychiatry. (2010) 10:113–42. doi: 10.1186/1471-244X-10-113

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Green H, McGinnity Á, Meltzer H, Ford T, Goodman R. Mental Health of Children and Young People in Great Britain, 2004. Basingstoke; New York, NY: Palgrave Macmillan Basingstoke (2005).

Google Scholar

21. Bandelow B, Schuller K. Mean age and gender distribution of patients with major mental disorders participating in clinical trials. Eur Arch Psychiatry Clin Neurosci. (2019) 270:655–9. doi: 10.1007/s00406-018-0972-5

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Seidler Z, Rice S, Ogrodniczuk J, Oliffe J, Shaw J, Dhillon H. Men, Masculinities, depression: implications for mental health services from a delphi expert consensus study. Prof Psychol Res Pract. (2019) 50:51–61. doi: 10.1037/pro0000220

CrossRef Full Text | Google Scholar

23. Stagnaro JC, Cia AH, Vommaro H, Sustas S, Vazquez N, Serfaty E, et al. Delays in making initial treatment contact after the first onset of mental health disorders in the Argentinean Study of Mental Health Epidemiology. Epidemiol Psychiatr Sci. (2019) 28:240–50. doi: 10.1017/S2045796018000094

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Clarke K. Childhood, parenting and early intervention: a critical examination of the Sure Start national programme. Crit Soc Policy. (2006) 26:699–721. doi: 10.1177/0261018306068470

CrossRef Full Text | Google Scholar

25. Calear AL, Batterham PJ, Torok M, McCallum S. Help-seeking attitudes and intentions for generalised anxiety disorder in adolescents: the role of anxiety literacy and stigma. Eur Child Adolesc Psychiatry. (2020) 30:243–51. doi: 10.1007/s00787-020-01512-9

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Nearchou FA, Bird N, Costello A, Duggan S, Gilroy J, Long R, et al. Personal and perceived public mental-health stigma as predictors of help-seeking intentions in adolescents. J Adolesc. (2018) 66:83–90. doi: 10.1016/j.adolescence.2018.05.003

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Andersson G, Titov N. Advantages and limitations of Internet-based interventions for common mental disorders. World Psychiatry. (2014) 13:4–11. doi: 10.1002/wps.20083

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Andrews G, Cuijpers P, Craske MG, McEvoy P, Titov N. Computer therapy for the anxiety and depressive disorders is effective, acceptable and practical health care: a meta-analysis. PLoS One. (2010) 5:e13196. doi: 10.1371/journal.pone.0013196

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Bradley KL, Robinson LM, Brannen CL. Adolescent help-seeking for psychological distress, depression, and anxiety using an Internet program. Int J Ment Health Promot. (2012) 14:23–34. doi: 10.1080/14623730.2012.665337

CrossRef Full Text | Google Scholar

30. Barak A, Grohol JM. Current and future trends in Internet-supported mental health interventions. J Technol Hum Serv. (2011) 29:155–96. doi: 10.1080/15228835.2011.616939

CrossRef Full Text | Google Scholar

31. European-Commission. Internet Access and Use Statistics—Households and Individuals. European-Commission (2016).

Google Scholar

32. Burns JM, Davenport T, Durkin LA, Luscombe GM. and Hickie IB. The internet as a setting for mental health service utilisation by young people. Med J Aust. (2010) 192:S22–S6. doi: 10.5694/j.1326-5377.2010.tb03688.x

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Hanley T, Prescott J, Gomez KU. A systematic review exploring how young people use online forums for support around mental health issues. J Ment Health. (2019) 28:1–11. doi: 10.1080/09638237.2019.1630725

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Cliffe B, Croker A, Denne M, Stallard P. Clinicians' use of and attitudes towards technology to provide and support interventions in child and adolescent mental health services. Child Adolesc Ment Health. (2020) 25:95–101. doi: 10.1111/camh.12362

PubMed Abstract | CrossRef Full Text | Google Scholar

35. The-Lancet-Psychiatry. Mental health and COVID-19: change the conversation. Lancet Psychiatry. (2020) 7:463. doi: 10.1016/S2215-0366(20)30194-2

CrossRef Full Text | Google Scholar

36. Wind TR, Rijkeboer M, Andersson G, Riper H. The COVID-19 pandemic: the 'black swan' for mental health care and a turning point for e-health. Internet Interv. (2020) 20:100317. doi: 10.1016/j.invent.2020.100317

PubMed Abstract | CrossRef Full Text | Google Scholar

37. DeLuca JS, Andorko ND, Chibani D, Jay SY, Rakhshan Rouhakhtar PJ, Petti E, et al. Telepsychotherapy with youth at clinical high risk for psychosis: clinical issues and best practices during the COVID-19 pandemic. J Psychother Integration. (2020) 30:304–31. doi: 10.1037/int0000211

CrossRef Full Text | Google Scholar

38. Horowitz JL, Garber J. The prevention of depressive symptoms in children and adolescents: A meta-analytic review. J Consult Clin Psychol. (2006) 74:401–15. doi: 10.1037/0022-006X.74.3.401

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Hickie IB, Scott EM, Hermens DF, Naismith SL, Guastella AJ, Kaur M, et al. Applying clinical staging to young people who present for mental health care. Early Interv Psychiatry. (2013) 7:31–43. doi: 10.1111/j.1751-7893.2012.00366.x

PubMed Abstract | CrossRef Full Text | Google Scholar

40. McGorry PD, Hickie IB, Yung AR, Pantelis C, Jackson HJ. Clinical staging of psychiatric disorders: a heuristic framework for choosing earlier, safer and more effective interventions. Aust N Z J Psychiatry. (2006) 40:616–22. doi: 10.1080/j.1440-1614.2006.01860.x

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Stice E, Shaw H, Bohon C, Marti CN, Rohde P. A meta-analytic review of depression prevention programs for children and adolescents: factors that predict magnitude of intervention effects. J Consult Clin Psychol. (2009) 77:486–503. doi: 10.1037/a0015168

PubMed Abstract | CrossRef Full Text | Google Scholar

42. McGorry PD. The specialist youth mental health model: strengthening the weakest link in the public mental health system. Med J Aust. (2007) 187(S7):S53–6. doi: 10.5694/j.1326-5377.2007.tb01338.x

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Schoemaker C, Kleinjan M, van der Borg W, Busch M, Muntinga M, Nuijen J, et al. Mentale gezondheid van jongeren: Enkele cijfers en ervaringen (Youth mental health: numbers and experiences). Amsterdam UMC: RIVM, Trimbos-instituut (2019). p. 1–24.

Google Scholar

44. Rice SM, Goodall J, Hetrick SE, Parker AG, Gilbertson T, Amminger GP, et al. Online and social networking interventions for the treatment of depression in young people: a systematic review. J Med Internet Res. (2014) 16:e206. doi: 10.2196/jmir.3304

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Ali K, Farrer L, Gulliver A, Griffiths KM. Online peer-to-peer support for young people with mental health problems: a systematic review. JMIR Ment Health. (2015) 2:e19. doi: 10.2196/mental.4418

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Pennant ME, Loucas CE, Whittington C, Creswell C, Fonagy P, Fuggle P, et al. Computerised therapies for anxiety and depression in children and young people: a systematic review and meta-analysis. Behav Res Ther. (2015) 67:1–18. doi: 10.1016/j.brat.2015.01.009

PubMed Abstract | CrossRef Full Text | Google Scholar

47. O'Dea B, Calear AL, Perry Y. Is e-health the answer to gaps in adolescent mental health service provision? Curr Opin Psychiatry. (2015) 28:336–42. doi: 10.1097/yco.0000000000000170

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Ebert DD, Zarski AC, Christensen H, Stikkelbroek Y, Cuijpers P, Berking M, et al. Internet and computer-based cognitive behavioral therapy for anxiety and depression in youth: a meta-analysis of randomized controlled outcome trials. PLoS ONE. (2015) 10:e0119895. doi: 10.1371/journal.pone.0119895

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Conley CS, Durlak JA, Shapiro JB, Kirsch AC, Zahniser E. A meta-analysis of the impact of universal and indicated preventive technology-delivered interventions for higher education students. Prev Sci. (2016) 17:659–78. doi: 10.1007/s11121-016-0662-3

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Arksey H, O'Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. (2005) 8:19–32. doi: 10.1080/1364557032000119616

CrossRef Full Text | Google Scholar

51. Alvarez-Jimenez M, Gleeson JF, Bendall S, Penn DL, Yung AR, Ryan RM, et al. Enhancing social functioning in young people at Ultra High Risk (UHR) for psychosis: a pilot study of a novel strengths and mindfulness-based online social therapy. Schizophr Res. (2018) 202:369–77. doi: 10.1016/j.schres.2018.07.022

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Alvarez-Jimenez M, Rice S, D'Alfonso S, Leicester S, Bendall S, Pryor I, et al. A novel Multi-modal Online Service (MOST+) for help-seeking young people experiencing mental ill-health: a pilot evaluation within a national youth e-mental health service. J Med Internet Res. (2020) 22:e17155. doi: 10.2196/17155

CrossRef Full Text | Google Scholar

53. Anttila K, Anttila M, Valimaki M. A web-based adolescent depression support system: feedback and implications for the future. Inform Health Soc Care. (2020) 45:111–29. doi: 10.1080/17538157.2019.1656206

PubMed Abstract | CrossRef Full Text | Google Scholar

54. van Aubel E, Bakker JM, Batink T, Michielse S, Goossens L, Lange I, et al. Blended care in the treatment of subthreshold symptoms of depression and psychosis in emerging adults: a randomised controlled trial of Acceptance and Commitment Therapy in Daily-Life (ACT-DL). Behav Res Ther. (2020) 128:103592. doi: 10.1016/j.brat.2020.103592

PubMed Abstract | CrossRef Full Text | Google Scholar

55. Berg M, Rozental A, Johansson S, Liljethorn L, Radvogin E, Topooco N, et al. The role of knowledge in internet-based cognitive behavioural therapy for adolescent depression: results from a randomised controlled study. Internet Interv. (2019) 15:10–7. doi: 10.1016/j.invent.2018.10.001

PubMed Abstract | CrossRef Full Text | Google Scholar

56. Cook L, Mostazir M, Watkins E. Reducing Stress and Preventing Depression (RESPOND): randomized controlled trial of web-based rumination-focused cognitive behavioral therapy for high-ruminating university students. J Med Internet Res. (2019) 21:e11349. doi: 10.2196/11349

PubMed Abstract | CrossRef Full Text | Google Scholar

57. Deady M, Mills KL, Teesson M, Kay-Lambkin F. An online intervention for co-occurring depression and problematic alcohol use in young people: primary outcomes from a randomized controlled trial. J Med Internet Res. (2016) 18:e71. doi: 10.2196/jmir.5178

PubMed Abstract | CrossRef Full Text | Google Scholar

58. Dickter B, Bunge EL, Brown LM, Leykin Y, Soares EE, Van Voorhees B, et al. Impact of an online depression prevention intervention on suicide risk factors for adolescents and young adults. Mhealth. (2019) 5:11. doi: 10.21037/mhealth.2019.04.01

PubMed Abstract | CrossRef Full Text | Google Scholar

59. Farrer LM, Gulliver A, Katruss N, Fassnacht DB, Kyrios M, Batterham PJ. A novel multi-component online intervention to improve the mental health of University students: randomised controlled trial of the Uni Virtual Clinic. Internet Interv. (2019) 18:100276. doi: 10.1016/j.invent.2019.100276

PubMed Abstract | CrossRef Full Text | Google Scholar

60. Fitzpatrick KK, Darcy A, Vierhile M. Delivering Cognitive Behavior Therapy to Young Adults With Symptoms of Depression and Anxiety Using a Fully Automated Conversational Agent (Woebot): A Randomized Controlled Trial. JMIR Ment Health. (2017) 4:e19. doi: 10.2196/mental.7785

PubMed Abstract | CrossRef Full Text | Google Scholar

61. Harrer M, Adam SH, Fleischmann RJ, Baumeister H, Auerbach R, Bruffaerts R, et al. Effectiveness of an internet- and app-based intervention for college students with elevated stress: randomized controlled trial. J Med Internet Res. (2018) 20:e136. doi: 10.2196/jmir.9293

PubMed Abstract | CrossRef Full Text | Google Scholar

62. Hickie IB, Davenport TA, Burns JM, Milton AC, Ospina-Pinillos L, Whittle L, et al. Project Synergy: co-designing technology-enabled solutions for Australian mental health services reform. Med J Aust. (2019) 211(Suppl 7):S3–S39. doi: 10.5694/mja2.50349

PubMed Abstract | CrossRef Full Text | Google Scholar

63. Hides L, Dingle G, Quinn C, Stoyanov SR, Zelenko O, Tjondronegoro D, et al. Efficacy and Outcomes of a Music-Based Emotion Regulation Mobile App in Distressed Young People: Randomized Controlled Trial. JMIR Mhealth Uhealth. (2019) 7:e11482. doi: 10.2196/11482

PubMed Abstract | CrossRef Full Text | Google Scholar

64. Hill RM, Pettit JW. Pilot randomized controlled trial of LEAP: a selective preventive intervention to reduce adolescents' perceived burdensomeness. J Clin Child Adolesc Psychol. (2019) 48(sup1):S45–S56. doi: 10.1080/15374416.2016.1188705

PubMed Abstract | CrossRef Full Text | Google Scholar

65. de Hullu E, Sportel BE, Nauta MH, de Jong PJ. Cognitive bias modification and CBT as early interventions for adolescent social and test anxiety: two-year follow-up of a randomized controlled trial. J Behav Ther Exp Psychiatry. (2017) 55:81–9. doi: 10.1016/j.jbtep.2016.11.011

PubMed Abstract | CrossRef Full Text | Google Scholar

66. Lattie EG, Ho J, Sargent E, Tomasino KN, Smith JD, Brown CH, et al. Teens engaged in collaborative health: the feasibility and acceptability of an online skill-building intervention for adolescents at risk for depression. Internet Interv. (2017) 8:15–26. doi: 10.1016/j.invent.2017.02.003

PubMed Abstract | CrossRef Full Text | Google Scholar

67. Levin ME, Haeger JA, Pierce BG, Twohig MP. Web-Based Acceptance and Commitment Therapy for Mental Health Problems in College Students: a randomized controlled trial. Behav Modif. (2017) 41:141–62. doi: 10.1177/0145445516659645

PubMed Abstract | CrossRef Full Text | Google Scholar

68. Levin ME, Hayes SC, Pistorello J, Seeley JR. Web-based self-help for preventing mental health problems in universities: comparing acceptance and commitment training to mental health education. J Clin Psychol. (2016) 72:207–25. doi: 10.1002/jclp.22254

PubMed Abstract | CrossRef Full Text | Google Scholar

69. McCall HC, Richardson CG, Helgadottir FD, Chen FS. Evaluating a web-based social anxiety intervention among University students: randomized controlled trial. J Med Internet Res. (2018) 20:e91. doi: 10.2196/jmir.8630

PubMed Abstract | CrossRef Full Text | Google Scholar

70. McDermott R, Dozois DJA. A randomized controlled trial of Internet-delivered CBT and attention bias modification for early intervention of depression. J Exp Psychopathol. (2019) 10:2043808719842502. doi: 10.1177/2043808719842502

CrossRef Full Text | Google Scholar

71. Motter JN, Grinberg A, Lieberman DH, Iqnaibi WB, Sneed JR. Computerized cognitive training in young adults with depressive symptoms: effects on mood, cognition, and everyday functioning. J Affect Disord. (2019) 245:28–37. doi: 10.1016/j.jad.2018.10.109

PubMed Abstract | CrossRef Full Text | Google Scholar

72. Poppelaars M, Tak YR, Lichtwarck-Aschoff A, Engels RCME, Lobel A, Merry SN, et al. A randomized controlled trial comparing two cognitive-behavioral programs for adolescent girls with subclinical depression: A school-based program (Op Volle Kracht) and a computerized program (SPARX). Behav Res Ther. (2016) 80:33–42. doi: 10.1016/j.brat.2016.03.005

PubMed Abstract | CrossRef Full Text | Google Scholar

73. Radomski AD, Bagnell A, Curtis S, Hartling L, Newton AS. Examining the Usage, User Experience, and Perceived Impact of an Internet-Based Cognitive Behavioral Therapy Program for Adolescents With Anxiety: randomized controlled trial. JMIR Ment Health. (2020) 7:e15795. doi: 10.2196/15795

PubMed Abstract | CrossRef Full Text | Google Scholar

74. Sanci L, Kauer S, Thuraisingam S, Davidson S, Duncan AM, Chondros P, et al. Effectiveness of a mental health service navigation website (link) for young adults: randomized controlled trial. JMIR Ment Health. (2019) 6:e13189. doi: 10.2196/13189

PubMed Abstract | CrossRef Full Text | Google Scholar

75. Simmons MB, Elmes A, McKenzie JE, Trevena L, Hetrick SE. Right choice, right time: Evaluation of an online decision aid for youth depression. Health Expect. (2017) 20:714–23. doi: 10.1111/hex.12510

PubMed Abstract | CrossRef Full Text | Google Scholar

76. Staples LG, Dear BF, Johnson B, Fogliati V, Gandy M, Fogliati R, et al. Internet-delivered treatment for young adults with anxiety and depression: evaluation in routine clinical care and comparison with research trial outcomes. J Affect Disord. (2019) 256:103–9. doi: 10.1016/j.jad.2019.05.058

PubMed Abstract | CrossRef Full Text | Google Scholar

77. Takahashi K, Takada K, Hirao K. Feasibility and preliminary efficacy of a smartphone application intervention for subthreshold depression. Early Interv Psychiatry. (2019) 13:133–6. doi: 10.1111/eip.12540

PubMed Abstract | CrossRef Full Text | Google Scholar

78. Topper M, Emmelkamp PMG, Watkins E, Ehring T. Prevention of anxiety disorders and depression by targeting excessive worry and rumination in adolescents and young adults: a randomized controlled trial. Behav Res Ther. (2017) 90:123–36. doi: 10.1016/j.brat.2016.12.015

PubMed Abstract | CrossRef Full Text | Google Scholar

79. Traber-Walker N, Gerstenberg M, Metzler S, Joris MR, Karr M, Studer N, et al. Evaluation of the combined treatment approach “robin” (standardized manual and smartphone app) for adolescents at clinical high risk for psychosis. Front Psychiatry. (2019) 10:384. doi: 10.3389/fpsyt.2019.00384

PubMed Abstract | CrossRef Full Text | Google Scholar

80. De Voogd EL, Wiers RW, Salemink E. Online visual search attentional bias modification for adolescents with heightened anxiety and depressive symptoms: a randomized controlled trial. Behav Res Ther. (2017) 92:57–67. doi: 10.1016/j.brat.2017.02.006

PubMed Abstract | CrossRef Full Text | Google Scholar

81. Shah JL, Scott J, McGorry PD, Cross SPM, Keshavan MS, Nelson B, et al. Transdiagnostic clinical staging in youth mental health: a first international consensus statement. World Psychiatry. (2020) 19:233–42. doi: 10.1002/wps.20745

PubMed Abstract | CrossRef Full Text | Google Scholar

82. Lindgren E, Soderberg S, Skar L. The gap in transition between child and adolescent psychiatry and general adult psychiatry. J Child Adolesc Psychiatr Nurs. (2013) 26:103–9. doi: 10.1111/jcap.12027

PubMed Abstract | CrossRef Full Text | Google Scholar

83. Signorini G, Singh SP, Marsanic VB, Dieleman G, Dodig-Curkovic K, Franic T, et al. Correction to: The interface between child/adolescent and adult mental health services: results from a European 28country survey. Eur Child Adolesc Psychiatry. (2018) 27:1511–2. doi: 10.1007/s00787-018-1132-1

PubMed Abstract | CrossRef Full Text | Google Scholar

84. Leijdesdorff S, Postma MR, van Kersbergen L, Marchetta N, van Amelsvoort T. No boundaries: a 2 year experience in a specialized youth mental health care program in the Netherlands. Early Interv Psychiatry. (2020) 14:228–34. doi: 10.1111/eip.12887

PubMed Abstract | CrossRef Full Text | Google Scholar

85. Mohr DC, Cuijpers P, Lehman K. Supportive accountability: a model for providing human support to enhance adherence to eHealth interventions. J Med Internet Res. (2011) 13:e30. doi: 10.2196/jmir.1602

PubMed Abstract | CrossRef Full Text | Google Scholar

86. Tremain HMC, Fletcher K & Murray G. The therapeutic alliance in digital mental health interventions for serious mental illnesses: Narrative review. JMIR Mental Health. 2019. doi: 10.2196/preprints.17204

PubMed Abstract | CrossRef Full Text | Google Scholar

87. Beintner I, Vollert B, Zarski AC, Bolinski F, Musiat P, Gorlich D, et al. Adherence Reporting in Randomized Controlled Trials Examining Manualized Multisession Online Interventions: Systematic Review of Practices and Proposal for Reporting Standards. J Med Internet Res. (2019) 21:e14181. doi: 10.2196/14181

PubMed Abstract | CrossRef Full Text | Google Scholar

88. Alvarez-Jimenez M, Alcazar-Corcoles MA, Gonzalez-Blanch C, Bendall S, McGorry PD, Gleeson JF. Online, social media and mobile technologies for psychosis treatment: a systematic review on novel user-led interventions. Schizophr Res. (2014) 156:96–106. doi: 10.1016/j.schres.2014.03.021

PubMed Abstract | CrossRef Full Text | Google Scholar