Mobile applications in adolescent psychotherapy during the COVID-19 pandemic: a systematic review

Background: To bridge the gap in adolescent psychotherapy created by the increasing need for mental health interventions and the limited possibilities of in-person treatment during the pandemic, many health care providers opted to offer online mental health care programs. As a result, the number of mental health apps available in app stores experienced a sharp increase during the COVID-19 pandemic.

Objective: The aim of the current review is to provide an overview of feasibility and effectiveness studies testing mobile applications in adolescent psychotherapy during the peak phase of the COVID-19 pandemic.

Methods: We conducted a literature search in Pubmed, PsychInfo, Google Scholar, OpenSIGLE and OpenGREY for papers published from June 2020 to June 2023. Studies were included if they evaluated app-based interventions intended for psychotherapeutic treatment and targeted adolescents between 12 and 27 years of age with symptoms of psychological disorders. The quality of each study was assessed using the Systematic Assessment of Quality in Observational Research (SAQOR). Effectiveness outcomes were analyzed by vote counting and calculating a binomial probability test.

Results: The search yielded 31 relevant studies that examined 27 different apps with a total of 1,578 adolescent participants. Nine articles were primary effectiveness studies and 22 focused on feasibility measures as primary outcome. There was evidence that mental health apps influenced adolescents' psychotherapy, with 83% of the studies with effectiveness outcomes favoring the intervention (p = 0.002). Sixty-one percent of the included studies were rated at low or very low quality.

Conclusions: The pandemic has given apps a firm and important role in healthcare that will probably continue to expand in the future. To ensure that mental health apps are truly effective and beneficial for adolescents' psychotherapy, we need a standardized measurement of quality features of mental health apps and higher quality app evaluation studies.

Systematic review registration: https://www.crd.york.ac.uk/PROSPERO/display_record.php?RecordID=406455, PROSPERO International Prospective Register of Systematic Reviews [CRD42023406455].

1 Introduction

Due to the Coronavirus Disease 2019 (COVID-19) pandemic mental health problems increased dramatically, especially among adolescents (1–7). Additionally, our health care system experienced rapid digitalization. Many organizations developed mobile applications to maintain their treatment offers under the conditions of social distancing (8, 9). This resulted in a sharp increase of available mental Health apps (10, 11) as well as published evaluation studies (9). Most of the published studies and reviews of evaluated mental health apps focus on the adult population. Research on app-based interventions specifically for adolescents is still scarce. Nevertheless, Ellis et al. (9) reported that children and adolescents were identified as one of the most frequently targeted specific populations in published app evaluation studies during the pandemic. Comparing the number of mental health apps available in app stores and published app evaluation studies, a high discrepancy can be found (9–11). The majority of available mental health apps failed to demonstrate their effectiveness (12, 13). However, evaluation studies are an important quality feature. Without evaluation studies it is difficult to determine whether mental health apps are truly beneficial or potentially harmful for the mental health of users (14, 15). Previous research found that several non-evaluated mental health apps provided incorrect psychoeducation information, inappropriate treatment strategies or wrong contact details of emergency services (16, 17). Given the importance of evaluated mental health apps, as well as the greater focus on app-based psychotherapeutic treatment options for adolescents, there is an urgent need for an updated review of evaluation studies of mental health apps in the context of adolescent psychotherapy during the COVID-19 pandemic.

1.1 Challenges for psychotherapeutic care during the COVID-19 pandemic

In Germany, Ravens-Sieberer et al. (18) reported an increase of overall mental health problems in adolescents from 18 to 28%. The most common disorders were anxiety disorders and depression, which is comparable to data before the pandemic (18). Studies from other countries found an increase in self-harm, suicidal ideation and attempted suicide (5, 19). Madigan et al. (5) showed in their review that emergency department visits due to self-harm, suicidal ideation or suicidal attempts increased in the beginning of the pandemic despite a reduction in total emergency department visits for mental-illness concerns. In times of social distancing and school closures, adolescents lost important resources for their wellbeing, resulting in negative consequences for their mental health (7, 20, 21). Additionally, most of the treatment services were curtailed or completely discontinued during the phases of social distancing (7). As a result, the youth was exposed to greater stressors during the pandemic, while less support was available. In Germany, we can see the consequences of this in a significant increase of emergency admissions since 2021, especially in child and adolescent psychiatry (22). To address the divergence between the increasing need for mental health treatments and the discontinuation of treatment offers in times of the pandemic, health care providers tried to find new ways to reach adolescents, including offering online mental health care programs (8, 9).

1.2 Chances of mental health apps in adolescent psychotherapy

The use of mental health apps with adolescents is promising. Digital media are an integral part of adolescent everyday lives. In Germany, 96% of 12–19-year-olds own a smartphone and use it daily (23). In 2022, adolescents spent on average 204 min per day on the internet. During the pandemic the online usage times were significantly elevated, averaging 244–258 min per day (23). Furthermore, younger people show greater affinity for online mental health care. They are more likely to use the internet to gather information about their mental health than older people (24, 25). Rauschenberg et al. (26) pointed out that a large proportion of young people with psychological distress and pandemic related anxiety would like to use mobile applications to overcome negative consequences of the COVID-19 pandemic. In addition to permanent availability, mental health apps have further advantages, such as allowing adolescents to have more autonomy. They can use apps flexibly and decide when and where to get involved with the app without having to go to a fixed treatment appointment as one would in case of face-to-face therapy (27, 28). Accessibility is one of the most important arguments for using mental health interventions when it comes to adolescents. Therefore, smartphone-based interventions are more attractive to them than interventions where a laptop or other digital device is needed (27). Furthermore, apps can offer immediate support in critical situations and crises, like acute cases of suicidal ideation or self-harm (29). Due to increased affordability of mental health care through apps, it is possible to reach a higher number of help-seeking adolescents. Access to mental health apps is given independent of the available health care infrastructure or severity of symptoms. As such, persons with low to moderate symptoms of mental health conditions can be treated to prevent the development of more severe symptomatology (30). In addition, adolescents perceive the use of mental health apps as less conspicuous and bulky, which may lead to increased adherence to psychotherapy. Feeling of connectedness is also an important factor for adolescents and a further advantage of mental health apps. Mental health apps can offer an opportunity to share own experiences with peers in an appropriate manner and mitigate the fear of stigmatization (27, 28). Finally, considering previous effectiveness research on mental health apps, several studies show comparable efficacy and cost-effectiveness between smartphone-based interventions and face-to-face therapy (26, 31–33).

1.3 Areas of application of mental health apps

Just as versatile as the reasons for app usage are their areas of application. Apps can be used as stand-alone or therapy-accompanying treatments. As a stand-alone treatment, apps offer interventions for self-help. For example, they can be used during waiting periods for psychotherapy or as early interventions to prevent the development of severe symptomatology (29, 30, 34–36). Therapy-accompanying apps are used as adjunction to psychotherapy (29). These apps can support adolescents between outpatient sessions. They can increase adherence to therapeutic homework, support application of skills acquired in therapy to everyday life or offer management plans for acute crises (34, 35). Most of the available apps focus on specific disorders rather than a transdiagnostic therapy approach covering the eight common disorders: psychosis, eating disorders, depression, autism, self-harm, anxiety, substance abuse, and suicidal behavior (35). Lui et al. (34) reported in their review of evidence-based mobile applications in a psychotherapy context that none of the 21 included apps focused on symptoms that may be transdiagnostic. Four years later, Ellis et al. (9) reported that during the COVID-19 pandemic the transdiagnostic approach increased in mental health app literature: they found that 38% of the included studies were about COVID-19-related transdiagnostic symptoms like stress, loneliness or general wellbeing.

1.4 Evaluation of mental health apps

As mentioned above, there is a large discrepancy between available mental health apps and published evaluation studies of mental health apps. In the first quarter 2021, 53,979 mental health apps were available in Apple App Store and 53,054 in Google Play Store (10, 11). In contrast, Ellis et al. (9) reported in their review that between January 2020 and March 2021 356 app evaluation articles were published, with 63% of these being non-empirical publication types like commentaries or opinions. In another review, Alyami et al. (12) pointed out that none of the 1,154 identified social anxiety apps for adults had published studies of their effectiveness. Three years later, Qu et al. (13) presented in their review that of 482 investigated depression apps for adults only seven percent had a sound evidence base. Other reviews of evidence-based mental health apps for adults highlighted an insufficient scientific evaluation of app-based interventions and a lack of standardized methods for assessing effectiveness of mental health apps (15, 33, 34, 37). One reason for the low rates of effectiveness studies is attributed to the high costs involved. Effectiveness studies require a great deal of effort and usually result in long study periods, which does not meet the requirements of the fast-moving app market (14, 38). Another reason for the high rates of non-evaluated mental health apps is that providers of health care apps are not required to provide information in the app stores about the effectiveness of their digital therapeutic tools (39, 40). Most providers still do not hesitate to claim effectiveness of their applications by means of non-empirical scientific explanations, field reports or technical expertise. If an evaluation of a mental health app is available, it is mostly an evaluation of feasibility (41). Feasibility is an important aspect of an overall assessment of interventions, but it does not provide information about the usefulness or effectiveness of mental health apps. To give a comprehensive overview it is important to include various kinds of evaluation studies, from feasibility to effectiveness.

1.5 Objectives

The aim of the current review is to provide an overview of feasibility and effectiveness studies testing mobile applications in adolescent psychotherapy during the COVID-19 pandemic. Furthermore, we investigate how effectiveness of mobile applications is measured. Additionally, we examine whether effects of mobile applications differ according to specific psychological disorders as well as between stand-alone psychotherapy apps and therapy-accompanying apps.

2 Methods

A protocol for reviewing the literature was developed using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines (42). The review was registered on PROSPERO (CRD42023406455).

2.1 Search strategy

A literature search was conducted in Pubmed, PsychInfo and Google Scholar for papers published from June 2020 to June 2023. Search parameters consisted of numerous combinations of keywords related to adolescents, apps and psychotherapy and included “adolescent^*,” “youth,” “young,” “app,” “mobile,” “smartphone,” “mental health,” “digital,” “psychotherapy,” “disorder,” “psychological,” “psychiatry,” “treatment,” “therapy,” and “intervention.” For eligible gray literature, we searched OpenSIGLE and OpenGREY. Furthermore, authors of study protocols were contacted to check for recently published studies or preliminary study results. References of reviews, meta-analyses, review protocols and included studies were scanned to identify any potentially relevant literature.

2.2 Eligibility criteria

Studies were included if they evaluated app-based interventions intended for psychotherapeutic treatment and targeted adolescents between 12 and 27 years of age with symptoms of psychological disorders. Studies addressing smoking were excluded because smoking is not a clinically relevant and psychiatrically or psychotherapeutically treated addictive disease. In addition, studies with only a subset of eligible participants were excluded if it was not possible to consider the subsample separately. Studies exclusively examining adults 18 years of age or older were likewise excluded. We included any mobile app-based intervention in a psychotherapy context for adolescents. The app had to be used as a supplement to or replacement of psychotherapy. It could focus on specific psychological disorders or transdiagnostic treatment. Solely psychoeducational or diagnostic mobile applications were excluded. We included all published, unpublished, or ongoing experimental and quasi-experimental trials in English and German that compared mobile applications in a psychotherapy context with usual psychotherapy or non-psychological mobile applications (e.g., gaming applications); non-experimental studies with repeated measurements design that included at least pre- and post-measurement; and non-experimental studies based on qualitative research methods. Trials described in Editorials, Comments or Letters to the editor were excluded. Due to the COVID-19 pandemic, only studies published from June 2020 to June 2023 were included.

2.3 Study selection process

The search yielded 31 studies fulfilling all inclusion criteria (see Figure 1 for the number of papers included at each stage of the review). Four reviewers were involved in the study selection process and applied eligibility criteria for sample identification. Studies were identified in two steps. First, one reviewer screened titles and abstracts of each of the chosen databases for eligibility. Articles that did not meet the inclusion criteria were excluded. Second, two reviewers screened the full texts of potentially eligible articles. If the two reviewers' assessment of an article was discordant, the disagreement was discussed until a consensus was reached, involving a third party if necessary. The data collection and selection process was managed using the free software rayyan.ai (43) and specifically developed Excel spreadsheets for documentation. We contacted authors of study protocols or studies with samples that were not completely within the age range with a maximum of three email attempts to ask for data provision of sub-samples. Included studies were transferred to a table that presented all key information of the studies (data items): bibliographical data (e.g., authors, contact details of the corresponding author, publication year), app information (name of the app, short description of the app), sample characteristics, trial methods (e.g., study design, type of comparison group), evaluation methods, and outcome data. Finally, two reviewers assessed the quality of each study and their ratings were compared and discussed.

Figure 1

Figure 1. PRISMA flow diagram of the study selection process. ¹Sample outside the age range, non-clinical sample, adults only, ²e.g., management of somatic disease, diagnostic tool, ³reviewes, protocols, letter to the editor, ⁴author contacted for providing data of an eligible subsample, because the examined age range was not within the age range of the systematic review, ⁵studies about secondary analyses of other included studies not providing further information for app evaluation, ⁶other reasons, e.g., wrong publication date, studies not about app evaluation.

2.4 Quality assessment

To assess the quality of each study, the Systematic Assessment of Quality in Observational Research [SAQOR; (44)] was used. The assessment tool enables a differentiated evaluation of the heterogeneous study designs and methods without being limited to randomized controlled trials (RCT). It rates the quality of studies in six categories: sample, control/comparison group, quality of measurement(s) and outcome(s), follow-up, distorting influences, and reporting data. Each category consists of three to five items. Each category is rated as “adequate,” “inadequate,” “unclear,” or “not applicable” according to the ratings of each item. A final quality rating (high, moderate, low, very low) is determined based on the assessment of the six categories. Adapting the tool to the psychotherapeutic context, the items of the category “distorting influences” were summarized in one item asking for potential confounders in general instead of differentiating between two potential key confounders and additional possible confounders mentioned in the article. Two independent researchers carried out the quality assessment. Disagreements in the category ratings were discussed, involving a third party. Interrater reliability between the two reviewers was calculated using Cohen's Kappa (45).

2.5 Data extraction and synthesis

To synthesize data of included studies we followed the Synthesis Without Meta-analysis (SWiM) guidelines (46). First, in order to address the high degree of heterogeneity in the methodology and data of evaluation studies, we grouped the included studies by potential outcome measures: (1) qualitative measures only, (2) feasibility measures, (3) measures of app quality, or (4) measures of effectiveness. For studies that only include qualitative outcome measures or examine feasibility or app quality, data were synthesized in the form of a narrative summary. Such studies were grouped under the category “feasibility studies”. Results of studies including effectiveness measures were analyzed separately in two subcategories: effectiveness studies and feasibility studies with (preliminary) effectiveness outcomes. The reported effect estimates were categorized as indicating benefit or harm based on the observed direction of effect of the main effectiveness outcome. Studies that reported no effect of the app intervention were also rated as harmful. For synthesis, votes based on the direction of effects were counted to report the percentage of studies favoring app-based interventions for adolescent psychotherapy context (47). To test if the vote counting results are a statistically significant indicator of app-based intervention being truly effective, a binomial probability test was calculated (48). Differences in effectiveness according to different disorders or areas of application were reported in a narrative synthesis.

3 Results

The search identified 31 relevant studies in which 27 different apps were examined. Four of these apps were each addressed in two separate studies. Nine articles were primary effectiveness studies and 19 focused on feasibility measures as primary outcome, of which 14 studies also examined preliminary effectiveness outcomes. Three studies reported qualitative data only. None of the included studies focused on app quality as a primary outcome, but one study examined app quality as a secondary outcome (49). Following the World Bank's definition of high-income economy (50), studies were predominantly conducted in high-income countries (30/31, 97%), with the United States having the highest number of studies (10/31, 32%). Only one study was conducted in a lower-middle income country (India). Most of the evaluated apps focused on specific symptoms or disorders (n = 23), with only four acting as transdiagnostic interventions. Overall, 15 apps were based on cognitive behavioral therapy (CBT). Of the 27 apps, 17 were used as stand-alone interventions, seven as therapy-accompanying and three apps were community-network apps, focusing on peer-to-peer treatment or parent-child interactions. Tables 1–3 provide detailed descriptions of the included studies. An overview of the app characteristics is presented in Table 4.

Table 1

Table 1. Study characteristics of each study with primary effectiveness outcomes.

Table 2

Table 2. Study characteristics of each study with primary feasibility outcomes and secondary preliminary effectiveness outcomes.

Table 3

Table 3. Study characteristics of each study with primary feasibility outcomes (without outcomes on preliminary effectiveness).

Table 4

Table 4. Characteristics of each app.

3.1 Results of quality appraisal

Of the 31 studies, four were rated as “high” quality, eight as “moderate” quality, thirteen as “low” quality and six as “very low” quality. The two independent raters showed a moderate interrater reliability of κ = 0.59. Ratings of studies examining effectiveness as primary outcome ranged from high to low quality, with three studies rated as “high”, two as “moderate” and four as “low” quality. The results of the quality appraisal for each study are displayed in the summary of findings Tables 1–3.

3.2 App intervention concepts

The evaluated mental health apps showed high variety in their areas of application. Most apps were intended as stand-alone, psychological self-help programs. One app was specifically designed to support adolescents during the waiting period for psychotherapy (70). Three out of 27 apps were used as additional treatment to therapy, with two apps designed for specific manualized treatments (67, 68, 74). One app could be used in all standard treatments (52). Furthermore, four apps worked as interrelated apps that connected adolescents with their therapist and primary caregivers (58, 67, 68, 74) or self-selected family members and friends (49). As previously mentioned, four apps were transdiagnostic programs, while 23 apps focused on specific disorders or symptoms. Six out of the 23 apps addressed symptoms across disorders like suicidality and self-harm (n = 5) or rumination (n = 1). Apps about depressive disorders (n = 6) were most common. An overview of disorders addressed by included mental health apps is presented in Figure 2.

Figure 2

Figure 2. Number of mental health disorder grouped into DSM-5 categories that were addressed in the included apps.

3.3 App features

Overall, included mental health apps showed a wide range in established features and methods. Five out of the 27 apps followed a defined CBT manual with fixed modules that patients progressed through sequentially (51, 55, 57, 59, 63). Three apps used just-in-time adaptive interventions to treat adolescents (53, 60, 62, 71). All other apps did not specify how the app should be used. Thirty-three percent of apps used gamification elements to motivate patients to use the app (55, 58, 62, 64, 73, 76, 77, 79). Two apps used chatbots in their program to provide patients with personalized treatment (55, 70). Nevertheless, there were also many commonalities between the mental health apps. Fifty-six percent of the mental health apps used mood monitoring as one feature. In total, 63% of apps provided psychoeducational content, with 77% of these apps also providing specific exercises based on the presented psychoeducation. Almost half of the apps aimed to support adolescents in specific behavior changes, with three apps focusing on setting goals (51, 61, 65) and three apps focusing on problem-solving strategies (59, 64, 74). Another frequently used feature was a toolbox with useful skills for difficult situations (n = 9). Four out of these nine apps provided a safety plan for suicidal or self-harm crisis management (66, 69, 78, 80). Furthermore, three apps provided a diary feature for app users (59, 61, 78) and four apps had a community forum that enabled app users to communicate with other affected adolescents (49, 54, 65, 75).

3.4 Study designs of app evaluations

3.4.1 Studies of effectiveness

Six out of nine effectiveness studies were randomized controlled trials. Two studies were quasi-experimental cohort studies with a pre-post treatment assessment (52, 55) and one study used a single-case experimental design to examine effectiveness of the app intervention (51). Sample sizes ranged from five to 264, with a total of 802 participants across the nine studies. Fifty-six percent of the effectiveness studies examined adolescents under the age of 18 years. Most of the effectiveness studies included a comparison group, except for two studies that did not include any comparison group (51, 55). Five studies used an active control group, of which two studies compared their app-intervention with groups using other similar apps (53, 58) and three studies had a control group in which health related input was provided through other digital technologies (54, 56, 57). Intervention period of app usage lasted between three and 12 weeks with different frequencies of required app usage per day. Most of the apps prescribed daily use (n = 7), while two studies required using the app at least three times per day (53, 54). Overall, merely four studies registered their clinical trial and none of the studies published a study protocol.

3.4.2 Feasibility studies with preliminary effectiveness outcomes

As for feasibility studies that also examined (preliminary) effectiveness, 71% of these used an observational one-group cohort study design without a control group. The other four studies conducted a randomized controlled trial, with two studies comparing results with a wait list control group (63, 70), one study using treatment as usual (72) and one study including an attention control group (65). Sample sizes ranged from three to 248, with a total of 774 participants across the 14 studies. Twenty-nine percent of these studies examined adolescents under the age of 18 years. Two studies involved primary caregivers in intervention and assessment, in addition to the adolescent sample (67, 68), and one study involved friends of the participants (49). Intervention periods lasted between 2 and 12 weeks, with two studies requiring daily app usage (60, 62) and three studies requiring app use frequency of at least once a week (63, 64, 68). Of the 14 included feasibility studies with preliminary effectiveness outcomes, three studies registered their clinical trial (63, 65, 72) and one also published a study protocol (72).

3.4.3 Feasibility studies

Five out of eight studies used a descriptive cross-sectional study design to examine feasibility and usability of the apps (73–76, 78). None of the five studies included an intervention period to test the app in real life. For the evaluation component, the app was shown to participants in a single evaluation and assessment session. The other three studies were observational studies, with two studies using the app in regular psychotherapy (77, 79) and one including an intervention period of 90 days as a stand-alone treatment (80). Of the three qualitative studies one included a control group (79). Sample sizes of feasibility studies without effectiveness outcomes and qualitative outcome measures ranged from eight to 36 participants with a mean sample size of 16 adolescents. Moreover, 63% of the studies included mental health professionals or primary caregivers in app evaluation, in addition to the adolescent sample. Overall, a study protocol was published for only one study (74).

3.5 Outcomes

3.5.1 Effectiveness outcomes

Considering altogether studies with primary effectiveness outcomes and feasibility studies that examined preliminary effectiveness, 23 studies reported data about effectiveness outcomes, with all studies measuring effectiveness as a reduction of symptoms. Therefore, most studies used standardized questionnaires of symptoms or disorders addressed by the evaluated app. Only one study developed a new questionnaire to measure suicidality (66). An overview of the used outcome measures of each study is presented in Tables 1, 2. There was evidence that mental health apps influenced adolescent psychotherapy, with 19 out of 23 studies favoring the intervention (83%, p = 0.002). Four out of 23 studies were judged to be high quality, and all four favored the intervention. Overall, 11 studies were rated low quality, with 73% favoring the intervention. However, looking only at the studies that recorded effectiveness as the primary outcome (n = 9), no significant evidence could be found, with six out of nine studies favoring the intervention (67%; p = 0.508). Results of vote counting and available effect estimates are presented in the summary of findings tables (Tables 1, 2). Due to the small sample size of the included studies, it was not possible to evaluate effects of mental health apps according to different disorders or areas of application.

3.5.2 Feasibility outcomes

Feasibility was measured with a high heterogeneity in definition and methodology. There was no consistent definition of feasibility aspects across the included studies. Thirteen out of 14 studies used non-validated, self-developed questions about feasibility and acceptability to evaluate their apps. Similarities to standardized questionnaires for feasibility assessment were only found in a few studies: the Client Satisfaction Questionnaire [CSQ; (81)] was used in three app evaluation studies (63, 64, 77), and the System Usability Scale [SUS; (82)] was also used in three app evaluation studies (70, 78, 80). Other validated questionnaires were only used in single studies. As an objective measure, twelve studies presented app usage data as an indicator of feasibility, again showing no consistency in the data categories examined. Furthermore, 11 studies collected qualitative interview data about users' perspectives on the mental health app, with three studies collecting data using solely qualitative methods (74, 75, 79).

4 Discussion

4.1 Principal findings

The present review gives an overview of studies testing mobile applications in the context of adolescent psychotherapy during the COVID-19 pandemic. In the past 3 years, from June 2020 to June 2023, 31 studies on 27 mental health apps for adolescents between ages 12 and 27 years were published. Table 4 presents an overview of all included mental health apps. Across all studies, effectiveness was defined as a reduction of symptoms and was mostly surveyed with standardized questionnaires about symptoms or disorders. The results of the included studies measuring effectiveness as a primary or secondary outcome indicate that mental health apps are effective for adolescent psychotherapy, with 83% of mental health app studies favoring app-based interventions and the other 17% showing no effect on symptom reduction. No published study showing negative effects on adolescents' wellbeing was found. Nevertheless, these results cannot be presented as evidence of the overall effectiveness of mental health apps for adolescents. Focusing on effectiveness as primary outcome only, we did not find significant evidence that mental health apps are truly effective for adolescents. These findings are consistent with the results of previous research, which also reported promising but inconclusive results of the overall effectiveness of mental health apps (33, 83, 84).

4.2 Quality of evaluation studies

One possible reason for the inconclusive results of effectiveness outcomes is the high heterogeneity of study methodology and quality appraisal. Among studies that examined effectiveness as the primary outcome, study quality ranged from low to high, with 44% rated low quality. Two thirds of the effectiveness studies were RCTs, with five studies including active control groups. Two of the high-quality studies used another app within their control group. Three studies did not include comparison groups and one effectiveness study made statements about effectiveness of their treatment using visual analysis of symptom reduction in five participants (51). Another indication of poor research quality in mental health application studies is the non-adherence to established standards, such as good clinical practice guidelines, particularly evident in failure to register their respective studies with a clinical trial registry. Out of the nine effectiveness studies, only four were registered. As other researchers have pointed out, most of the available mental health apps do not provide evidence on their effectiveness (12, 13, 32, 37, 41). In line with the above, we rated a high number of the evaluation studies included in the current review at low quality. Therefore, it remains unclear whether the few apps that show some evidence for their effectiveness were evaluated with studies ensuring good clinical practice and quality.

The other 71% of the included studies focused on feasibility as primary outcome. Considering the steep increase in mental health apps released in app stores (10, 11), the predominance of published feasibility studies over effectiveness studies is not surprising. Following the steps of developing and evaluating new clinical interventions, focusing on feasibility and overall user experience with the new intervention is a common first step before organizing an elaborate effectiveness study. However, Larsen et al. (41) reported that app providers are frequently content with positive results about feasibility and acceptability of mental health apps and do not continue the evaluation of the app further, for example by doing studies on the effectiveness. We also find a high heterogeneity in the quality appraisal and study methodology in the included feasibility studies. Out of the studies measuring feasibility as primary outcome, 68% were rated low or very low quality. Study designs ranged from RCTs to descriptive cross-sectional design studies. Some feasibility studies did not include an intervention period but had single evaluation sessions to rate the developed mental health app. Sample sizes likewise had a wide range from three to 248 included participants. In addition to the high heterogeneity in study design and procedure, measurement of feasibility did not follow a consistent definition. Most of the studies developed own items ranging from questions about having fun using the app or being satisfied with the app, to detailed questions about app functionality or design, to objective usage data like the number of logins or the duration of use. Considering this range of feasibility aspects, it is not possible to make generalized statements about the feasibility of mental health apps for adolescents.

In order to determine the feasibility and effectivity of mental health apps properly, we need researchers to define concepts like app quality and to develop and use corresponding measurements. Future studies should try to apply common scientific standards like study registration, control-group designs, adequate sample size to the field of app evaluation. Multi-method and multi-informand approaches seem promising. However, it is challenging to combine high quality evaluation studies (feasibility studies as well as effectiveness studies) with the fast pace of new developments of mental health apps (14, 38). Nevertheless, this is the only way to expand the knowledge on apps in psychotherapy.

4.3 Overview of evaluated mental health apps for adolescents during the COVID-19 pandemic

The area of application of mental health apps did not change significantly during the COVID-19 pandemic. About 70% of the mental health apps were offered as stand-alone treatments, replacing usual psychotherapy services, or being used as a bridge-over during the waiting period for psychotherapy. These results are comparable with previous research, showing that more stand-alone mental health apps were provided than therapy-accompanying mental health apps (34). Most therapy-accompanying apps included in our review provide interrelated app-versions for adolescents and their therapists. All of them were designed for supporting outpatient psychotherapy. Therefore, app features are designed with the aim of supporting patients in therapy homework, monitoring their mood between outpatient sessions or facilitating communication between patient and therapist. Two therapy-accompanying apps did not connect patients directly with their therapists. These apps were designed as accompanying tools for specific manualized treatments with fixed therapy modules following the same order. A new development in this area are apps involving the community network of affected youths. Like Diano et al. (29), former research could typically be divided into two subgroups: stand-alone or therapy-accompanying apps. In the current review, community-network apps were discovered as a third subgroup of the app intervention concept. Three out of the 27 apps were used as community-network apps, connecting youth with primary caregivers or peers. These interventions were predominantly based on family-focused therapy that included primary caregivers or peers as lay counselors and main support for the affected youths. What is remarkable is that two out of the three community-network apps supported the mental health of youth by addressing the issue of self-harm and suicidality. The results of the evaluation studies showed evidence of effectiveness and feasibility without showing negative side effects either on the side of the affected youth or the selected peers or caregivers (49, 54). The development of specific community-network mental health apps could potentially be a consequence of the increased mental health concerns and the resulting pressure to treat during the pandemic. Involving the social environment in supporting the affected youth could relieve the burden on the healthcare system (4, 7, 22) and enable more youth to receive support, especially at an early stage (30). In addition, the feeling of connectedness is an important factor for adolescents (27, 28) and could be another reason for having a greater focus on the social environment in treatment development.

Overall, there is still a greater focus on developing mental health apps for specific disorders or symptoms rather than following a transdiagnostic approach. The current review cannot confirm the trend found by Ellis et al. (9) that transdiagnostic approaches were increasing in the mental health app development. In fact, we found a comparable percentage of transdiagnostic approaches as opposed to those for specific disorders. The increased focus on transdiagnostic treatment approaches is in line with the current psychotherapy research. Instead of developing treatment concepts for clearly defined, specific disorders, psychotherapy research is increasingly trying to develop treatment concepts that focus on symptoms and treatment principles that transcend disorders (85, 86). Moreover, the most common app features, also used in the included disorder-specific mental health apps, were not disorder specific features. For example, mood monitoring and features supporting specific behavior changes were used by almost half of the mental health apps, while skill toolboxes were part of one third of the apps. It can therefore be argued that most of the developed applications exhibited the potential to offer support beyond their primary focus on specific mental health concerns, as each of them incorporated fundamentally transdiagnostic features within their respective platforms. Having a look at the reported disorders of youth, we can see that most adolescents had comorbidities and were diagnosed with more than one specific disorder (18, 87, 88). In order to address the complexity of mental health issues effecting young people, it is necessary to offer transdiagnostic mental health apps. It is not feasible to equip an adolescent with several apps at the time, each one for another psychological problem.

4.4 Strengths and limitations

The current review focused on a narrowly defined period from June 2020 to June 2023 in order to examine the pandemic's influence on mental health app development and evaluation. Including only studies published during the COVID-19 pandemic provides a good overview of recently published, studies on mental health apps that possibly take into account the new needs that have arisen in the healthcare sector due to the pandemic. However, the current review did not exclusively include pandemic-related mental health apps, as evaluation studies conducted before the pandemic but published after June 2020 were also included. Another methodological limitation has to be mentioned for the reviewing process. One reviewer for each database, which could introduce selection bias, conducted the initial screening process. Including all kinds of evaluation studies is another strength of the current review. The given overview presents evidence from different evaluation stages and provides a more complete picture of the current developments in the field of mental health apps. However, looking at all types of evaluation studies also weakens the robustness of the results on the effectiveness of mental health apps. When interpreting the results of the included mental health app evaluation studies, we have to consider the overall low quality of study methodology.

4.5 Conclusion

Taken altogether, it is evident that within the spectrum of evaluation studies, there are significant disparities in both quality and methodological approaches. There is an urgent need to improve quality of evaluation studies and to ensure that research on mobile mental health complies with the established scientific standards. At the latest since the pandemic, mental health apps have a firm and important role in our healthcare system and are likely to continue to grow in influence in the future. Clinicians as well as adolescents in need are more likely to use digital mental health support, but actual app development policy aggravates the access to high-qualitative evaluated apps. Only with defined standards and high-quality research can we ensure that feasible and effective apps are implemented in psychotherapy with adolescents.

Data availability statement

The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.

Author contributions

SW: Conceptualization, Formal analysis, Investigation, Methodology, Visualization, Writing—original draft. KH: Conceptualization, Methodology, Writing—review & editing. MK: Investigation, Writing—review & editing. I-KP: Investigation, Writing—review & editing. TH: Supervision, Writing—review & editing. MS: Funding acquisition, Project administration, Writing—review & editing.

Funding

The author(s) declare financial support was received for the research, authorship, and/or publication of this article. This publication was funded by the German Federal Ministry of Education and Research (BMBF) Network of University Medicine 2.0: NUM 2.0, Grant No. 01KX2121, Project: COVID-19 Research Infrastructure Platform for Children and Adolescents—coverCHILD. We acknowledge support for the publication costs by the Open Access Publication Fund of Bielefeld University and the Deutsche Forschungsgemeinschaft (DFG).

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

1. Alessandro G, Julio T, O'Higgins M, Figueredo P, Mauricio Castaldelli-Maia J, De Berardis D, et al. (2022). Phone-based outpatients' follow-up in mental health centers during the COVID-19 quarantine. Int J Soc Psychiatry. 68:129–33. doi: 10.1177/0020764020979732

PubMed Abstract | Crossref Full Text | Google Scholar

2. Amerio A, Vai E, Bruno E, Costanza A, Escelsior A, Odone A, et al. COVID-19 impact on the Italian community-based system of mental health care: reflections and lessons learned for the future. Clin Psychopharmacol Neurosci. (2023) 21:2–9. doi: 10.9758/cpn.2023.21.1.2

PubMed Abstract | Crossref Full Text | Google Scholar

3. Bouter DC, Zarchev M, de Neve-Enthoven NGM, Ravensbergen SJ, Kamperman AM, Hoogendijk WJG, et al. A longitudinal study of mental health in at-risk adolescents before and during the COVID-19 pandemic. Eur Child Adolesc Psychiatry. (2023) 32:1109–17. doi: 10.1007/s00787-021-01935-y

PubMed Abstract | Crossref Full Text | Google Scholar

4. Kauhanen L, Wan Mohd Yunus WMA, Lempinen L, Peltonen K, Gyllenberg D, Mishina K, et al. A systematic review of the mental health changes of children and young people before and during the COVID-19 pandemic. Eur Child Adolesc Psychiatry. (2023) 32:995–1013. doi: 10.1007/s00787-022-02060-0

PubMed Abstract | Crossref Full Text | Google Scholar

5. Madigan S, Korczak DJ, Vaillancourt T, Racine N, Hopkins WG, Pador P, et al. Comparison of paediatric emergency department visits for attempted suicide, self-harm, and suicidal ideation before and during the COVID-19 pandemic: a systematic review and meta-analysis. Lancet Psychiatry. (2023) 10:342–51. doi: 10.1016/S2215-0366(23)00036-6

PubMed Abstract | Crossref Full Text | Google Scholar

6. Newlove-Delgado T, Russell AE, Mathews F, Cross L, Bryant E, Gudka R, et al. Annual Research Review: the impact of Covid-19 on psychopathology in children and young people worldwide: systematic review of studies with pre- and within-pandemic data. J Child Psychol Psychiatry. (2023) 64:611–40. doi: 10.1111/jcpp.13716

PubMed Abstract | Crossref Full Text | Google Scholar

7. Panchal U, Salazar de Pablo G, Franco M, Moreno C, Parellada M, Arango C, et al. The impact of COVID-19 lockdown on child and adolescent mental health: systematic review. Eur Child Adolesc Psychiatry. (2023) 32:1151–77. doi: 10.1007/s00787-021-01856-w

Crossref Full Text | Google Scholar

8. Torous J, Myrick KJ, Rauseo-Ricupero N, Firth J. Digital mental health and COVID-19: using technology today to accelerate the curve on access and quality tomorrow. JMIR Ment Health. (2020) 7:e18848. doi: 10.2196/18848

PubMed Abstract | Crossref Full Text | Google Scholar

9. Ellis LA, Meulenbroeks I, Churruca K, Pomare C, Hatem S, Harrison R, et al. The application of e-Mental health in response to COVID-19: scoping review and bibliometric analysis. JMIR Ment Health. (2021) 8:e32948. doi: 10.2196/32948

PubMed Abstract | Crossref Full Text | Google Scholar

10. Appfigures. Number of mHealth Apps Available in the Apple App Store From 1st Quarter 2015 to 3rd Quarter 2022 [Graph]. (2022). Available online at: https://www.statista.com/statistics/779910/health-apps-available-ios-worldwide/ (accessed August 30, 2023).

Google Scholar

11. Appfigures. Number of mHealth Apps Available in the Google Play Store From 1st Quarter 2015 to 3rd Quarter 2022 [Graph]. (2022). Available online at: https://www.statista.com/statistics/779919/health-apps-available-google-play-worldwide/ (accessed August 30, 2023).

Google Scholar

12. Alyami M, Giri B, Alyami H, Sundram F. Social anxiety apps: a systematic review and assessment of app descriptors across mobile store platforms. Evid Based Ment Health. (2017) 20:65–70. doi: 10.1136/eb-2017-102664

PubMed Abstract | Crossref Full Text | Google Scholar

13. Qu C, Sas C, Daudén Roquet C, Doherty G. Functionality of top-rated mobile apps for depression: systematic search and evaluation. JMIR Ment Health. (2020) 7:e15321. doi: 10.2196/15321

PubMed Abstract | Crossref Full Text | Google Scholar

14. Neary M, Schueller SM. State of the field of mental health apps. Cogn Behav Pract. (2018) 25:531–7. doi: 10.1016/j.cbpra.2018.01.002

Crossref Full Text | Google Scholar

15. Marshall JM, Dunstan DA, Bartik W. Smartphone psychology: new approaches towards safe and efficacious mobile mental health apps. Prof Psychol. (2020) 51:214–22. doi: 10.1037/pro0000278

Crossref Full Text | Google Scholar

16. Nicholas J, Larsen ME, Proudfoot J, Christensen H. Mobile apps for bipolar disorder: a systematic review of features and content quality. J Med Internet Res. (2015) 17:e198. doi: 10.2196/jmir.4581

PubMed Abstract | Crossref Full Text | Google Scholar

17. Martinengo L, van Galen L, Lum E, Kowalski M, Subramaniam M, Car J. Suicide prevention and depression apps' suicide risk assessment and management: a systematic assessment of adherence to clinical guidelines. BMC Med. (2019) 17:231. doi: 10.1186/s12916-019-1461-z

PubMed Abstract | Crossref Full Text | Google Scholar

18. Ravens-Sieberer U, Erhart M, Devine J, Gilbert M, Reiss F, Barkmann C, et al. Child and adolescent mental health during the COVID-19 pandemic: results of the three-wave longitudinal COPSY study. J Adolesc Health. (2022) 71:570–8. doi: 10.1016/j.jadohealth.2022.06.022

PubMed Abstract | Crossref Full Text | Google Scholar

19. Wan Mohd Yunus WMA, Kauhanen L, Sourander A, Brown JSL, Peltonen K, Mishina K, et al. Registered psychiatric service use, self-harm and suicides of children and young people aged 0-24 before and during the COVID-19 pandemic: a systematic review. Child Adolesc Psychiatry Ment Health. (2022) 16:15. doi: 10.1186/s13034-022-00452-3

Crossref Full Text | Google Scholar

20. O'Sullivan K, Clark S, McGrane A, Rock N, Burke L, Boyle N, et al. A qualitative study of child and adolescent mental health during the COVID-19 pandemic in Ireland. Int J Environ Res Public Health. (2021) 18:1062. doi: 10.3390/ijerph18031062

PubMed Abstract | Crossref Full Text | Google Scholar

21. Ravens-Sieberer U, Kaman A, Erhart M, Devine J, Schlack R, Otto C. Impact of the COVID-19 pandemic on quality of life and mental health in children and adolescents in Germany. Eur Child Adolesc Psychiatry. (2022) 31:879–89. doi: 10.1007/s00787-021-01726-5

PubMed Abstract | Crossref Full Text | Google Scholar

22. Sevecke K, Wenter A, Schickl M, Kranz M, Krstic N, Fuchs M. Stationäre Versorgungskapazitäten in der Kinder- und Jugendpsychiatrie – Zunahme der Akutaufnahmen während der COVID-19 Pandemie? Neuropsychiatr. (2023) 37:12–21. doi: 10.1007/s40211-022-00423-2

PubMed Abstract | Crossref Full Text | Google Scholar

23. Feierabend S, Rathgeb T, Kheredmand H, Glöckler S. JIM-Studie 2022: Jugend, Information, Medien. Stuttgart: Basisuntersuchung zum Medienumgang 12- bis 19-jähriger (2022).

Google Scholar

24. Hennemann S, Beutel ME, Zwerenz R. Drivers and barriers to acceptance of web-based aftercare of patients in inpatient routine care: a cross-sectional survey. J Med Int Res. (2016) 18:e337. doi: 10.2196/jmir.6003

PubMed Abstract | Crossref Full Text | Google Scholar

25. Breil B, Dederichs M, Kremer L, Richter D, Angerer P, Apolinário-Hagen J. Bekanntheit und Nutzung von digitalen Gesundheitsangeboten in Deutschland: eine bevölkerungsrepräsentative Querschnittsuntersuchung. Das Gesundheitswesen. (2021) 83:1019–28. doi: 10.1055/a-1335-4245

PubMed Abstract | Crossref Full Text | Google Scholar

26. Rauschenberg C, Schick A, Hirjak D, Seidler A, Paetzold I, Apfelbacher C, et al. Evidence synthesis of digital interventions to mitigate the negative impact of the COVID-19 pandemic on public mental health: rapid meta-review. J Med Internet Res. (2021) 23:e23365. doi: 10.2196/23365

PubMed Abstract | Crossref Full Text | Google Scholar

27. Liverpool S, Mota CP, Sales CMD, Cuš A, Carletto S, and Hancheva C, et al. Engaging children and young people in digital mental health interventions: systematic review of modes of delivery, facilitators, and barriers. J Med Internet Res. (2020) 22:e16317. doi: 10.2196/16317

PubMed Abstract | Crossref Full Text | Google Scholar

28. d'Halluin A, Costa M, Morgiève M, Sebbane D. Attitudes of children, adolescents, and their parents toward digital health interventions: scoping review. J Med Internet Res. (2023) 25:e43102. doi: 10.2196/43102

Crossref Full Text | Google Scholar

29. Diano F, Ponticorvo M, Sica LS. Mental health mobile apps to empower psychotherapy: a narrative review. In: 2022 IEEE International Conference on Metrology for Extended Reality, Artificial Intelligence and Neural Engineering (MetroXRAINE) Rome: IEEE. (2022). p. 306–11.

Google Scholar

30. Wies B, Landers C, Ienca M. Digital mental health for young people: a scoping review of ethical promises and challenges. Front Digit Health. (2021) 3:697072. doi: 10.3389/fdgth.2021.697072

PubMed Abstract | Crossref Full Text | Google Scholar

31. Ly KH, Topooco N, Cederlund H, Wallin A, Bergström J, Molander O, et al. Smartphone-supported versus full behavioural activation for depression: a randomised controlled trial. PLoS ONE. (2015) 10:e0126559. doi: 10.1371/journal.pone.0126559

PubMed Abstract | Crossref Full Text | Google Scholar

32. Firth J, Torous J, Nicholas J, Carney R, Pratap A, Rosenbaum S, et al. The efficacy of smartphone-based mental health interventions for depressive symptoms: a meta-analysis of randomized controlled trials. World Psychiatry. (2017) 16:287–98. doi: 10.1002/wps.20472

PubMed Abstract | Crossref Full Text | Google Scholar

33. Zhou X, Edirippulige S, Bai X, Bambling M. Are online mental health interventions for youth effective? A systematic review. J Telemed Telecare. (2021) 27:638–66. doi: 10.1177/1357633X211047285

Crossref Full Text | Google Scholar

34. Lui JHL, Marcus DK, Barry CT. Evidence-based apps? A review of mental health mobile applications in a psychotherapy context. Prof Psychol Res Pract. (2017) 48:199–210. doi: 10.1037/pro0000122

Crossref Full Text | Google Scholar

35. Melbye S, Kessing LV, Bardram JE, Faurholt-Jepsen M. Smartphone-based self-monitoring, treatment, and automatically generated data in children, adolescents, and young adults with psychiatric disorders: systematic review. JMIR Ment Health. (2020) 7:e17453. doi: 10.2196/17453

Crossref Full Text | Google Scholar

36. Pires IM, Marques G, Garcia NM, Flórez-Revuelta F, Ponciano V, Oniani S. A research on the classification and applicability of the mobile health applications. J Pers Med. (2020) 10:11. doi: 10.3390/jpm10010011

PubMed Abstract | Crossref Full Text | Google Scholar

37. Marshall JM, Dunstan DA, Bartik W. Apps with maps-anxiety and depression mobile apps with evidence-based frameworks: systematic search of major app stores. JMIR Ment Health. (2020) 7:e16525. doi: 10.2196/16525

PubMed Abstract | Crossref Full Text | Google Scholar

38. Olff M. Mobile mental health: a challenging research agenda. Eur J Psychotraumatol. (2015) 6:27882. doi: 10.3402/ejpt.v6.27882

PubMed Abstract | Crossref Full Text | Google Scholar

39. Apple Developer. App Store Review Guidelines. (2023). Available online at: https://developer.apple.com/app-store/review/guidelines/ (accessed September 6, 2023).

Google Scholar

40. Google Play. Programmrichtlinien für Entwickler. (2023). Available online at: https://play.google.com/intl/de/about/developer-content-policy/ (accessed September 6, 2023).

Google Scholar

41. Larsen ME, Huckvale K, Nicholas J, Torous J, Birrell L, Li E, et al. Using science to sell apps: Evaluation of mental health app store quality claims. NPJ Digit Med. (2019) 2:18. doi: 10.1038/s41746-019-0093-1

PubMed Abstract | Crossref Full Text | Google Scholar

42. Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Syst Rev. (2021) 10:89. doi: 10.1186/s13643-021-01626-4

Crossref Full Text | Google Scholar

43. Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan-a web and mobile app for systematic reviews. Syst Rev. (2016) 5:210. doi: 10.1186/s13643-016-0384-4

PubMed Abstract | Crossref Full Text | Google Scholar

44. Ross LE, Grigoriadis S, Mamisashvili L, Koren G, Steiner M, Dennis C-L, et al. Quality assessment of observational studies in psychiatry: an example from perinatal psychiatric research. Int J Methods Psychiatr Res. (2011) 20:224–34. doi: 10.1002/mpr.356

PubMed Abstract | Crossref Full Text | Google Scholar

45. Cohen J. A coefficient of agreement for nominal scales. Educ Psychol Meas. (1960) 20:37–46. doi: 10.1177/001316446002000104

Crossref Full Text | Google Scholar

46. Campbell M, McKenzie JE, Sowden A, Katikireddi SV, Brennan SE, Ellis S, et al. Synthesis without meta-analysis (SWiM) in systematic reviews: reporting guideline. BMJ. (2020) 368:l6890. doi: 10.1136/bmj.l6890

PubMed Abstract | Crossref Full Text | Google Scholar

47. Higgins JP, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al. (editors). Cochrane Handbook for Systematic Reviews of Interventions. Chichester: John Wiley and Sons (2019).

Google Scholar

48. McKenzie JE, Brennan SE. Chapter 12: Synthesizing and presenting findings using other methods. In:Higgins JP, Thomas J, Chandler J, Cumpston M, Li T, Page MI, editors. Cochrane Handbook for Systematic Reviews of Interventions. Chichester: John Wiley and Sons (2019). p. 321–48.

Google Scholar

49. Thabrew H, Kumar H, Steadman E. Acceptability and feasibility of “village,” a digital communication app for young people experiencing low mood, thoughts of self-harm, and suicidal ideation to obtain support from family and friends: mixed methods pilot open trial. JMIR Form Res. (2023) 7:e41273. doi: 10.2196/41273

Crossref Full Text | Google Scholar

50. World Bank. World Bank Country and Lending Groups. (2023). Available online at: https://datahelpdesk.worldbank.org/knowledgebase/articles/906519-world-bank-country-and-lending-groups (accessed September 15, 2023).

Google Scholar

51. Badesha K, Wilde S, Dawson DL. Mental health mobile application self-help for adolescents exhibiting psychological distress: a single case experimental design. Psychol Psychother. (2023) 96:223–48. doi: 10.1111/papt.12436

PubMed Abstract | Crossref Full Text | Google Scholar

52. Dubad M, Elahi F, Marwaha S. The clinical impacts of mobile mood-monitoring in young people with mental health problems: The MeMO Study. Front Psychiatry. (2021) 12:687270. doi: 10.3389/fpsyt.2021.687270

PubMed Abstract | Crossref Full Text | Google Scholar

53. Hilt LM, Swords CM, Webb CA. Randomized controlled trial of a mindfulness mobile application for ruminative adolescents. J Clin Child Adolesc Psychol. (2023) 2158840. doi: 10.1080/15374416.2022.2158840

PubMed Abstract | Crossref Full Text | Google Scholar

54. Kruzan KP, Whitlock J, Bazarova NN, Bhandari A, Chapman J. Use of a mobile peer support app among young people with nonsuicidal self-injury: small-scale randomized controlled trial. JMIR Form Res. (2022) 6:e26526. doi: 10.2196/26526

PubMed Abstract | Crossref Full Text | Google Scholar

55. Li SH, Graham BM, Werner-Seidler A. Gender differences in adolescent sleep disturbance and treatment response to smartphone app–delivered cognitive behavioral therapy for insomnia: exploratory study. JMIR Form Res. (2021) 5:e22498. doi: 10.2196/22498

PubMed Abstract | Crossref Full Text | Google Scholar

56. Werner-Seidler A, Li SH, Spanos S, Johnston L, O'Dea B, Torok M, et al. The effects of a sleep-focused smartphone application on insomnia and depressive symptoms: a randomised controlled trial and mediation analysis. J Child Psychol Psychiatry. (2023) 64:1324–35. doi: 10.1111/jcpp.13795

PubMed Abstract | Crossref Full Text | Google Scholar

57. Rempel S, Backhausen LL, McDonald M, Roessner V, Vetter NC, Beste C, et al. App-based mindfulness meditation training and an audiobook intervention reduce symptom severity but do not modify backward inhibition in adolescent obsessive-compulsive disorder: evidence from an EEG study. J Clin Med. (2023) 12:2486. doi: 10.3390/jcm12072486

PubMed Abstract | Crossref Full Text | Google Scholar

58. Schaeffer CM, Dimeff LA, Koerner K, Chung J, Kelley-Brimer A, Kako N, et al. A smartphone app for parental management of adolescent conduct problems: randomized clinical trial of iKinnect. J Clin Child Adolesc Psychol. (2022) 1–15. doi: 10.1080/15374416.2022.2070851

PubMed Abstract | Crossref Full Text | Google Scholar

59. Yang YJ, Chung K-M. Pilot randomized control trial of an app-based CBT program for reducing anxiety in individuals with ASD without intellectual disability. J Autism Dev Disord. (2023) 53:1331–46. doi: 10.1007/s10803-022-05617-9

PubMed Abstract | Crossref Full Text | Google Scholar

60. Hilt LM, Swords CM. Acceptability and preliminary effects of a mindfulness mobile application for ruminative adolescents. Behav Ther. (2021) 52:1339–50. doi: 10.1016/j.beth.2021.03.004

PubMed Abstract | Crossref Full Text | Google Scholar

61. Carmona NE, Usyatynsky A, Kutana S, Corkum P, Henderson J, McShane K, et al. A transdiagnostic self-management web-based app for sleep disturbance in adolescents and young adults: Feasibility and acceptability study. JMIR Form Res. (2021) 5:e25392. doi: 10.2196/25392

PubMed Abstract | Crossref Full Text | Google Scholar

62. Coughlin LN, Nahum-Shani I, Philyaw-Kotov ML, Bonar EE, Rabbi M, Klasnja P, et al. Developing an adaptive mobile intervention to address risky substance use among adolescents and emerging adults: usability study. JMIR Mhealth Uhealth. (2021) 9:e24424. doi: 10.2196/24424

PubMed Abstract | Crossref Full Text | Google Scholar

63. Geirhos A, Domhardt M, Lunkenheimer F, Temming S, Holl RW, Minden K, et al. Feasibility and potential efficacy of a guided internet- and mobile-based CBT for adolescents and young adults with chronic medical conditions and comorbid depression or anxiety symptoms (youthCOACH(CD)): a randomized controlled pilot trial. BMC Pediatr. (2022) 22:69. doi: 10.1186/s12887-022-03134-3

PubMed Abstract | Crossref Full Text | Google Scholar

64. Gonsalves PP, Hodgson ES, Bhat B, Sharma R, Jambhale A, Michelson D, et al. App-based guided problem-solving intervention for adolescent mental health: a pilot cohort study in Indian schools. Evid Based Ment Health. (2021) 24:11–8. doi: 10.1136/ebmental-2020-300194

PubMed Abstract | Crossref Full Text | Google Scholar

65. Grasaas E, Helseth S, Fegran L, Stinson J, Småstuen M, Lalloo C, et al. App-based intervention among adolescents with persistent pain: a pilot feasibility randomized controlled trial. Pilot Feasibility Stud. (2022) 8:158. doi: 10.1186/s40814-022-01113-0

PubMed Abstract | Crossref Full Text | Google Scholar

66. Jeong YW, Chang HJ, Kim JA. Development and feasibility of a safety plan mobile application for adolescent suicide attempt survivors. Comp Informat Nurs. (2020) 38:382–92. doi: 10.1097/CIN.0000000000000592

PubMed Abstract | Crossref Full Text | Google Scholar

67. Miklowitz DJ, Weintraub MJ, Posta F, Walshaw PD, Frey SJ, Morgan-Fleming GM, et al. Development and open trial of a technology-enhanced family intervention for adolescents at risk for mood disorders. J Affect Disord. (2021) 281:438–46. doi: 10.1016/j.jad.2020.12.012

PubMed Abstract | Crossref Full Text | Google Scholar

68. Weintraub MJ, Ichinose MC, Zinberg J, Done M, Morgan-Fleming GM, Wilkerson CA, et al. App-enhanced transdiagnostic CBT for adolescents with mood or psychotic spectrum disorders. J Affect Disord. (2022) 311:319–26. doi: 10.1016/j.jad.2022.05.094

PubMed Abstract | Crossref Full Text | Google Scholar

69. Muscara F, Ng O, Crossley L, Lu S, Kalisch L, Melvin G, et al. The feasibility of using smartphone apps to manage self-harm and suicidal acts in adolescents admitted to an inpatient mental health ward. Digit Health. (2020) 6:2055207620975315. doi: 10.1177/2055207620975315

PubMed Abstract | Crossref Full Text | Google Scholar

70. Nicol G, Wang R, Graham S, Dodd S, Garbutt J. Chatbot-delivered cognitive behavioral therapy in adolescents with depression and anxiety during the COVID-19 pandemic: feasibility and acceptability study. JMIR Form Res. (2022) 6:e40242. doi: 10.2196/40242

PubMed Abstract | Crossref Full Text | Google Scholar

71. Rauschenberg C, Boecking B, Paetzold I, Schruers K, Schick A, van Amelsvoort T, et al. A compassion-focused ecological momentary intervention for enhancing resilience in help-seeking youth: uncontrolled pilot study. JMIR Ment Health. (2021) 8:e25650. doi: 10.2196/25650

PubMed Abstract | Crossref Full Text | Google Scholar

72. Reininghaus U, Paetzold I, Rauschenberg C, Hirjak D, Banaschewski T, Meyer-Lindenberg A, et al. Effects of a novel, transdiagnostic ecological momentary intervention for prevention, and early intervention of severe mental disorder in youth (EMIcompass): findings from an exploratory randomized controlled trial. Schizophr Bull. (2023) 49:592–604. doi: 10.1093/schbul/sbac212

PubMed Abstract | Crossref Full Text | Google Scholar

73. Adams Z, Grant M, Hupp S, Scott T, Feagans A, Phillips ML, et al. Acceptability of an mHealth app for youth with substance use and mental health needs: iterative, mixed methods design. JMIR Form Res. (2021) 5:e30268. doi: 10.2196/30268

PubMed Abstract | Crossref Full Text | Google Scholar

74. Gómez-Restrepo C, Marroquín-Rivera A, Calvo-Valderrama MG, Ospina-Pinillos L, Sureshkumar DS, Bird VJ. Stakeholder perspectives on a patient-centred intervention (DIALOG+) for adolescents with common mental disorders in Colombia: a qualitative study. PLoS ONE. (2022) 17:e0272066. doi: 10.1371/journal.pone.0272066

PubMed Abstract | Crossref Full Text | Google Scholar

75. Li SH, Achilles MR, Spanos S, Habak S, Werner-Seidler A, O'Dea B. A cognitive behavioural therapy smartphone app for adolescent depression and anxiety: co-design of ClearlyMe. Cogn Behav Ther. (2022) 15. doi: 10.1017/S1754470X22000095

Crossref Full Text | Google Scholar

76. Naccache B, Mesquida L, Raynaud J-P, Revet A. Smartphone application for adolescents with anorexia nervosa: An initial acceptability and user experience evaluation. BMC Psychiatry. (2021) 21:467. doi: 10.1186/s12888-021-03478-7

PubMed Abstract | Crossref Full Text | Google Scholar

77. Newton A, Bagnell A, Rosychuk R, Duguay J, Wozney L, Huguet A, et al. A mobile phone-based app for use during cognitive behavioral therapy for adolescents with anxiety (MindClimb): user-centered design and usability study. JMIR Mhealth Uhealth. (2020) 8:e18439. doi: 10.2196/18439

PubMed Abstract | Crossref Full Text | Google Scholar

78. O'Grady C, Melia R, Bogue J, O'Sullivan M, Young K, Duggan J. A mobile health approach for improving outcomes in suicide prevention (SafePlan). J Med Internet Res. (2020) 22:e17481. doi: 10.2196/17481

PubMed Abstract | Crossref Full Text | Google Scholar

79. Patterson Silver Wolf DA, Ramsey AT, Epstein J, Beeler-Stinn S, Asher Black Deer A. Bridges to sobriety: testing the feasibility and acceptability of a mobile app designed to supplement an adolescent substance use disorder treatment program. Clin Soc Work J. (2022) 50:308–15. doi: 10.1007/s10615-020-00765-w

PubMed Abstract | Crossref Full Text | Google Scholar

80. Sharma AN, Barron-Millar E, Gaskell M, Glod M, Kaushal P, Grieve K, et al. Technology matters: collaboratively augmenting longitudinal monitoring (CALM) in bipolar disorder—Co-design, co-production and evaluation of the alpha prototype app. Child Adolesc Ment Health. (2022) 27:427–9. doi: 10.1111/camh.12548

PubMed Abstract | Crossref Full Text | Google Scholar

81. Attkisson CC, Greenfield TK. Client satisfaction questionnaire-8 and service satisfaction scale-30. In:Maruish ME, editor. The Use of Psychological Testing for Treatment Planning and Outcome Assessment. Lawrence Erlbaum Associates, Inc. (1994). p. 402–20.

Google Scholar

82. Brooke J. SUS: a quick and dirty usability scale. In:Jordan PW, Thomas B, McClelland IL, Weerdmeester B, editor. Usability Evaluation in Industry. London: Taylor & Francis (1996). p. 189–94.

Google Scholar

83. Buttazzoni A, Brar K, Minaker L. Smartphone-based interventions and internalizing disorders in youth: systematic review and meta-analysis. J Med Internet Res. (2021) 23:e16490. doi: 10.2196/16490

PubMed Abstract | Crossref Full Text | Google Scholar

84. Lehtimaki S, Martic J, Wahl B, Foster KT, Schwalbe N. Evidence on digital mental health interventions for adolescents and young people: systematic overview. JMIR Ment Health. (2021) 8:e25847. doi: 10.2196/25847

PubMed Abstract | Crossref Full Text | Google Scholar

85. Gros DF, Allan NP, Szafranski DD. Movement towards transdiagnostic psychotherapeutic practices for the affective disorders. Evid Based Ment Health. (2016) 19:e10–2. doi: 10.1136/eb-2015-102286

PubMed Abstract | Crossref Full Text | Google Scholar

86. Fusar-Poli P, Solmi M, Brondino N, Davies C, Chae C, Politi P, et al. Transdiagnostic psychiatry: a systematic review. World Psychiatry. (2019) 18:192–207. doi: 10.1002/wps.20631

Crossref Full Text | Google Scholar

87. Merikangas KR, He J-P, Burstein M, Swanson SA, Avenevoli S, Cui L, et al. Lifetime prevalence of mental disorders in U.S. adolescents: results from the National Comorbidity Survey Replication–Adolescent Supplement (NCS-A). J Am Acad Child Adolesc Psychiatry. (2010) 49:980–9. doi: 10.1016/j.jaac.2010.05.017

Crossref Full Text | Google Scholar

88. Ormel J, Raven D, van Oort F, Hartman CA, Reijneveld SA, Veenstra R, et al. Mental health in Dutch adolescents: a TRAILS report on prevalence, severity, age of onset, continuity and co-morbidity of DSM disorders. Psychol Med. (2015) 45:345–60. doi: 10.1017/S0033291714001469

PubMed Abstract | Crossref Full Text | Google Scholar