Long-term Results from the Empowering a Multimodal Pathway Toward Healthy Youth Program, a Multimodal School-Based Approach, Show Marked Reductions in Suicidality, Depression, and Anxiety in 6,227 Students in Grades 6–12 (Aged 11–18)

Here, we report on findings from a 15-month follow-up of a school-based program called Empowering a Multimodal Pathway Toward Healthy Youth (EMPATHY). This was primarily intended to reduce suicidal thinking in pre-teens, adolescents, and youth students aged 11–18 in middle schools (Grades 6–8) and high SCHOOLS (Grades 9–12). It also aimed to reduce depression and anxiety. The EMPATHY multimodal program consisted of repeated data collection, identification of a high-risk group, a rapid intervention for this high-risk group including offering supervised online cognitive behavioral therapy (CBT) program, a universal CBT intervention for those in Grades 6–8, a variety of interactions with trained staff (“Resiliency Coaches”), and referral to external medical and psychiatric services where appropriate. There were four time-points at which assessments were made: baseline, 3, 7, and 15 months. Here, we report cross-sectional findings over 15 months in a total of 6,227 students who were assessed at least once during the study period. Additionally, we report longitudinal findings from the 1,884 students who completed all 4 assessments. Our results found highly statistically significant decreases in suicidality rates, with the percentage of the total school population who were actively suicidal decreasing from 4.4% at baseline (n = 143 of 3,244) to 2.8% at 15 months (n = 125 of 4,496) (p < 0.001). There were also highly statistically significant reductions in depression and anxiety scores at each time-point. Thus, Mean Depression scores at baseline for the entire student population were 3.73 ± 3.87 (n = 3,244) at baseline and decreased to 3.22 ± 3.52 (n = 4,496) (p < 0.001). Since most students were not depressed, whole population changes such as this may indicate impact in many areas. In the longitudinal analysis of students who completed all four assessments, there were also highly statistically significant improvements in depression and anxiety scores at all time-points. For example, depression scores decreased from a mean of 3.43 ± 3.67 (n = 1,884) at baseline to 2.95 ± 3.53 (n = 1,884) at 15-months (p < 0.001), while the number who were actively suicidal decreased from 69 to 37. These results suggest that school-based multimodal programs, utilizing a combination of interventions, can have meaningful benefits across entire school populations.

Introduction

It is widely recognized that depression is common in those aged 11–17, with up to 10% of this group meeting criteria for depression, with diagnostic rates possibly increasing (1–3). Those within this age range are referred to, depending upon their exact age, as children, “pre-teen” children, adolescents, or youth (particularly those aged at least 13). This age range (from 11 to 17) is also the time at which many psychiatric disorders first appear (4). Tragically, suicide rates are also higher in this group (5, 6), with two large studies suggesting that 4–7% of pre-teens, adolescents, and youth have made at least one suicide attempts in the previous 12 months (7, 8). Those who have the highest risk for subsequent suicide appear to be pre-teens, adolescents, and youth students aged 11–18 who have previously harmed themselves and/or who have depression (9).

Addressing this issue is therefore critical, and there are several approaches that may help based on cognitive behavioral therapy (CBT) interventions primarily, although other approaches have been found to be helpful as well (10–13). Nonetheless, the research evidence to date has not clarified if it is more effective to target entire populations (so-called “universal” interventions) (14–17), or whether it is more effective to identify and then intervene in a smaller “high-risk” group (18–21). Overall, research findings and reviews have been supportive of a variety of both high-risk and universal interventions potentially reducing suicide rates in those aged 11–18, without definitively determining if one approach has better outcomes (22–27). For this reason, some have suggested that the most effective method to address depression and suicide in pre-teens, adolescents, and youth is to combine both universal programs as well as screening for those at highest risk, followed by targeted interventions (5, 28–35). It is also accepted that schools are the most appropriate setting to screen and intervene for those aged 11–18, including to increase resiliency against both depression and suicidality (17, 36–39). Therefore, combining both universal and high-risk approaches in schools may offer the potential to offer the most positive outcomes (40, 41). Supporting such an approach, a recent review and meta-analysis concluded that future “refinement of school-based prevention programs has the potential to reduce mental health burden and advance public health outcomes” (28).

We worked with a school district in Alberta, Canada, to help them design a new program to be given during designated “health” classroom time, with the intention being to try and improve resiliency against depression and to reduce suicidal thinking in their students. This was of particular relevance to them since this school district had experienced a sharp increase in the number of youth suicides. We suggested a multimodal approach to potentially best address this issue, including regular assessments of students’ progress and thinking, including the potential for subsequent clinical interventions and referral (following informed consent). Studying complex interventions is problematic and the present study is neither randomized nor controlled, which would have been unethical when studying youth who are suicidal, but instead uses an interrupted time series design which may be appropriate for youth participation (40). This entire approach we termed Empowering a Multimodal Pathway Toward Healthy Youth (EMPATHY) (41). Previously, we have reported initial 3-month outcomes from the EMPATHY program, which found a highly significant short-term reductions in suicidality combined with improvements in both depression and anxiety across the entire school district population (41). Here, we report on the longer term outcomes in a larger group of students that were took part in this school-based EMPATHY program over a 15-month period, focusing on the key outcomes of suicidal thinking, depression, and outcome.

Materials and Methods

Program Location and Timing

The EMPATHY program took place in Red Deer, AB, Canada, a small city with a population of approximately 100,000 people. Sadly, in 2012 and early 2013, there had been a significant number of youth suicides in students at Red Deer Public Schools, and the community was looking for alternative approaches to try and reduce future student risks for this. We engaged with Red Deer Public Schools in May of 2013 and a widespread multi-sectoral community partnership approach was implemented involving education services, primary health care, specialist mental health care, social services, as well as others involved with youth (such as the police services). Following extensive consultation with large numbers of interested parties, researchers, and community groups, implementation of this new program (which we refer to as the EMPATHY program) started across the entire school district in February 2014, with final data being collected in June 2015 as program funding was terminated suddenly and unexpectedly by the newly elected Provincial Government. Here, we present the longer term outcomes, which consisted of data collected by the school district at four separate assessment time-points from February 2014 until June 2015. These were baseline (February/March 2014), at 3 months (May/June 2014), at 7 months (September/October 2014), and at 15 months (April/May 2015). Because of the unexpected and sudden termination determination of other, previously planned, outcomes of potential interest are not available. These included possible changes in educational attendance and educational outcomes, as well as longer term follow-up for mood and suicidality, are not available.

The program was carried out in all of the nine schools educating those aged 11–18 (Grades 6–12) located within Red Deer Public School district. Schools involved were three middle schools for those aged 11–14 (Grades 6–8); three schools that had a wider range of grades in the school buildings, but only those in Grades 6–8 at these schools were included in the study; one special school for those aged 15–18 (Grades 9–12); and two high schools for those aged 15–18 (Grades 9–12) (Table 1).

TABLE 1

Table 1. Number of students screened for depression in each grade at each time-point.

Assessment Tools

For depression, the EMPATHY program utilized questions from the 9-item patient health questionnaire (PHQ-9) (42), adapted for adolescents (PHQ-A), which has been well validated in youth (43–45), but which were very slightly modified from the original publications (Table 2). Suicidal risk was assessed using the questions in the PHQ-A with two separate questions: “over the past 2 weeks how often have you been bothered by thoughts of hurting yourself”; and “over the past 2 weeks how often have you been bothered by thoughts that you would be better off dead” (41) (Table 2). If a student answered positively to the first question, then two other subsequent questions were asked: “Has there been a time in the past month when you have had serious thoughts about ending your life?”; and “Have you ever, in your WHOLE LIFE, tried to kill yourself or made a suicide attempt.” These questions were asked since previous research has indicated that positive answers may indicate a higher future risk for future self-harm (9, 46).

TABLE 2

Table 2. List of questions asked.

To assess the degree of anxiety, questions were included from the anxiety section of the Hospital Anxiety and Depression Scale (47) (Table 2), as these have previously been used to measure anxiety in youth in several studies (48–50).

Multimodal Components Included in the EMPATHY Program

The multimodal program consisted of repeated data collection, identification of a high-risk group, a rapid intervention for this high-risk group including offering supervised online CBT program, a universal CBT intervention for those in Grades 6–8, non-specific interactions with Resiliency Coaches, and referral to external medical and psychiatric services where appropriate (41).

Data Collection

All data collection was carried out on dedicated electronic tablets within a 25-min period occurring during a standard classroom health lesson. Students logged on using only their student IDs. Electronic data collection complied with all privacy and security requirements. Questions were presented to students in a randomized order, and no data were stored on the tablets as they were directly linked to the school intranet. The data were stored in a dedicated and secure database within the Red Deer Public School system, in the same manner as all other confidential student information. Because assessments were collected electronically, it was possible for school staff to identify those potentially as “at risk” based on their scores. It should be noted that students were identifiable to school staff only by a unique study number assigned when assessments occurred, and only if the student was flagged would the school staff be able to determine the student identity. This was also the only time that information about individual student results and scores was available to school staff. Apart from these specific instances, information about individual student results and scores was not available to school staff.

All students completed questionnaires for five separate areas of interest: (1) depression (including questions on suicidal thinking); (2) anxiety; (3) use of drugs, alcohol, and tobacco; (4) self-esteem; and (5) quality of life. A measure combining all of these measurements, the so-called “EMPATHY scale,” was also captured. Measurement occurred at Baseline (prior to any other interventions), and was repeated at 3, 7, and 15 months. In the present publication, only the data on suicidal thinking (“suicidality”), depression, and anxiety are considered. Other data will be published elsewhere subsequently.

Identification of the “Actively Suicidal” Group and “High-Risk” Group

After each assessment, the results were rapidly available. From the responses to the questions on potential self-harm, students were placed in a category of suicide risk being either (i) none, (ii) low, (iii) medium, or (iv) high suicide risk (Figure 1). Those considered at “Higher risk” (i.e., those in the (iii) medium suicide risk group or (iv) high suicide risk group based on their scores) were identified to school staff. It should be noted that a priori it had been agreed that any student who scores indicated they were either in the high suicide risk group or the medium suicide group would be considered “actively suicidal.” Those students who were deemed “actively suicidal,” as well as the group with the highest scores on the depression and anxiety rating scales, were considered the “High Risk” group.

FIGURE 1

Figure 1. Algorithm for determining which students were in the “actively suicidal” group. Shows how scores on the questions regarding suicide risk determined if the student was in the high suicide risk group or the medium suicide risk group. Together, these students were considered the “actively suicidal" group, and were interviewed individually within 2 days in most cases, and everybody had been interviewed within 5 days of completing the questionnaire.

Rapid Intervention and Focused CBT Intervention

Students who were identified as belonging to the “High Risk” group were interviewed in school hours and their families were subsequently contacted, usually within 2 days of data completion (in 83% of cases). All others in this group were interviewed and families were contacted within 5 days. This interview was performed by a trained member of staff (usually somebody familiar to the student). This took the form of a 1-h semi-structured interview, during which there was a more detailed assessment of suicide risk (enhanced by previous training of school staff). There was also an open-ended discussion of issues that were relevant to the individual student. Following this interview, the student’s parents (or guardian) was contacted by school staff and informed that there were concerns. At this time, a plan was agreed upon with the parent (or guardian), which could include referral to Emergency Room, specialized Mental Health services, or their primary care physicians.

This group was also provided with information regarding the possibility of taking part in guided CBT approaches that have previously been recognized as clinically effective in this age group (51–54). However, only those students whose parents signed an informed consent form, and where the student also completed an assent form (as this was not part of the regular school program), had access to this additional program. It was also emphasized to both the student and parent that subsequent support and treatment was independent of whether or not the student took part in any subsequent CBT study.

Universal Intervention—OVK

The second intervention was only for students in Grades 6–8 (mean ages 11.3 at baseline in Grade 6, 12.3 in Grade 7, and 13.3 in Grade 8). This was a “Universal” intervention, since all students in these grades received this CBT-based interactive program designed to reduce rates of depression, which was given during regular classroom time. This program is an updated version of the Penn Resiliency Program (55–58) which was modified and used in the Netherlands, and it is referred to by its Dutch initials (“OVK”) (59–62). The school district translated and modified this program to better suit their students (in collaboration with the authors of OVK), and it was administered by the Resiliency Coaches after appropriate training (41). However, it should be noted that in the first year it was given as a limited version consisting of the initial 8 CBT sessions, while in the second year, all students were given a full set of 16 sessions, the initial 8 CBT sessions followed by the second 8 sessions focused on social and educational learning.

Resiliency Coaches

All staff hired for the EMPATHY program had experience working with youth but were deliberately chosen not to be highly qualified individuals (thus excluding psychologists, nurses, or teachers for example). This was to determine if the EMPATHY program could be successful with staff who could potentially allow the program to scale up, recognizing the potential shortage (and cost) of more highly trained individuals. These individuals were termed “Resiliency Coaches,” and each was attached to a specific school, but were not therapists or counselors, and did not act in those roles. They carried out all screening, supervised CBT programs and were available during unstructured student time. As such, the resiliency coaches were encouraged to be active and available during many periods when they could have spontaneous interactions with any student that wanted this. This occurred during breaks, play time, in addition to scheduled sessions. There was no formal measurement of the amount of interventions or interactions, but Resiliency Coaches were highly visible resources available to all students to access if they chose.

Referral to External Medical and Psychiatric Services

We worked closely with local services for both family practice and specialist psychiatric care. This included dedicated training on diagnosis and treatment approaches for community physicians and mental health staff working in primary care. We also provided specific training on CBT and other treatment approaches for youth. We worked to support the local mental health providers to ensure that if a large number of students were identified as needing support, appropriate resources would be available to help treat individuals identified by this program. During the study, careful tracking of all referrals to both primary care and specialist mental health care was carried out. In fact, in the first phase of the program only 60 students (2% of total screened) required external referral during the first 24 weeks (41).

Statistical Analysis

Since the data showed evidence of non-normality, non-parametric tests were carried out to compare the differences between the median scores at baseline and each follow-up. In all tests we used a significance level of α = 0.05. For the statistical tests on all students screened, we used Wilcoxon rank-sum tests to test the equality of medians from two independent groups. For the statistical tests on students who completed all 4 ratings, we used Wilcoxon signed-rank tests (paired). Here, we assume the sample is a paired design, in which each student who completed both baseline ratings and follow-up ratings was their own control. A statistical power analysis had been completed prior to our initial study (41), which determined that the study was adequately powered.

Statistical analysis was carried out on an “intention to treat” basis utilizing R, version 3.1.0 (R Foundation for Statistical Computing, Vienna, Austria) and Stata/IC 13.1 for Windows (StataCorp, College Station, TX, USA). Correlations were calculated using IBM SPSS Statistics 20.0 (Chicago, IL, USA).

Note that although individuals measurements of self-esteem, quality of life, and use of drugs, alcohol, and tobacco, were made, as was a summary scale, all of these issues will be considered separately in other publications.

Results

Assessment #1 (Baseline) occurred during February and March 2014 (n = 3,244); Assessment #2 was the 3-month follow-up screening which occurred during May and June 2014 (n = 3,229); Assessment #3 was the 7-month follow-up screening which occurred during September and October 2014 (n = 4,860); and Assessment #4, the final 15-month follow-up assessment, occurred during the period April to June 2015 (n = 4,496). The timing of the follow-up screenings were based around the school year, with Assessments #1 and #2 in school year 1, and Assessments #3 and #4 occurring in school year 2, with all schools being on vacation in July and August 2014. Note that only five schools took part in school year 1, the three middle schools and two high schools. Of these students, a total of 6,227 fully completed at least one assessment, 4,917 completed at least two assessments, 2,796 completed at least three assessments, while 1,884 completed all four ratings (Table 1; Figure 2). There were an additional 324 students who signed in but then did not complete any assessments and were not therefore included in the analysis. Thus, a total of 6,651 students potentially were involved, but we have data on only 6,227. Of the total group of 6,651 students, 2,121 identified as males (32%), 2,273 identified as females (34%), while 2,257 students (34%) declined to identify their gender. For this reason, we did not analyze compare male and female data.

FIGURE 2

Figure 2. Flowchart demonstrating assessment completion by students. Shows how many students completed an assessment at baseline, 3, 7, and 15 months (on the left-hand side) as well as how many students completed multiple assessments (on the right-hand side).

Of the 1,884 students who completed all four assessments over both years, this group consisted only of students in Grades 6–11 from the first school year. This is because they could not be in either Grade 6 in school year 2 (as they would be new to the system) or Grade 12 in school year 1 (as they would have graduated before school year 2). The number of students who completed Depression ratings for each grade at each time-point is shown in Table 1, and it can be seen that there were more participants in every grade in the second year as more schools were involved.

Depression and Anxiety Scores in Total Study Population

At baseline, the scores for the total of 3,244 students who completed the depression and anxiety scales varied by age (or Grade) as we have previously shown (41). However, in the present study, we found a highly significant decrease in suicidal thinking, both in terms of the percentage who were actively suicidal (either high suicide risk or medium suicide risk) (Figure 3) as well as those who had a higher suicide risk (Table 3). This may suggest that the multimodal EMPATHY decreased the risk of suicide, although data regarding actual suicide rates in the region are not available at this time-point. Additionally, scores for all students for both depression and anxiety significantly decrease over time, even though the vast majority were not depressed or anxious at baseline (Table 4). This may suggest that the EMPATHY program had a beneficial effect on the entire school population.

FIGURE 3

Figure 3. Percentage of students deemed “Actively Suicidal” at each assessment. Shows the percentage of students who were deemed “actively suicidal” according to defined criteria at each assessment. Data are shown both for cross-sectional group of 6,227 who completed at least one of the 4 assessments (Baseline n = 3,244; 3-month assessment n = 3,228; 7-month assessment n = 4,856; and 15-month assessment n = 4,496), as well as for the 1,884 students who completed all 4 assessments and consist of the longitudinal group. It can be seen that there are highly significant differences from baseline assessments in both groups at all subsequent assessments (***p < 0.001).

TABLE 3

Table 3. Changes in suicidality for entire study population.

TABLE 4

Table 4. Changes in Depression and Anxiety Scores for entire study population.

Suicidality, Depression, and Anxiety Scores in Those Who Completed All Four Ratings

A total of 1,884 students completed all 4 assessments and therefore constitute a longitudinal follow-up group (Tables 5 and 6). As with the total study population, the scores for depression and anxiety significantly decreased at each time-point compared to baseline (Figure 3), as well as having highly significant decreases in the percentage of those who were actively suicidal. Thus, there was a decrease of nearly 50% in the number of individuals who were actively suicidal compared to baseline, from 69 to 37 (Table 5).

TABLE 5

Table 5. Changes in suicidality for study population of 1,884 who completed all 4 ratings.

TABLE 6

Table 6. Changes in Depression and Anxiety Scores for study population of 1,884 who completed all 4 ratings.

Discussion

The present results demonstrate that a complex multimodal intervention can impact entire school populations in a positive manner. The results build on the initial 3-month findings we have previously reported (41) and demonstrate that there can be significant longer term improvements in the rates of suicidality, depression, and anxiety across large school populations. Given that the majority of students did not have depression or anxiety, the ability to demonstrate such changes on a school-wide basis suggests that these are meaningful. Supporting this suggestion was the finding that the percentage of students who were actively suicidal decreased from 4.4% of the entire school population at baseline to 2.8% when the EMPATHY program ended. This positive improvement occurred despite the fact that most students received no direct interventions, i.e., those in Grades 8–12 did not receive resiliency training, and most students in all Grades did not have guided Internet CBT interventions.

It is possible that several components that may have contributed to these findings. First, since students received rapid feedback after competing questionnaires, with interviews in most cases being less than 48 h in most case, this may have been impactful. Anecdotal feedback from several students was that this was seen as very positive, as the rapid feedback made them feel that their issues were taken seriously, an idea which has previously been reported (63). Again, anecdotal feedback suggested that friends of the individuals identified also gained positively as they now were aware of additional support. Additionally, involvement of parents may also have been a contributing factor (64, 65). In this instance, there was extensive anecdotal feedback from parents who were extremely positive about how this process helped their understanding of the issues their pre-teen, child, or youth was experiencing. The impact of other, somewhat similar, programs on school cultures has been reported previously (66–68). However, determining if this occurred would have required specific qualitative study which was not carried out.

An additional possibility is that improvements were due, in part, to Resiliency Coaches spending a lot of recreational time in the same environment as the students and, while they were specifically trained to not act as “therapists,” they found that on many occasions students would spontaneously approach them and sometimes confide in them. This included disclosures regarding physical and/or sexual abuse they had experienced or witnessed. The impact of this is uncertain, but this is a different role to that held by specific school counselors, and the Resiliency Coaches were surprised by how frequently their availability allowed such spontaneous interactions to occur. It is recognized that friendship skills can help adolescent depressive symptoms in combination with other approaches (69), so it is possible this may also have played a role. Currently, however, these possibilities all remain speculative. In future studies, it will be important to examine them more specifically to determine what, if any, impact each of these may have.

In terms of the potential benefit for a universal CBT intervention (OVK), this remains uncertain. Given that in the present study, it was used in a different and truncated manner in the first year and administered by a different group of individuals than those who have previously been shown to deliver it effectively, it cannot be considered to have been adequately tested in the present study. That universal interventions may be more effective when delivered by psychologists has been suggested from other studies (70) means that we are uncertain what our negative findings may indicate. Determining the relative importance of this aspect of the program will be required in future studies, and most importantly, it is necessary to study the effectiveness of the full 16-session program. Nonetheless, it is well recognized that it can be difficult to detect the overall effect of such universal interventions, and this may explain the large number of studies that are unable to show a benefit. The potential benefit of universal approaches such as OVK, as well as those recently examined for suicide prevention (27, 33), needs to be evaluated as part of the approach of combination programs such as EMPATHY, but are unlikely to be measures that on their own significantly reduce youth depression and suicide rates.

In terms of the measurement scales used, it is important to recognize that simply using depression rating scales on their own may not be sufficient. This is because many pre-teens, children, and youth aged 11–18 have “sub-threshold” depression (i.e., depressive symptoms that do not meet diagnostic criteria) (71, 72), and many of these also have active suicidal thoughts (8). For these reasons, and the fact that scales for depression do not examine previous self-harm behaviors, most standard depression screening tools likely miss many youth who have active suicidal thoughts (73). Nonetheless, it does appear that the questions contained within the PHQ-9 may be very useful as part of a screening tool (74).

A further advantage of the present findings was that we could examine findings from two different groups: first those who completed assessments at least once (n = 6,227), giving cross-sectional data, and second the smaller group of 1,884 students who completed all 4 assessments and therefore provided longitudinal data. Utilizing findings from both groups increases our ability to have confidence in the findings since they both had similar outcomes.

An additional, anecdotal report was that many of the teachers and school Principals reported a marked decrease in the number of bullying cases brought to their attention. Although the schools did not formally collect specific data to measure this suggestion, these findings are compatible with our previous suggestions that the positive changes found initially at 3 months (41) (and in the present study) may have been due to a combination of several interventions, as well as non-specific interactions. Further research is needed to validate any such suggestions, and if so to more accurately determine whether any specific interventions, on their own, can have similar impacts across a school district.

When comparing the present findings to those from previous research it is important to note that we included all students within a complete school district. There was no exclusion of particular schools or groups of students, both of can conceivably bias any findings. This is also different in that here we have analyzed data collected as part of a multimodal program introduced on a district-wide basis to all schools, rather than a research study in which single, or combined, factors are compared. However, such an approach leads to the significant limitation that the lack of any control groups limits conclusions that may otherwise be made. Nonetheless, since one of the challenges facing those who wish to carry out research to improve the mental state of youth is the fact that students move between schools, only by examining depression rates or suicidal thinking across entire student populations can more accurate conclusions be made.

Another potential limitation is that the EMPATHY program took place during two different school years, and it was therefore conceivable that there might have been timing effects that impacted the outcomes. Thus, there could have been a major change only in school year 1 and not school year 2, in which case there should be no improvement in scores between Assessments #2 and Assessments #3. Or, alternatively, students may improved at Assessment #2 simply because of they were close to the summer vacation but then got worse when they returned (Assessment #3), only to improve again and have good results at the end of the second school year (Assessment #4). However, the results showed no such impacts, and in particular when we examined changes between the end of the first school year and the start of the second school year (Assessments #2 and #3, respectively), we found ongoing statistically significant improvements between these two visits for depression scores. This suggests that the EMPATHY program had long-lasting benefits which continued over time, and further might imply that the more interactions the students had, the greater the benefits. This was also suggested by the findings from the longitudinal group of 1,884 students. Both of these findings suggest no strong seasonal component underlay the current findings.

Other study limitations are that children, pre-teens, and youth all develop cognitively over time. However, we had no measurements either at baseline or at 15 months regarding how they may have possible changed during the 15-month follow-up period. It is conceivable that this may have, in some non-specified manner, altered the findings. We think this unlikely, given the consistency across grades, but we cannot be certain this had no impact. For example, all the 1,884 children and youth who were followed-up repeatedly were 15 months older than at baseline age; therefore, their ability to respond to questions and ease of handling computerized assessments most probably had improved. Similarly, we have no information regarding the general cognitive ability of study participants. In any classroom cohort, there are be adolescents with learning problems, or intellectual difficulties, but we have no information about individual students and whether this impacted outcomes, and no information about gender and age dispersion in classrooms or across grades, or cognitive ability at baseline regarding readability or understanding of questions.

Our inability to discuss the data in terms of possible gender differences is also a limitation. The large number of individuals who declined to identify their gender may have been, in part, a response to the fact that there were only two choices. Clearly, future youth surveys should give greater choice to individuals as to how they choose to identify their gender, and this may increase compliance rates.

In conclusion, the need for longer term, effective programs that can reduce youth suicidality and depression remains clear (5), and although a recent review showed that many programs do have some positive benefits (28), we believe the results from the present EMPATHY study program suggest that multimodal approaches may lead to significant mental health benefits over a longer time period. The present results from this program give support to the utility of such multimodal interventions, including combining both universal and targeted approaches, although controlled studies are needed before widespread implementation can be recommended. Initially, we had planned further studies, in part to determine reproducibility both in similar and different school populations. Unfortunately, although this was initially supported, with the unexpected ending of funding it was not possible to do this. However, we strongly encourage others to do this as implementation of any major programs in youth should only be instituted after rigorous research demonstrating reproducibility.

Ethics Statement

This program was approved by the Health Research Ethics Committee of the University of Alberta on December 5, 2013, ethics protocol number Pro00041063. Amendments to the original protocol and consent letters were approved in January of 2014, with all changes to the informed consent letter subsequently approved prior to the start of the program. The first student was screened in February 2014 and the follow-up screening was completed by June 2015. No additional student follow-up is possible, since program funding was terminated unexpectedly in June 2015 following a change in Provincial Government in Alberta in May 2015. This study is registered with http://ClinicalTrials.gov identifier: NCT02169960. Although an application for this was completed at the time of the ethics approval, due to an unfortunate administrative oversight, the actual submission to the registration database did not occur until July 2014. This omission was corrected as soon as it was recognized and was noted in our original publication (41). It should be noted upon consultation with the school district (and with the agreement of the Health Ethics Board of the University of Alberta, the supervisory ethics board), the screening process was deemed not part of the study since the school district was adamant that programs that they implement school district-wide did not need specific parental consent. This was based on the fact that education for any subject matter within these schools, even areas that may be considered controversial (such as discussions about sexual matters or sexual orientation, or ethical matters such as physician-assisted suicide) do not require parental consent. Thus, they did not believe that teaching students on a universal, district-wide basis about mental health issues required a different process than any other subject. Similarly, they asserted that assessing students on a range of topics is done frequently, again, on a universal, district-wide basis, and the assessments in the screening were not something for which specific consent was required. Indeed, they felt that requesting specific consent for what was taught or collected as part of class would be antithetical to their processes. Per their normal processes, they sent an information letter to all parents about the new program, thereby informing the parents/guardians that the school district was introducing a school district-wide “Resiliency Project” (they did not want to use the term “EMPATHY program” as they had recently introduced a different program that had a similar name). As noted, this approach was specifically agreed by the University of Alberta Health Ethics Review Board. The University of Alberta, as well as the Strategic Clinical Network for Addiction and Mental Health of Alberta Health Services, helped design the most appropriate screening tool and intervention program, train teachers about assessment, provide close liaison to clinical services in the region when required, and evaluate the effectiveness of the program. In discussions, the school set a threshold to trigger invitation to the targeted intervention, and this threshold varied by age and by school. This component included the option of taking part in a guided Internet-based CBT program. Since participation in this additional CBT program was not part of regular teaching, prior to receiving any such additional Internet-based CBT, the parents or guardians of the students provided written consent on behalf of the youth. Additionally, the student provided written assent, and all documents and the process were approved by the University of Alberta Health Ethics Review Board. The signed consent and assent forms were kept as part of the trial documentation, in a secure, private location. The data collected by the school as part of their regular classroom assessment were made available for analysis by the research team, but only on an anonymized basis. From a scientific viewpoint, the most rigorous findings would have come from a study in which the design included a group of students who were randomized to one group or another, one of which would have been to not have any intervention. Indeed, such an approach has been carried out in studies looking at depression rates in students and in theory randomization can be carried at the class level or school level. However, this was a school district-wide program that was specifically focused on decreasing the number of youth who were actively suicidal, with assessments of students to measure this. To have considered a program (or study) in which a group of students was assessed, found to be suicidal, but then not offered an intervention would clearly have been unethical. For this reason, we determined that the most appropriate methods to analyze the data collected by the school district was by examining data from both a cross-sectional analysis of all students who completed at least one assessment, as well as using a longitudinal analysis for those students who completed all four assessments (where we would be using each student as their own control). This approach to analysis of the data was approved by the Health Research Ethics Committee, with subsequent modification in April 2016 to approve secondary analysis of the information obtained from the EMPATHY program (presented here in the present publication) with the ethics protocol number Pro00061164.

Author Contributions

PS, MB, VS, AA, JG, SH, CP, PL, KR, and RE were responsible for the design of the program. IC, SC, and CM were responsible for all statistical analysis, assisted by VS and KR.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

This study would not have been possible without the assistance of several individuals at the start of this study (many of whom have now moved positions). These include those in the Red Deer community and in Red Deer Public Schools (Elizabeth Fargey and Phil Penner), in Alberta Health Services (Michael Trew, Dwight Hunks, Noreen McCallum, Marti Rustulka, Tanya Drescher, and Chris Wilkes), in the Red Deer Primary Care Network (Lorna Milkovich), at the Central Alberta Child and Family Services Authority (David Tunney), and within Justice and Solicitor General Community Corrections (Kim Fay). The authors also want to thank Rowella Kuijpers and Karlijn Kindt for training the study team in the correct methodology for administration of the OVK program.

Funding

Funding was provided by Alberta Health. The authors also had some institutional support in Alberta including that of the Science, Policy Practice Network, a multi-sectoral partnership supported in part by the Norlien Foundation of Alberta. In-kind support was also provided by Red Deer Public Schools and Alberta Health Services.

References

1. Perou R, Bitsko RH, Blumberg SJ, Pastor P, Ghandour RM, Gfroerer JC, et al. Mental health surveillance among children-United States, 2005-2011. MMWR Surveill Summ (2013) 62(Suppl 2):1–35.

Google Scholar

2. Nardi B, Francesconi G, Catena-Dell’osso M, Bellantuono C. Adolescent depression: clinical features and therapeutic strategies. Eur Rev Med Pharmacol Sci (2013) 17:1546–51.

PubMed Abstract | Google Scholar

3. Bor W, Dean AJ, Najman J, Hayatbakhsh R. Are child and adolescent mental health problems increasing in the 21st century? A systematic review. Aust N Z J Psychiatry (2014) 48:606–16. doi:10.1177/0004867414533834

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Fichter MM, Kohlboeck G, Quadflieg N, Wyschkon A, Esser G. From childhood to adult age: 18-year longitudinal results and prediction of the course of mental disorders in the community. Soc Psychiatry Psychiatr Epidemiol (2009) 44:792–803. doi:10.1007/s00127-009-0501-y

CrossRef Full Text | Google Scholar

5. Hawton K, Saunders KE, O’Connor RC. Self-harm and suicide in adolescents. Lancet (2012) 379:2373–82. doi:10.1016/S0140-6736(12)60322-5

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Clifford AC, Doran CM, Tsey K. A systematic review of suicide prevention interventions targeting indigenous peoples in Australia, United States, Canada and New Zealand. BMC Public Health (2013) 13:463. doi:10.1186/1471-2458-13-463

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Eaton DK, Kann L, Kinchen S, Shanklin S, Flint K, Hawkins J, et al. Youth Risk Behavior Surveillance—United States, 2011 (Vol. 61). Centers for Disease Control and Prevention MMWR, Surveillance Summaries (2012). p. SS–4. Available: http://www.cdc.gov/mmwr/pdf/ss/ss6104.pdf

Google Scholar

8. Carli V, Hoven CW, Wasserman C, Chiesa F, Guffanti G, Sarchiapone M, et al. A newly identified group of adolescents at “invisible” risk for psychopathology and suicidal behavior: findings from the SEYLE study. World Psychiatry (2014) 13:78–86. doi:10.1002/wps.20088

CrossRef Full Text | Google Scholar

9. Beghi M, Rosenbaum JF. Risk factors for fatal and nonfatal repetition of suicide attempt: a critical appraisal. Curr Opin Psychiatry (2010) 23:349–55. doi:10.1097/YCO.0b013e32833ad783

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Weisz JR, Kuppens S, Ng MY, Eckshtain D, Ugueto AM, Vaughn-Coaxum R, et al. What five decades of research tells us about the effects of youth psychological therapy: a multilevel meta-analysis and implications for science and practice. Am Psychol (2017) 72:79–117. doi:10.1037/a0040360

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Devenish B, Berk L, Lewis AJ. The treatment of suicidality in adolescents by psychosocial interventions for depression: a systematic literature review. Aust N Z J Psychiatry (2016) 50:726–40. doi:10.1177/0004867415627374

CrossRef Full Text | Google Scholar

12. Takagaki K, Okamoto Y, Jinnin R, Mori A, Nishiyama Y, Yamamura T, et al. Behavioral activation for late adolescents with subthreshold depression: a randomized controlled trial. Eur Child Adolesc Psychiatry (2016) 25:1171–82. doi:10.1007/s00787-016-0842-5

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Henje-Blom E, Tymofiyeva O, Chesney MA, Ho TC, Moran P, Connolly CG, et al. Feasibility and Preliminary Efficacy of a Novel RDoC-Based Treatment Program for adolescent depression: "Training for Awareness Resilience and Action" (TARA)-A Pilot Study. Front Psychiatry (2017) 7:208. doi:10.3389/fpsyt.2016.00208

CrossRef Full Text | Google Scholar

14. Ahlen J, Lenhard F, Ghaderi A. Universal prevention for anxiety and depressive symptoms in children: a meta-analysis of randomized and cluster-randomized trials. J Prim Prev (2015) 36:387–403. doi:10.1007/s10935-015-0405-4

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Merry SN, Hetrick SE, Cox GR, Brudevold-Iversen T, Bir JJ, McDowell H. Psychological and educational interventions for preventing depression in children and adolescents. Cochrane Database Syst Rev (2011) (12):CD003380. doi:10.1002/14651858.CD003380.pub3

CrossRef Full Text | Google Scholar

16. van Zoonen K, Buntrock C, Ebert DD, Smit F, Reynolds CF III, Beekman AT, et al. Preventing the onset of major depressive disorder: a meta-analytic review of psychological interventions. Int J Epidemiol (2014) 43:318–29. doi:10.1093/ije/dyt175

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Durlak JA, Weissberg RP, Dymnicki AB, Taylor RD, Schellinger KB. The impact of enhancing students’ social and emotional learning: a meta-analysis of school-based universal interventions. Child Dev (2011) 82:405–32. doi:10.1111/j.1467-8624.2010.01564.x

CrossRef Full Text | Google Scholar

18. O’Leary-Barrett M, Topper L, Al-Khudhairy N, Pihl RO, Castellanos-Ryan N, Mackie CJ, et al. Two-year impact of personality-targeted, teacher-delivered interventions on youth internalizing and externalizing problems: a cluster-randomized trial. J Am Acad Child Adolesc Psychiatry (2013) 52:911–20. doi:10.1016/j.jaac.2013.05.020

CrossRef Full Text | Google Scholar

19. Bond L, Butler H, Thomas L, Carlin J, Glover S, Bowes G, et al. Social and school connectedness in early secondary school as predictors of late teenage substance use, mental health, and academic outcomes. J Adolesc Health (2007) 40(357):e9–18. doi:10.1016/j.jadohealth.2006.10.013

CrossRef Full Text | Google Scholar

20. Wei Y, Kutcher S. International school mental health: global approaches, global challenges, and global opportunities. Child Adolesc Psychiatr Clin N Am (2012) 21:11–27. doi:10.1016/j.chc.2011.09.005

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Dever BV, Kamphaus RW, Dowdy E, Raines TC, Distefano C. Surveillance of middle and high school mental health risk by student self-report screener. West J Emerg Med (2013) 14:384–90. doi:10.5811/westjem.2013.2.15349

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Garber J, Brunwasser SM, Zerr AA, Schwartz KT, Sova K, Weersing VR. Treatment and prevention of depression and anxiety in youth: test of cross-over effects. Depress Anxiety (2016) 33:939–59. doi:10.1002/da.22519

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Pirkis J, Too LS, Spittal MJ, Krysinska K, Robinson J, Cheung YT. Interventions to reduce suicides at suicide hotspots: a systematic review and meta-analysis. Lancet Psychiatry (2015) 2:994–1001. doi:10.1016/S2215-0366(15)00266-7

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Di Napoli WA, Della Rosa A. Suicide and attempted suicide: epidemiological surveillance as a crucial means of a local suicide prevention project in Trento’s Province. Psychiatr Danub (2015) 27(Suppl 1):S279–84.

PubMed Abstract | Google Scholar

25. Calear AL, Christensen H, Freeman A, Fenton K, Busby Grant J, van Spijker B, et al. A systematic review of psychosocial suicide prevention interventions for youth. Eur Child Adolesc Psychiatry (2016) 25:467–82. doi:10.1007/s00787-015-0783-4

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Wilcox HC, Wyman PA. Suicide prevention strategies for improving population health. Child Adolesc Psychiatr Clin N Am (2016) 25:219–33. doi:10.1016/j.chc.2015.12.003

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Strunk CM, King KA, Vidourek RA, Sorter MT. Effectiveness of the surviving the teens suicide prevention and depression awareness program: an impact evaluation utilizing a comparison group. Health Educ Behav (2014) 41:605–13. doi:10.1177/1090198114531774

CrossRef Full Text | Google Scholar

28. Werner-Seidler A, Perry Y, Calear AL, Newby JM, Christensen H. School-based depression and anxiety prevention programs for young people: a systematic review and meta-analysis. Clin Psychol Rev (2017) 51:30–47. doi:10.1016/j.cpr.2016.10.005

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Weare K, Nind M. Mental health promotion and problem prevention in schools: what does the evidence say? Health Promot Int (2011) 26(Suppl 1):i29–69. doi:10.1093/heapro/dar075

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Robinson J, Pirkis J. Research priorities in suicide prevention: an examination of Australian-based research 2007-11. Aust Health Rev (2014) 38:18–24. doi:10.1071/AH13058

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Teesson M, Newton NC, Slade T, Chapman C, Allsop S, Hides L, et al. The CLIMATE schools combined study: a cluster randomised controlled trial of a universal Internet-based prevention program for youth substance misuse, depression and anxiety. BMC Psychiatry (2014) 14:32. doi:10.1186/1471-244X-14-32

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Gould MD, Greenberg T, Velting DD, Shaffer DD. Youth suicide risk and preventive interventions: a review of the past 10 years. J Am Acad Child Adolesc Psychiatry (2003) 42:386–405. doi:10.1097/01.CHI.0000046821.95464.CF

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Robinson J, Hetrick SE, Martin C. Preventing suicide in young people: systematic review. Aust N Z J Psychiatry (2011) 45:3–26. doi:10.3109/00048674.2010.511147

CrossRef Full Text | Google Scholar

34. Sheffield JK, Spence SH, Rapee RM, Kowalenko N, Wignall A, Davis A, et al. Evaluation of universal, indicated, and combined cognitive-behavioral approaches to the prevention of depression among adolescents. J Consult Clin Psychol (2006) 74:66–79. doi:10.1037/0022-006X.74.1.66

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Sawyer MG, Pfeiffer S, Spence SH, Bond L, Graetz B, Kay D, et al. School-based prevention of depression: a randomised controlled study of the Beyond Blue Schools research initiative. J Child Psychol Psychiatry (2010) 51:199–209. doi:10.1111/j.1469-7610.2009.02136.x

CrossRef Full Text | Google Scholar

36. Calear AL, Christensen H. Systematic review of school-based prevention and early intervention programs for depression. J Adolesc (2010) 33:429–38. doi:10.1016/j.adolescence.2009.07.004

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Duong MT, Kelly BM, Haaland WL, Matsumiya B, Huey SJ Jr, McCarty CA. Mediators and moderators of a School-Based Cognitive-Behavioral Depression Prevention Program. Cognit Ther Res (2016t) 40:705–16. doi:10.1007/s10608-016-9780-2

CrossRef Full Text | Google Scholar

38. Dahlqvist HZ, Landstedt E, Gådin KG. What students do schools allocate to a cognitive-behavioural intervention? Characteristics of adolescent participants in Northern Sweden. Int J Circumpolar Health (2015) 74:29805. doi:10.3402/ijch.v74.29805

CrossRef Full Text | Google Scholar

39. Brière FN, Rohde P, Shaw H, Stice E. Moderators of two indicated cognitive-behavioral depression prevention approaches for adolescents in a school-based effectiveness trial. Behav Res Ther (2014) 53:55–62. doi:10.1016/j.brat.2013.12.005

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Anaby D, Lal S, Huszczynski J, Maich J, Rogers J, Law M. Interrupted time series design: a useful approach for studying interventions targeting participation. Phys Occup Ther Pediatr (2014) 34:457–70. doi:10.3109/01942638.2013.866612

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Silverstone PH, Bercov M, Suen VYM, Allen A, Goodrick J, Cribben I, et al. Initial findings from a novel school-based program, EMPATHY, which may help reduce depression and suicidality in youth. PLoS One (2015) 10:e0125527. doi:10.1371/journal.pone.0125527

CrossRef Full Text | Google Scholar

42. Spitzer RL, Kroenke K, Williams JB. Validation and utility of a self-report version of the PRIME-MD: the PHQ primary care study. Primary Care Evaluation of Mental Disorders. Patient Health Questionnaire. JAMA (1999) 282:1737–44. doi:10.1001/jama.282.18.1737

CrossRef Full Text | Google Scholar

43. Ganguly S, Samanta M, Roy P, Chatterjee S, Kaplan DW, Basu B. Patient health questionnaire-9 as an effective tool for screening of depression among Indian adolescents. J Adolesc Health (2013) 52:546–51. doi:10.1016/j.jadohealth.2012.09.012

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Richardson LP, McCauley E, McCarty CA, Grossman DC, Myaing M, Zhou C, et al. Predictors of persistence after a positive depression screen among adolescents. Pediatrics (2012) 130:e1541–8. doi:10.1542/peds.2012-0450

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Richardson LP, McCauley E, Grossman DC, McCarty CA, Richards J, Russo JE, et al. Evaluation of the patient health questionnaire-9 item for detecting major depression among adolescents. Pediatrics (2010) 126:1117–23. doi:10.1542/peds.2010-0852

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Spirito A, Esposito-Smythers C. Attempted and completed suicide in adolescence. Annu Rev Clin Psychol (2006) 2:237–66. doi:10.1146/annurev.clinpsy.2.022305.095323

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Zigmond AS, Snaith RP. The hospital anxiety and depression scale. Acta Psychiatr Scand (1983) 67:361–70. doi:10.1111/j.1600-0447.1983.tb09716.x

CrossRef Full Text | Google Scholar

48. Wiklund M, Malmgren-Olsson EB, Ohman A, Bergström E, Fjellman-Wiklund A. Subjective health complaints in older adolescents are related to perceived stress, anxiety and gender – a cross-sectional school study in Northern Sweden. BMC Public Health (2012) 12:993. doi:10.1186/1471-2458-12-993

CrossRef Full Text | Google Scholar

49. van der Geest IM, van Dorp W, Hop WC, Neggers SJ, de Vries AC, Pieters R, et al. Emotional distress in 652 Dutch very long-term survivors of childhood cancer, using the hospital anxiety and depression scale (HADS). J Pediatr Hematol Oncol (2013) 35:525–9. doi:10.1097/MPH.0b013e31829f2799

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Wen L, Jiang G, Zhang X, Lai R, Wen X. Relationship between acne and psychological burden evaluated by ASLEC and HADS Surveys in high school and college students from central China. Cell Biochem Biophys (2014) 71:1083–8. doi:10.1007/s12013-014-0312-y

CrossRef Full Text | Google Scholar

51. Elison S, Ward J, Davies G, Moody M. Implementation of computer-assisted therapy for substance misuse: a qualitative study of Breaking Free Online using Roger’s diffusion of innovation theory. Drugs Alcohol Today (2014) 14:207–18. doi:10.1108/DAT-05-2014-0025

CrossRef Full Text | Google Scholar

52. Andersson G. Using the Internet to provide cognitive behavioural therapy. Behav Res Ther (2009) 47:175–80. doi:10.1016/j.brat.2009.01.010

CrossRef Full Text | Google Scholar

53. Carrard I, Crépin C, Rouget P, Lam T, Golay A, Van der Linden M. Randomised controlled trial of a guided self-help treatment on the Internet for binge eating disorder. Behav Res Ther (2011) 49:482–91. doi:10.1016/j.brat.2011.05.004

PubMed Abstract | CrossRef Full Text | Google Scholar

54. Titov N, Andrews G, Davies M, McIntyre K, Robinson E, Solley K. Internet treatment for depression: a randomized controlled trial comparing clinician vs. technician assistance. PLoS One (2010) 5:e10939. doi:10.1371/journal.pone.0010939

PubMed Abstract | CrossRef Full Text | Google Scholar

55. Jaycox LH, Reivich KJ, Gillham J, Seligman ME. Prevention of depressive symptoms in school children. Behav Res Ther (1994) 32:801–16. doi:10.1016/0005-7967(94)90160-0

PubMed Abstract | CrossRef Full Text | Google Scholar

56. Gillham JE, Reivich KJ. Prevention of depressive symptoms in school children: a research update. Psychol Sci (1999) 10:461–2. doi:10.1111/1467-9280.00188

CrossRef Full Text | Google Scholar

57. Gillham JE, Reivich KJ, Freres DR, Chaplin TM, Shatté AJ, Samuels B, et al. School-based prevention of depressive symptoms: a randomized controlled study of the effectiveness and specificity of the Penn Resiliency Program. J Consult Clin Psychol (2007) 75:9–19. doi:10.1037/0022-006X.75.1.9

PubMed Abstract | CrossRef Full Text | Google Scholar

58. Gillham JE, Reivich KJ, Brunwasser SM, Freres DR, Chajon ND, Kash-Macdonald VM, et al. Evaluation of a group cognitive-behavioral depression prevention program for young adolescents: a randomized effectiveness trial. J Clin Child Adolesc Psychol (2012) 41:621–39. doi:10.1080/15374416.2012.706517

PubMed Abstract | CrossRef Full Text | Google Scholar

59. Tak YR, Van Zundert RM, Kuijpers RC, Van Vlokhoven BS, Rensink HF, Engels RC. A randomized controlled trial testing the effectiveness of a universal school-based depression prevention program ‘Op Volle Kracht’ in the Netherlands. BMC Public Health (2012) 12:21. doi:10.1186/1471-2458-12-21

PubMed Abstract | CrossRef Full Text | Google Scholar

60. Wijnhoven LA, Creemers DH, Vermulst AA, Scholte RH, Engels RC. Randomized controlled trial testing the effectiveness of a depression prevention program (’Op Volle Kracht’) among adolescent girls with elevated depressive symptoms. J Abnorm Child Psychol (2014) 42:217–28. doi:10.1007/s10802-013-9773-5

CrossRef Full Text | Google Scholar

61. Kindt KCM, van Zundert RMP, Engels RC. Evaluation of a Dutch school-based depression prevention program for youths in high-risk neighborhoods: study protocol of a two-armed randomized controlled trial. BMC Public Health (2012) 12:212. doi:10.1186/1471-2458-12-212

PubMed Abstract | CrossRef Full Text | Google Scholar

62. Kindt KCM, Kleinjan M, Janssens JMAM, Scholte RHJ. Evaluation of a School-Based Depression Prevention Program among adolescents from low-income areas: a randomized controlled effectiveness trial. Int J Environ Res Public Health (2014) 11:5273–93. doi:10.3390/ijerph110505273

CrossRef Full Text | Google Scholar

63. Shochet I, Montague R, Smith C, Dadds M. A qualitative investigation of adolescents’ perceived mechanisms of change from a universal school-based depression prevention program. Int J Environ Res Public Health (2014) 11:5541–54. doi:10.3390/ijerph110505541

PubMed Abstract | CrossRef Full Text | Google Scholar

64. Cutuli JJ, Gillham JE, Chaplin TM, Reivich KJ, Seligman ME, Gallop RJ, et al. Preventing adolescents’ externalizing and internalizing symptoms: effects of the Penn Resiliency Program. Int J Emot Educ (2013) 5:67–79.

PubMed Abstract | Google Scholar

65. Sander JB, McCarty CA. Youth depression in the family context: familial risk factors and models of treatment. Clin Child Fam Psychol Rev (2005) 8:203–19. doi:10.1007/s10567-005-6666-3

PubMed Abstract | CrossRef Full Text | Google Scholar

66. Kutcher S, Wei Y. Mental health and the school environment: secondary schools, promotion and pathways to care. Curr Opin Psychiatry (2012) 25:311–6. doi:10.1097/YCO.0b013e3283543976

PubMed Abstract | CrossRef Full Text | Google Scholar

67. Lima-Serrano M, Lima-Rodríguez JS. Impact of school-based health promotion interventions aimed at different behavioral domains: a systematic review. Gac Sanit (2014) 28:411–7. doi:10.1016/j.gaceta.2014.05.003

PubMed Abstract | CrossRef Full Text | Google Scholar

68. Everett BG. Changes in neighborhood characteristics and depression among sexual minority young adults. J Am Psychiatr Nurses Assoc (2014) 20:42–52. doi:10.1177/1078390313510319

PubMed Abstract | CrossRef Full Text | Google Scholar

69. Rose K, Hawes DJ, Hunt CJ. Randomized controlled trial of a friendship skills intervention on adolescent depressive symptoms. J Consult Clin Psychol (2014) 82:510–20. doi:10.1037/a0035827

PubMed Abstract | CrossRef Full Text | Google Scholar

70. Wahl MS, Adelson JL, Patak MA, Pössel P, Hautzinger M. Teachers or psychologists: who should facilitate depression prevention programs in schools? Int J Environ Res Public Health (2014) 11:5294–316. doi:10.3390/ijerph110505294

PubMed Abstract | CrossRef Full Text | Google Scholar

71. Balázs J, Miklósi M, Keresztény A, Hoven CW, Carli V, Wasserman C, et al. Adolescent subthreshold-depression and anxiety: psychopathology, functional impairment and increased suicide risk. J Child Psychol Psychiatry (2013) 54:670–7. doi:10.1111/jcpp.12016

PubMed Abstract | CrossRef Full Text | Google Scholar

72. Bertha EA, Balázs J. Subthreshold depression in adolescence: a systematic review. Eur Child Adolesc Psychiatry (2013) 22:589–603. doi:10.1007/s00787-013-0411-0

PubMed Abstract | CrossRef Full Text | Google Scholar

73. Saunders K, Brand F, Lascelles K, Hawton K. The sad truth about the SADPERSONS Scale: an evaluation of its clinical utility in self-harm patients. Emerg Med J (2014) 31:796–8. doi:10.1136/emermed-2013-202781

PubMed Abstract | CrossRef Full Text | Google Scholar

74. Simon GE, Rutter CM, Peterson D, Oliver M, Whiteside U, Operskalski B, et al. Does response on the PHQ-9 depression questionnaire predict subsequent suicide attempt or suicide death? Psychiatr Serv (2013) 64:1195–202. doi:10.1176/appi.ps.201200587

PubMed Abstract | CrossRef Full Text | Google Scholar