Kristel Jenniskens1,2,3*
Sanne Rasing1,3
Arne Popma4Daan Creemers1,3Chaimae Ghalit5,6Leonie van Vuuren2
Saskia Mérelle2
Jan Spijker3,7
Femke van Nassau5,6- 1GGZ Oost Brabant, Boekel, Netherlands
- 2113 Suicide Prevention, Amsterdam, Netherlands
- 3Behavioural Science Institute, Radboud University, Nijmegen, Netherlands
- 4Child and Adolescent Psychiatry & Psychosocial Care, Amsterdam, Netherlands
- 5Department of Public and Occupational Health, Amsterdam UMC Location Vrije Universiteit Amsterdam, Amsterdam, Netherlands
- 6Amsterdam Public Health Research Institute, Health Behaviors & Chronic Diseases, Amsterdam, Netherlands
- 7Pro Persona, Nijmegen, Netherlands
Strong Teens and Resilient Minds (STORM) is a multimodal, school-based approach for depression and suicide prevention in adolescents that is currently implemented in a region in the Netherlands. The STORM approach will be implemented in new regions in the coming years. This study used the implementation mapping protocol to report on the development of the STORM implementation plan. First, a needs assessment was conducted through semi-structured interviews with stakeholders and brainstorming sessions with regional programme leaders in the two regions that started implementing STORM in 2023. This led to the identification of six main barriers to implementation: high level of demands for schools, insufficient understanding of the programme content, insufficient network collaboration, no perceived relative advantage of STORM by stakeholders, lack of attention to sustainability, and high work pressure. Second, performance and change objectives were formulated based on these barriers. For example, a performance objective for potential providers was that they felt supported by STORM. Third, implementation strategies were selected from theory and translated into practical applications through brainstorming sessions with programme leaders. The following strategies were included in the implementation plan: collaborate with similar initiatives within the region, free up time for STORM tasks, tailor strategies, identify and prepare STORM champions, and promote network weaving. Last, a plan to evaluate the implementation of STORM and the application of the STORM implementation plan was formulated. Planned evaluation research will provide more insight into the usefulness and impact of the STORM implementation plan.
Introduction
Globally, depression and suicide prevalence in adolescents is high and appears to be increasing (1–8). Adolescent depression is associated with poor social well-being, poor school attendance, failure to complete secondary school, depression recurrence, and the onset of other psychiatric disorders (9–12). Moreover, suicide is the fourth leading cause of death among adolescents aged 15–29 worldwide (13). This stresses the need to implement evidence-based depression and suicide prevention programmes.
Educational settings offer opportunities to reach a large number of adolescents, since most adolescents attend school. Several review studies have found small effects and moderate effects on students’ mental health for universal and indicated school-based depression prevention interventions, respectively (14–17). School-based suicide prevention interventions have shown small positive effects on suicidal ideation and behaviours (18, 19). Katz et al. (20) and Hofstra et al. (21) have suggested combining several interventions to further increase the efficacy of depression and suicide prevention.
Such an approach has been developed in the Netherlands and is called Strong Teens and Resilient Minds (STORM) (22, 23). Currently, the STORM approach consists of four interventions (22, 23): (1) universal prevention through mental health lessons in schools, (2) a gatekeeper training (GKT) for school personnel to create a support network around adolescents, (3) early detection of depressive symptoms and suicidality and further assessment and referral when needed, and (4) Op Volle Kracht (OVK, which translates to “at full force”), an indicated depression prevention intervention based on cognitive behavioural group therapy. The STORM approach is science-based, and several programme components have been found to be effective (22, 23): The GKT has been found effective at increasing knowledge of suicide prevention and confidence to discuss suicidality (24). The OVK training has been found effective at reducing depressive symptoms in adolescent (25, 26).
Despite the existing evidence on the effectiveness of interventions for mental health promotion, prevention, and treatment, most people affected by mental health problems do not receive appropriate intervention (27). Therefore, scaling up effective prevention approaches is warranted. As part of the Dutch National Agenda Suicide Prevention 2021–2025 (28), which states national-level goals and activities in the context of suicide prevention, STORM will be scaled up to a national level. STORM is currently implemented in one region in the Netherlands that has about 250,000 inhabitants. Several new Dutch regions will be financially supported to also implement the approach in the coming years. Higher levels of implementation in various implementation outcomes, such as fidelity or dose, are related to better programme outcomes (29–31). This requires applying strategies that fit the context of new user settings (32). Therefore, developing an implementation plan in collaboration with stakeholders is essential to enhance the level of implementation and the potential programme outcomes.
The current study reports on the development of an implementation plan for STORM using the implementation mapping protocol, a systematic approach to developing an implementation plan by combining theory and co-creation with stakeholders in practice (32). Studies reporting on the development of an implementation plan for school-based mental health approaches in preparation for implementation are scarce. While we studied the example case of STORM, our approach to identifying these strategies and our outcomes could inform other school-based mental health approaches as well. This case is of particular interest to others, because of the complexity of STORM considering the multiple components, and because many stakeholders are involved in providing and implementing the approach.
Methods
The current study used a qualitative case study design to develop an implementation plan for STORM that was co-created with stakeholders in practice, and was guided by thematic analysis (33). The report followed the Standards for Reporting Qualitative Research formulated by O’Brien et al. (34), which was filled in and included in Supplementary File 1. All participants in this study signed an informed consent form before participation. This study was approved by the Ethics Commission Social Sciences of Radboud University, approval number ECSW-LT-2023-2-2-33415.
The STORM approach
First, mental health lessons are offered by mental healthcare professionals in schools to improve mental health literacy. Second, schoolteachers can undergo GKT, through which they learn to identify adolescents who show signs of suicidal behaviours and how to respond to those students. Third, a screening of students’ depression and suicide risk is conducted by the Public Health Service (PHS, in Dutch: GGD) using the Childhood Depression Inventory 2 (35) and the Questionnaire Assessing Suicide and Self Injury (36). Students identified as at risk for suicide are seen within 48 h by Child and Youth Health (CYH) professionals from the PHS for further assessment and referral, if necessary. Students with elevated depressive symptoms based on the Child Depression Inventory 2 are offered the indicated depression prevention intervention called OVK, which is based on cognitive behavioural group therapy (23). This intervention is usually provided by a duo of a care professionals within the school and a care professional in the youth care domain.
An integral part of STORM is collaboration within the network of care and education for adolescents (22, 23, 37). There are four main partners in this network: secondary and vocational schools, municipalities, PHS, and mental health professionals. Secondary and vocational schools are the settings for all interventions that are part of STORM (23). Within a region, these schools collaborate with municipalities in educational partnerships in supporting and caring for youth (38). Also, municipalities financially facilitate the implementation of STORM in practice (23). While regions can apply for a start-up budget through the Dutch National Agenda Suicide Prevention 2021–2025 subsidised by the Ministry of Health, Welfare, and Sports (28), municipalities still have to be involved for sustained financing after 2025. A team of mental health professionals provides consultation, training and personnel for carrying out the interventions (23).
An overview of the regional STORM programme structure, including the tasks of each partner, is provided in the second and third columns of Figure 1. Stakeholders from education, the PHS, mental health services, and municipalities collaborate in each part of the programme structure. For the current study, we defined four stakeholder categories: regional management, regional programme leaders, policymakers, and service providers. These are also indicated in the first column of Figure 1.
Figure 1. Overview of regional STORM programme structure.
National scaling up of STORM
As part of the Dutch National Agenda Suicide Prevention 2021–2025 (28), STORM is being scaled-up to new regions in the Netherlands between 2021 and 2025. Interested regions could apply for a start-up implementation budget. The first two regions to receive this budget were selected in December 2022 and started implementing STORM in the academic year of 2023–2024 (39). To apply for the budget, the regions had to prepare for implementation and had thus already initiated several implementation strategies before the start of the current study. Furthermore, existing STORM regions have already introduced several implementation strategies in recent years. These strategies have already been formulated and provided to the new regions. An overview of the existing implementation strategies can be found in Supplementary File 2. The current study seeked to identify additional strategies from the literature that can help to overcome implementation barriers.
Theoretical background
The tasks of implementation mapping (IM) described by Fernandez et al. (32) offer a systematic approach to developing an implementation plan by combining theory and co-creation with stakeholders in practice. IM has previously helped to identify implementation strategies for various preventive interventions and programmes (40–46). The five tasks of IM are the following: (1) conduct an implementation needs assessment to identify barriers and facilitators for implementation, (2) identify adoption and implementation outcomes, performance objectives, and change objectives, (3) select theoretical methods and design implementation strategies, (4) produce implementation protocols and materials, and (5) evaluate implementation outcomes (32).
We used the Consolidated Framework for Implementation Research (CFIR) from Damschroder et al. (47) to identify barriers to and facilitators for implementation in Task 1. The CFIR describes constructs in five domains to consider as potential barriers or facilitators. First, the innovation domain, which includes constructs related to the innovation being implemented. Second is the inner setting into which the innovation is implemented. Third is the outer setting within which the inner setting exists. The fourth domain concerns individuals and pertains to the roles and characteristics of individuals involved in the innovation being implemented. The last domain implementation process consists of constructs related to the activities and strategies used to implement the innovation (47).
For the selection of theoretical implementation strategies in Task 3, Powell, Waltz (48) compiled a list of 73 implementation strategies based on the results of the Expert Recommendations for Implementing Change (ERIC) study. Recently, this compilation has been adapted to improve its utility in educational settings in the School Implementation Strategies, Translating ERIC Resources (SISTER) Project (49). This project resulted in a list of 75 school-adapted implementation strategies. Both the ERIC and SISTER compilations guided the selection of strategies in this study.
Below, we specify our study procedures in terms of sample and recruitment, data collection, and data analysis conducted for each of the five IM tasks. An overview of our procedures for each task is described in Figure 2.
Figure 2. Overview of procedures and sample per IM task.
Task 1: conduct a needs assessment
The needs assessment helps to identify important actors and potential barriers to and facilitators for implementation (32). For this, we conducted semi-structured interviews of approximately half an hour in February and March 2023. Study participants were selected from the region that has already implemented STORM (region 1) and two regions that were planning to implement STORM (regions 2 and 3) using purposive snowball sampling. First, we invited the national STORM team and regional programme leaders (n = 6) for interviews. Next, they helped to identify and contact other relevant stakeholders within the regions. We aimed to represent all stakeholder groups, and reached out to management and intervention providers from schools (n = 9), the PHS (n = 5), mental health organisations (n = 5), and municipalities (n = 4). Additionally, we invited mentors from secondary schools that had already implemented STORM (n = 2) for interviews, because mentors from secondary schools in the new regions had not yet been informed about the STORM approach. Participants were included until data saturation was reached, which meant that two researchers (KJ and CG) agreed that the last two interviews did not lead to new information. Researchers met with the national coordinators and regional programme leaders prior to the interviews to discuss which other stakeholders to include in the study. Researchers did not meet with any of the other participants prior to the interviews. The researchers’ characteristics did not influence the research questions, approach, methods, results, or transferability.
The interviews followed an interview guide based on the updated CIFR (47). The topics included STORM characteristics (example question: what is your perspective on STORM?) and barriers and facilitators (example question: what are things you think could complicate the implementation of STORM?), as well as the sub-topics Outer Setting, Inner Setting, Individuals, and Implementation Process. The full topic list is added in Supplementary File 3. Interviews were conducted by two researchers (KJ & CG) and audio-recorded. KJ was a PhD student at the time of the study with previous experience in conducting and analysing qualitative research. CG was a bachelor student and intern at the time of the study with no previous experience in qualitative research. The recordings were transcribed verbatim. After the interviews, a short summary was sent to the participants for verification.
To draw up a codebook, three researchers (KJ, CG, and FN) analysed six of the 20 interviews using open coding in Atlas.ti. FN (PhD) has previous experience in conducting and analysing qualitative research. Two researchers (KJ & FN) ordered the codes under the five major domains of the CFIR framework (47) and then combined them into overarching codes using axial coding. The complete codebook can be found in Supplementary File 4. Next, two researchers (KJ & CG) separately coded three transcripts, after which the coding was compared and variations in coding were discussed until both researchers agreed. Subsequently, all interview transcripts were analysed using deductive coding. Finally, the researchers analysed the coded data to identify barriers to and facilitators for the implementation of STORM in new regions.
Two researchers (KJ and FN) presented the identified barriers to four programme leaders from regions 2 and 3 and two stakeholders who had been involved in the implementation of STORM in region 1 during a brainstorming session in May 2023. We asked these participants to indicate, on a scale from 1 to 5 per barrier, whether they thought a barrier required immediate action or not using Mentimeter. The results of this brainstorming are available in Supplementary File 5 (in Dutch). Barriers that were scored higher than 3.5 were selected, while barriers that were scored lower than 2.5 were not. For barriers that were scored between 2.5 and 3.5, a group discussion determined whether the barrier was selected.
Tasks 2, 3, and 4: formulate goals, objectives, implementation strategies, and implementation protocols
We formulated performance objectives based on the most important barriers identified in Task 1. For each performance objective, we formulated change objectives across five determinants based on the example of Kang and Foster (46): knowledge, awareness, skills, outcome expectancy, and self-efficacy. We chose this example, because it was the most complete objectives matrix we found.
Next, we selected theoretical implementation strategies to achieve the change objectives. First, for determinants that match the first-version CFIR constructs (47), strategies were identified using the CFIR-ERIC tool (48, 50). These strategies were then compared to the adapted compilation of ERIC implementation strategies for school-based implementation, SISTER (49), to identify strategies that are suitable for school-based implementation projects. Then, for determinants that did not fit the first-version CFIR constructs, suitable strategies were selected from the SISTER strategies (49).
A second round of brainstorming sessions was hosted in May 2023, one session with two national coordinators and one with four regional programme leaders. In both sessions, the main author (KJ) presented identified strategies for the selected barriers. Participants first discussed which of the identified strategies overlapped with existing implementation strategies. For the remaining strategies, participants discussed the extent to which they were realistic and relevant for practice. The implementation strategies that were considered both realistic and relevant for practice by national coordinators and regional programme leaders were selected for the implementation plan. If participants did not reach consensus about how realistic or relevant a strategy would be, the main author (KJ) made the final decision to include or exclude the strategy. A detailed description of our selection process can be found in Supplementary File 2. Next, the selected strategies were translated into practical applications in collaboration with the programme leaders from regions 2 and 3. The applications were reported following the recommendations for reporting implementation strategies by Proctor et al. (51). The implementation strategies found in the current study were added to the STORM implementation guide developed by the national STORM team. Besides implementation strategies, this document contains a detailed description of the programme components and programme structure of STORM. All future STORM regions will be offered this guide to aid their implementation efforts.
Task 5: develop an evaluation plan
Based on the implementation plan, KJ and FN developed a plan to evaluate the implementation of STORM, as well as the application of the implementation plan in practice.
Results
Task 1
Twenty stakeholders were interviewed. Stakeholders were included until data saturation was reached. Their characteristics are presented in Table 1. Most participants had a coordinating role in the project, followed by providers, management, and policymakers. The number of participants per organisation type were spread evenly, except for municipalities, which were represented by only two participants.
Table 1. Participant characteristics.
We identified 21 barriers to and 13 facilitators for the implementation of STORM in new regions. An overview of all identified barriers and facilitators can be found in Supplementary File 6. These determinants include CFIR constructs and barriers that did not match the CFIR constructs. In the first brainstorming session with programme leaders, five barriers were selected for which implementation strategies should be identified. An overview of these barriers is presented in Table 2.
Table 2. Barrier descriptions.
Multiple participants mentioned that schools are seen as the ideal setting for prevention, not only regarding mental health, but also for prevention of obesity or smoking. This leads to a high demand for schools. For example, one of the regional program leaders mentioned that “many societal developments are occurring and often it’s the schools who have to solve it.” Related to this, most participants recognised high work pressure in both education and healthcare as an important barrier for implementing a new approach. A school principal indicated that “it is an important theme, but honestly I do not have the people, the time, and the money to properly implement STORM.”
Another important issue raised by participants was that, at the time of the interviews, they, nor their colleagues, sufficiently understood the content of the STORM approach and what it means in practice. A school therapist mentioned, for example, that they “still need to receive a lot of information.” Moreover, participants thought that not all stakeholders saw added value in STORM compared to other interventions. A national coordinator indicated that “[organizations] struggle to de-implement [what they were already doing] to implement of STORM.”
It was also noted by some participants that network collaboration required improvement, especially between schools and mental healthcare services. A manager in mental healthcare mentioned that “education sometimes complains: ‘[mental healthcare organizations] do some test, but they never refer back to us’,” while a school therapist mentions that “collaboration [with mental healthcare organizations] does not exist in our school.”
Additional to the interview results, lack of attention to the sustainability of STORM in the current implementation efforts was identified as a barrier during the brainstorming sessions. Programme leaders felt that long-term sustainability was not receiving enough attention yet from stakeholders involved.
Task 2
For each barrier selected in Task 1, we formulated performance and change objectives. The performance and change objectives are listed per stakeholder category in Table 3. For example, a performance objective for programme leaders related to the barrier ‘insufficient network collaboration’ and was that they should stimulate the development of sustainable partnerships between involved organisations. Change objectives for this performance objective were formulated under skills (i.e., able to connect organisations within the STORM network) and self-efficacy (i.e., are confident that they are able to connect organisations within the STORM network).
Table 3. Matrix of change.
Not all barriers were relevant for each stakeholder category. ‘Partnerships and connections’ and ‘Sustainability’ were relevant for all categories, because all stakeholders are part of the STORM network, and the sustainable implementation of STORM should be achieved for all categories. ‘Relative advantage’ was not relevant for the programme leaders, because this is not a barrier for this stakeholder category based on the interviews. Since only the schools, the PHS, and mental healthcare organisations are involved in implementing STORM, the barriers ‘High demand for schools’ and ‘High work pressure’ were only relevant to these stakeholder categories. Finally, ‘Insufficient understanding of programme content’ was regarded as a relevant barrier for management only, because management has the final decision to participate in STORM, and thus needs to be well informed of the content.
Tasks 3 and 4
To address the performance and change objectives formulated in Task 2, 14 implementation strategies were selected using the CFIR-ERIC tool and SISTER that matched the barriers identified (49, 50). Using the second round of brainstorming sessions with national coordinators and regional programme leaders, five implementation strategies were deemed relevant and realistic for practice: ‘pruning competing initiatives’, ‘change/alter environment’, ‘tailor strategies’, ‘identify and prepare champions’, and ‘promote network weaving’ (49). We translated these to practical applications and report on the strategies in Table 4 following the recommendations of Proctor et al. (51). The first strategy described is ‘collaborate with similar initiatives’, in which the idea is that regional program leaders actively identify other mental health school-based initiatives that are (being) implemented in their region, and look for ways to collaborate in the implementation process. The goal is to relieve the pressure on schools and minimise extra workload for service providers. Second is ‘free up time for STORM tasks’, in which organisation management allocate time to for implementing and executing STORM, while program leaders reserve budget to support organisations in doing so. Third is ‘tailor strategies’, meaning programme leaders adapt their communication style and message about STORM to the specific needs of various stakeholders, with the aim of improving adoption. Fourth is ‘identify and prepare STORM champions’, which entails both identifying and supporting individuals within involved organisations that are enthusiastic about the approach. The goal is to promote sustainment within those organisations through these individuals. Last is ‘regional network weaving’ through developing a social map of the organisations and individuals involved in STORM, and organising a joint kick-off session for those organisations and individuals.
Table 4. Implementation strategies
Task 5
In the final task, we developed a plan to evaluate the implementation of STORM in the new regions, as well as the application of the implementation plan over the course of two academic years. We identified outcomes from the implementation outcomes defined by Proctor et al. (52) and process evaluation guidelines from Moore at al. (53) and Saunders et al. (54). In Table 5, we summarise the outcomes for the implementation of STORM, including definitions, and how and when the outcomes will be measured. Providers, regional management, and policymakers will be involved in the evaluation of the implementation process through a survey and interviews at multiple time points. The measurement instruments to be used in the survey comprise a shortened version of the Acceptability of Intervention Measure, Intervention Appropriateness Measure, and Feasibility of Intervention Measure from Weiner et al. (55), and the Normalisation Measure Development Questionnaire (56). In Table 6, we summarise the outcomes for the application of the implementation plan, including definitions of the outcomes, and how and when the outcomes will be measured. Programme leaders and programme groups (see Figure 1) will be involved in the evaluation through a checklist of implementation strategies and focus group sessions. Additionally, we will analyse the administrative data for both evaluations.
Table 5. Evaluation plan for the implementation of STORM.
Table 6. Evaluation plan for the application of the implementation plan.
Discussion
This study aimed to develop an implementation plan for a school-based approach to depression and suicide prevention. To our knowledge, this is the first study reporting on the development of an implementation plan for school-based mental health interventions. The IM tasks from Fernandez et al. (32) helped us to combine practical needs and perceptions with theoretical strategies. We identified six main barriers to implementation, on the basis of which we formulated performance and change objectives. We found five new implementation strategies to achieve these objectives. Lastly, we developed a plan to evaluate the implementation of STORM in new regions over the course of two school years.
One of the most relevant barriers to implementation of STORM that we found was limited network collaboration within regions, while network collaboration is an essential part of the STORM approach (22, 23, 37). A study into determinants for the screening and subsequent referral to the OVK revealed that even in the region where STORM has been implemented for years, collaboration between organisations involved in STORM is not optimal (57). This led us to the selection of several ERIC strategies categorised under ‘develop stakeholder interrelationships’ aimed at improving network collaboration (58). The implementation evaluation will determine whether these strategies indeed helped us increase network collaboration.
Some of the identified barriers were also some of the most frequently mentioned barriers for other school-based mental health interventions (59), including ‘insufficient understanding among stakeholders of programme content’ and ‘insufficient network collaboration’. However, costs and the availability of resources, which are often reported as barriers to implementation (59), were not identified as barriers in the current study. The fact that these factors were not discussed in any of the interviews is most likely due to the implementation budget that regions receive to implement STORM (28). However, this is only a start-up budget that can only be provided to a limited number of regions. Moreover, lack of funding or financial resources was identified as a barrier to sustaining school-based mental health interventions (60). Therefore, keeping track of implementation costs is relevant and has been included as an outcome in our evaluation plan.
We selected implementation strategies based on how realistic and relevant the STORM programme leaders thought they were. In a study by Lyon et al. (61), school-based consultants who provided social, emotional, and mental health services rated the feasibility and importance of all SISTER strategies. Most strategies we selected were also rated important in this study (61). Yet, we included some strategies which were rated low on feasibility in the study from Lyon et al. (61), including ‘collaboration with similar initiatives’, ‘use advisory boards and workgroups’, and ‘promote network weaving’, because they were considered realistic and relevant by the program leaders. These different perceptions might be explained by the difference in the stakeholders involved: we spoke with programme leaders, whereas Lyon et al. (61) consulted stakeholders within schools. However, these differences might also indicate the importance of context when considering the feasibility of an implementation strategy. Our evaluation of the implementation strategies should provide more insight into this difference.
The goal of the implementation plan developed in the current study is to improve the level of implementation of STORM in new regions in the Netherlands. We selected several implementation strategies that were found in the literature to have a positive effect on programme adoption and fidelity, including ‘conduct ongoing training’, ‘identify and prepare champions’, ‘use train-the-trainer strategies’, and ‘facilitation/problem solving’ (62). Still, knowledge about the mechanisms by which implementation strategies target their linked barriers, as well as about the effectiveness of most strategies, is lacking (62–64). Thus, while the IM approach helped us to select strategies that are likely to positively impact the implementation of STORM, our evaluation should confirm whether our selection was accurate.
Strengths and limitations
A strength of the current study is that we systematically developed an implementation plan by following the tasks of IM (32). We did this in close collaboration with stakeholders who will implement STORM in practice, ensuring that the implementation plan matches the needs in practice. Additionally, we enhanced the credibility and transferability of our results through member checks, data and investigator triangulation, and sampling until we reached data saturation.
We recognise some limitations to our study as well. To begin, we mainly identified determinants related to the adoption and implementation of the intervention, and not to sustaining STORM over time. This is mostly likely because sustainability was not an explicit topic in our interviews and interviewees were in an early stage of pre-implementation. We discussed the lack of determinants with programme leaders and accordingly added a general determinant for sustainability. Furthermore, we reached out to multiple stakeholders and interviewed those who responded. Possibly, this led to selection bias if only participants with strong opinions about STORM, be these negative or positive, responded to our invitation. However, we asked participants to reflect on the perceptions of others in their field to minimise this bias.
Recommendations
Building on our strengths and limitations, we first recommend following the tasks of IM when developing an implementation plan, as this helped us to systematically select appropriate strategies. Furthermore, it encourages close collaboration with practice, which we found to be very helpful for developing a plan that is both achievable and relevant for practice. In doing so, we recommend including sustainability in the needs assessment to identify determinants and strategies for sustainability within the implementation plan.
Second, we recommend consulting multiple sources for the selection of implementation strategies. We found it helpful to first use the CFIR-ERIC tool to get a first idea of possible strategies, and then compare them to the SISTER strategies to identify more suitable strategies for the school context. We recommend others developing an implementation plan to consult such strategy compilations for specific intervention settings, if available.
For new STORM regions, we recommend using this implementation plan as guidance rather than a prescription. Some strategies might prove not to be as relevant and/or feasible as we originally believed. The implementation plan could also be helpful for the implementation of other school-based mental health interventions as these might encounter similar barriers. However, tailoring the implementation strategies to the specific context for these interventions is warranted.
Conclusion
In this study, we followed the tasks of IM, which helped us to develop a STORM implementation plan systematically and in collaboration with practice. The implementation plan offers guidance for new regions implementing STORM. Following the implementation plan could help to improve implementation outcomes and might even lead to better programme outcomes. Moreover, our approach and the strategies we identified could inform the implementation of other school-based mental health programmes, although we recommend tailoring our strategies to the specific context into which it will be implemented. Future research evaluating the implementation of STORM across the Netherlands will provide more insight into the usefulness of the implementation plan.
Data availability statement
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.
Ethics statement
This study was approved by the Ethics Commission Social Sciences of Radboud University, approval number ECSW-LT-2023-2-2-33415. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.
Author contributions
KJ: Conceptualization, Formal analysis, Investigation, Methodology, Project administration, Visualization, Writing – original draft, Writing – review & editing. SR: Conceptualization, Supervision, Writing – review & editing, Writing – original draft. AP: Conceptualization, Supervision, Writing – review & editing, Writing – original draft. DC: Conceptualization, Supervision, Writing – review & editing, Writing – original draft. CG: Formal analysis, Investigation, Methodology, Writing – review & editing, Writing – original draft. LV: Investigation, Writing – review & editing, Writing – original draft. SM: Writing – review & editing, Writing – original draft. JS: Writing – review & editing, Writing – original draft. FN: Conceptualization, Formal analysis, Investigation, Methodology, Supervision, Writing – review & editing, Writing – original draft.
Funding
The author(s) declare that financial support was received for the research, authorship, and/or publication of this article. This study was funded by a subsidy of the Ministry of Health, Welfare, and Sport in the programme of the National Agenda Suicide Prevention 2021–2025.
Acknowledgments
We would like to thank the national and regional STORM teams from 113 Suicide Prevention, Groningen, Nijmegen and East Brabant for helping us to recruit study participants and for helping us select implementation strategies that are relevant for practice. We also thank all study participants for participating in an interview.
Conflict of interest
SR and DC were involved in the development of the STORM approach. SR, DC, and LV were involved in the national scale-up of STORM. LV was involved as an interview participant and was therefore not involved in revising, reading and approving the results section.
The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Supplementary material
The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpubh.2024.1386031/full#supplementary-material
References
1. Avenevoli, S, Swendsen, J, He, J-P, Burstein, M, and Merikangas, KR. Major depression in the national comorbidity survey–adolescent supplement: prevalence, correlates, and treatment. J Am Acad Child Adolesc Psychiatry. (2015) 54:37–44.e2. doi: 10.1016/j.jaac.2014.10.010
2. Merikangas, KR, He, J-p, Burstein, M, Swanson, SA, Avenevoli, S, Cui, L, et al. Lifetime prevalence of mental disorders in U.S. adolescents: results from the National Comorbidity Survey Replication–Adolescent Supplement (NCS-A). J Am Acad Child Adolesc Psychiatry. (2010) 49:980–9. doi: 10.1016/j.jaac.2010.05.017
3. Mojtabai, R, Olfson, M, and Han, B. National trends in the prevalence and treatment of depression in adolescents and young adults. Pediatrics. (2016) 138:1–10. doi: 10.1542/peds.2016-1878
4. Oderinde, K, Dada, M, Ogun, O, Awunor, N, Kundi, B, Ahmed, H, et al. Prevalence and predictors of depression among adolescents in Ido Ekiti, south West Nigeria. Int J Clinical Med. (2018) 9:187–202. doi: 10.4236/ijcm.2018.93017
5. Ormel, J, Raven, D, van Oort, F, Hartman, C, Reijneveld, S, Veenstra, R, et al. Mental health in Dutch adolescents: a TRAILS report on prevalence, severity, age of onset, continuity and co-morbidity of DSM disorders. Psychol Med. (2015) 45:345–60. doi: 10.1017/S0033291714001469
6. Orri, M, Scardera, S, Perret, LC, Bolanis, D, Temcheff, C, Séguin, JR, et al. Mental health problems and risk of suicidal ideation and attempts in adolescents. Pediatrics. (2020) 146:1–10. doi: 10.1542/peds.2019-3823
7. Paul, B, and Usha, V. Prevalence and predictors of depression among adolescents. Indian J Pediatr. (2021) 88:441–4. doi: 10.1007/s12098-020-03491-w
8. Uddin, R, Burton, NW, Maple, M, Khan, SR, and Khan, A. Suicidal ideation, suicide planning, and suicide attempts among adolescents in 59 low-income and middle-income countries: a population-based study. Lancet Child & Adolescent Health. (2019) 3:223–33. doi: 10.1016/S2352-4642(18)30403-6
9. Clayborne, ZM, Varin, M, and Colman, I. Systematic review and meta-analysis: adolescent depression and long-term psychosocial outcomes. J Am Acad Child Adolesc Psychiatry. (2019) 58:72–9. doi: 10.1016/j.jaac.2018.07.896
10. Finning, K, Ukoumunne, OC, Ford, T, Danielsson-Waters, E, Shaw, L, De Jager, IR, et al. The association between child and adolescent depression and poor attendance at school: a systematic review and meta-analysis. J Affect Disord. (2019) 245:928–38. doi: 10.1016/j.jad.2018.11.055
11. Verboom, CE, Sijtsema, JJ, Verhulst, FC, Penninx, BW, and Ormel, J. Longitudinal associations between depressive problems, academic performance, and social functioning in adolescent boys and girls. Dev Psychol. (2014) 50:247–57. doi: 10.1037/a0032547
12. Thapar, A, Eyre, O, Patel, V, and Brent, D. Depression in young people. Lancet (London, England). (2022) 400:617–31. doi: 10.1016/S0140-6736(22)01012-1
13. World Health Organization . Live life: an implementation guide for suicide prevention in countries. Geneva: World Health Organization (2021).
14. Caldwell, DM, Davies, SR, Thorn, JC, Palmer, JC, Caro, P, Hetrick, SE, et al. Public Health Research. School-based interventions to prevent anxiety, depression and conduct disorder in children and young people: A systematic review and network meta-analysis. Southampton (UK): NIHR Journals Library; (2021), 9, 1–284.
15. Corrieri, S, Heider, D, Conrad, I, Blume, A, König, H-H, and Riedel-Heller, SG. School-based prevention programs for depression and anxiety in adolescence: a systematic review. Health Promot Int. (2013) 29:427–41. doi: 10.1093/heapro/dat001
16. Feiss, R, Dolinger, SB, Merritt, M, Reiche, E, Martin, K, Yanes, JA, et al. A systematic review and meta-analysis of school-based stress, anxiety, and depression prevention programs for adolescents. J Youth Adolesc. (2019) 48:1668–85. doi: 10.1007/s10964-019-01085-0
17. Werner-Seidler, A, Spanos, S, Calear, AL, Perry, Y, Torok, M, O'Dea, B, et al. School-based depression and anxiety prevention programs: an updated systematic review and meta-analysis. Clin Psychol Rev. (2021) 89:102079. doi: 10.1016/j.cpr.2021.102079
18. Gijzen, MW, Rasing, SP, Creemers, DH, Engels, RC, and Smit, F. Effectiveness of school-based preventive programs in suicidal thoughts and behaviors: a meta-analysis. J Affect Disord. (2022) 298:408–20. doi: 10.1016/j.jad.2021.10.062
19. Robinson, J, Calear, AL, and Bailey, E. Suicide prevention in educational settings: a review. Australas Psychiatry. (2018) 26:132–40. doi: 10.1177/1039856218759406
20. Katz, C, Bolton, SL, Katz, LY, Isaak, C, Tilston-Jones, T, Sareen, J, et al. A systematic review of school-based suicide prevention programs. Depress Anxiety. (2013) 30:1030–45. doi: 10.1002/da.22114
21. Hofstra, E, van Nieuwenhuizen, C, Bakker, M, Özgül, D, Elfeddali, I, de Jong, SJ, et al. Effectiveness of suicide prevention interventions: a systematic review and meta-analysis. Gen Hosp Psychiatry. (2020) 63:127–40. doi: 10.1016/j.genhosppsych.2019.04.011
22. Gijzen, MW, Creemers, DH, Rasing, S, Smit, F, and Engels, RC. Evaluation of a multimodal school-based depression and suicide prevention program among Dutch adolescents: design of a cluster-randomized controlled trial. BMC Psychiatry. (2018) 18:1–12. doi: 10.1186/s12888-018-1710-2
23. Rasing, S, Spijker, J, Stikkelbroek, Y, and Creemers, D. De beste behandeling voor depressie is vroegsignalering en preventie bij jeugdigen. Tijdschr Psychiatr. (2021) 2021:111–4.
24. Terpstra, S, Beekman, A, Abbing, J, Jaken, S, Steendam, M, and Gilissen, R. Suicide prevention gatekeeper training in the Netherlands improves gatekeepers' knowledge of suicide prevention and their confidence to discuss suicidality, an observational study. BMC Public Health. (2018) 18:637. doi: 10.1186/s12889-018-5512-8
25. de Jonge-Heesen, KW, Rasing, S, Vermulst, AA, Scholte, RH, van Ettekoven, KM, Engels, RC, et al. Randomized control trial testing the effectiveness of implemented depression prevention in high-risk adolescents. BMC Med. (2020) 18:1–13. doi: 10.1186/s12916-020-01656-0
26. Wijnhoven, LA, Creemers, DH, Vermulst, AA, Scholte, RH, and Engels, RC. Randomized controlled trial testing the effectiveness of a depression prevention program (‘Op Volle Kracht’) among adolescent girls with elevated depressive symptoms. J Abnorm Child Psychol. (2014) 42:217–28. doi: 10.1007/s10802-013-9773-5
27. Herrman, H, Patel, V, Kieling, C, Berk, M, Buchweitz, C, Cuijpers, P, et al. Time for united action on depression: a lancet–world psychiatric association commission. Lancet. (2022) 399:957–1022. doi: 10.1016/S0140-6736(21)02141-3
28. 113 Suicide Prevention . Derde Landelijke Agenda Suïcidepreventie, 2021–2025. Amsterdam: 113 Suicide Prevention (2020).
29. Durlak, JA, and DuPre, EP. Implementation matters: a review of research on the influence of implementation on program outcomes and the factors affecting implementation. Am J Community Psychol. (2008) 41:327–50. doi: 10.1007/s10464-008-9165-0
30. Seral-Cortes, M, De Miguel-Etayo, P, Zapata, P, Miguel-Berges, ML, and Moreno, LA. Effectiveness and process evaluation in obesity and type 2 diabetes prevention programs in children: a systematic review and meta-analysis. BMC Public Health. (2021) 21:348. doi: 10.1186/s12889-021-10297-8
31. Wierenga, D, Engbers, LH, Van Empelen, P, Duijts, S, Hildebrandt, VH, and Van Mechelen, W. What is actually measured in process evaluations for worksite health promotion programs: a systematic review. BMC Public Health. (2013) 13:1190. doi: 10.1186/1471-2458-13-1190
32. Fernandez, ME, Ten Hoor, GA, Van Lieshout, S, Rodriguez, SA, Beidas, RS, Parcel, G, et al. Implementation mapping: using intervention mapping to develop implementation strategies. Front Public Health. (2019) 7:158. doi: 10.3389/fpubh.2019.00158
33. Braun, V, and Clarke, V. Using thematic analysis in psychology. Qual Res Psychol. (2006) 3:77–101. doi: 10.1191/1478088706qp063oa
34. O'Brien, BC, Harris, IB, Beckman, TJ, Reed, DA, and Cook, DA. Standards for reporting qualitative research: a synthesis of recommendations. Acad Med. (2014) 89:1245–51. doi: 10.1097/ACM.0000000000000388
35. Bodden, D, Braet, C, and Stikkelbroek, Y. CDI-2 Screeningsvragenlijst voor depressie bij kinderen en jongeren. Amsterdam: Uitgeverij Hoogrefe (2016).
36. Kerkhof, A, and Huisman, A. VOZZ: Vragenlijst over Zelfdoding en Zelfbeschadiging. Houten: Bohn Stafleu van Loghum (2016).
37. de Jonge-Heesen, KW, van Ettekoven, KM, Rasing, S, Liempd, FH, Vermulst, AA, Engels, RC, et al. Evaluation of a school-based depression prevention program among adolescents with elevated depressive symptoms: study protocol of a randomized controlled trial. BMC Psychiatry. (2016) 16:1–9. doi: 10.1186/s12888-016-1119-8
38. de Haan, A, Keij, M, and Lammers, J. Handreiking bevordering mentale gezondheid jeugd. Utrecht: Trimbos Instituut & Pharos (2021).
39. Samen Minder Suïcide . Bewezen effectieve aanpak voor depressie nu voor veel jongerne in Nederland beschikbaar (2023). [cited 2023 July 7th]. Available from: https://samenmindersuicide.nl/bewezen-effectieve-aanpak/.
40. Domlyn, AM, Crowder, C, Eisenson, H, Pollak, KI, Davis, JM, Calhoun, PS, et al. Implementation mapping for tobacco cessation in a federally qualified health center. Front Public Health. (2022) 10:908646. doi: 10.3389/fpubh.2022.908646
41. Ibekwe, LN, Walker, TJ, Ebunlomo, E, Ricks, KB, Prasad, S, Savas, LS, et al. Using implementation mapping to develop implementation strategies for the delivery of a Cancer prevention and control phone navigation program: a collaboration with 2-1-1. Health Promot Pract. (2022) 23:86–97. doi: 10.1177/1524839920957979
42. Odawara, M, Saito, J, Yaguchi-Saito, A, Fujimori, M, Uchitomi, Y, and Shimazu, T. Using implementation mapping to develop strategies for preventing non-communicable diseases in Japanese small-and medium-sized enterprises. Front Public Health. (2022) 10:873769. doi: 10.3389/fpubh.2022.873769
43. Pérez Jolles, M, Fernández, ME, Jacobs, G, De Leon, J, Myrick, L, and Aarons, GA. Using implementation mapping to develop protocols supporting the implementation of a state policy on screening children for adverse childhood experiences in a system of health centers in inland Southern California. Front Public Health. (2022) 10:876769. doi: 10.3389/fpubh.2022.876769
44. Roth, IJ, Tiedt, MK, Barnhill, JL, Karvelas, KR, Faurot, KR, Gaylord, S, et al. Feasibility of implementation mapping for integrative medical group visits. J Alt Complement Med (New York, NY). (2021) 27:S-71–80. doi: 10.1089/acm.2020.0393
45. Thackeray, A, Waring, J, Hoogeboom, TJ, Nijhuis-van Der Sanden, MW, Hess, R, Fritz, JM, et al. Implementing a Dutch physical therapy intervention into a US health system: selecting strategies using implementation mapping. Front Public Health. (2022) 10:908484. doi: 10.3389/fpubh.2022.908484
46. Kang, E, and Foster, ER. Use of implementation mapping with community-based participatory research: development of implementation strategies of a new goal setting and goal management intervention system. Front Public Health. (2022) 10:10. doi: 10.3389/fpubh.2022.834473
47. Damschroder, LJ, Reardon, CM, Widerquist, MAO, and Lowery, J. The updated consolidated framework for implementation research based on user feedback. Implement Sci. (2022) 17:75. doi: 10.1186/s13012-022-01245-0
48. Powell, BJ, Waltz, TJ, Chinman, MJ, Damschroder, LJ, Smith, JL, Matthieu, MM, et al. A refined compilation of implementation strategies: results from the expert recommendations for implementing change (ERIC) project. Implement Sci. (2015) 10:1–14. doi: 10.1186/s13012-015-0209-1
49. Cook, CR, Lyon, AR, Locke, J, Waltz, T, and Powell, BJ. Adapting a compilation of implementation strategies to advance school-based implementation research and practice. Prev Sci. (2019) 20:914–35. doi: 10.1007/s11121-019-01017-1
50. Waltz, TJ, Powell, BJ, Fernández, ME, Abadie, B, and Damschroder, LJ. Choosing implementation strategies to address contextual barriers: diversity in recommendations and future directions. Implement Sci. (2019) 14:42. doi: 10.1186/s13012-019-0892-4
51. Proctor, EK, Powell, BJ, and McMillen, JC. Implementation strategies: recommendations for specifying and reporting. Implement Sci. (2013) 8:1–11. doi: 10.1186/1748-5908-8-139
52. Proctor, EK, Silmere, H, Raghavan, R, Hovmand, P, Aarons, G, Bunger, A, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health Ment Health Serv Res. (2011) 38:65–76. doi: 10.1007/s10488-010-0319-7
53. Moore, GF, Audrey, S, Barker, M, Bond, L, Bonell, C, Hardeman, W, et al. Process evaluation of complex interventions: Medical Research Council guidance. BMJ. (2015) 350:350. doi: 10.1136/bmj.h1258
54. Saunders, RP, Evans, MH, and Joshi, P. Developing a process-evaluation plan for assessing health promotion program implementation: a how-to guide. Health Promot Pract. (2005) 6:134–47. doi: 10.1177/1524839904273387
55. Weiner, BJ, Lewis, CC, Stanick, C, Powell, BJ, Dorsey, CN, Clary, AS, et al. Psychometric assessment of three newly developed implementation outcome measures. Implement Sci. (2017) 12:108. doi: 10.1186/s13012-017-0635-3
56. Vis, C, Ruwaard, J, Finch, T, Rapley, T, de Beurs, D, van Stel, H, et al. Toward an objective assessment of implementation processes for innovations in health care: psychometric evaluation of the normalization measure development (NoMAD) questionnaire among mental health care professionals. J Med Internet Res. (2019) 21:e12376. doi: 10.2196/12376
57. Braam, MWG, Rasing, SPA, Heijs, DAM, Lokkerbol, J, van Bergen, DD, Creemers, DHM, et al. Closing the gap between screening and depression prevention: a qualitative study on barriers and facilitators from the perspective of public health professionals in a school-based prevention approach. BMC Public Health. (2023) 23:884. doi: 10.1186/s12889-023-15705-9
58. Waltz, TJ, Powell, BJ, Matthieu, MM, Damschroder, LJ, Chinman, MJ, Smith, JL, et al. Use of concept mapping to characterize relationships among implementation strategies and assess their feasibility and importance: results from the expert recommendations for implementing change (ERIC) study. Implement Sci. (2015) 10:1–8. doi: 10.1186/s13012-015-0295-0
59. Richter, A, Sjunnestrand, M, Romare Strandh, M, and Hasson, H. Implementing school-based mental health services: a scoping review of the literature summarizing the factors that affect implementation. Int J Environ Res Public Health. (2022) 19:1–30. doi: 10.3390/ijerph19063489
60. March, A, Stapley, E, Hayes, D, Town, R, and Deighton, J. Barriers and facilitators to sustaining school-based mental health and wellbeing interventions: a systematic review. Int J Environ Res Public Health. (2022) 19:3587. doi: 10.3390/ijerph19063587
61. Lyon, AR, Cook, CR, Locke, J, Davis, C, Powell, BJ, and Waltz, TJ. Importance and feasibility of an adapted set of implementation strategies in schools. J Sch Psychol. (2019) 76:66–77. doi: 10.1016/j.jsp.2019.07.014
62. Baffsky, R, Ivers, R, Cullen, P, Wang, J, McGillivray, L, and Torok, M. Strategies for enhancing the implementation of universal mental health prevention programs in schools: a systematic review. Prev Sci. (2022) 24:337–52. doi: 10.1007/s11121-022-01434-9
63. Driessen, A, and Van Nassau, F. Kennisagenda Implementatie 2022: Verkenning van kennishiaten en vraagstukken op het gebied van implementatie in de gezondheidszorg in Nederland Dutch Implementation Collective (2022).
Keywords: implementation mapping, implementation, adolescents, prevention, depression, suicide
Citation: Jenniskens K, Rasing S, Popma A, Creemers D, Ghalit C, van Vuuren L, Mérelle S, Spijker J and van Nassau F (2024) Development of an implementation plan for a school-based multimodal approach for depression and suicide prevention in adolescents. Front. Public Health. 12:1386031. doi: 10.3389/fpubh.2024.1386031
Edited by:
Fabia Morales-Vives, University of Rovira i Virgili, SpainReviewed by:
Herman Hay-ming Lo, Hong Kong Polytechnic University, Hong Kong SAR, ChinaJose A. Piqueras, Miguel Hernández University of Elche, Spain
Copyright © 2024 Jenniskens, Rasing, Popma, Creemers, Ghalit, van Vuuren, Mérelle, Spijker and van Nassau. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Kristel Jenniskens, Kwj.jenniskens@ggzoostbrabant.nl