Multiple Health Behavior Programs in School Settings: Strategies to Promote Transfer-of-Learning Through Life Skills Education

Typically, schools implement health promotion programs that focus on a single behavioral domain. Multiple related health topics may be addressed using separate interventions, potentially producing overlap in program content. However, integrative approaches in health promotion have the potential to produce interventions capable of improving multiple health behaviors. In particular, more research is needed to identify the conditions and the factors that can promote the transfer of learning to broaden the target outcomes of health promotion programs. The present study aims to identify the characteristics of an evidence-based life skills education program that can facilitate the transfer of learning to different health behaviors not initially targeted by the program, and the strategies for achieving successful transfer. A two round Delphi method was used with a diverse group of 21 experts in health promotion, life skills education, and methods of pedagogy for early adolescent students. Questionnaires with open and closed-ended questions were administered on-line. Content qualitative analysis was run, integrating codes, subcategories, and categories of the two rounds of the study. Results showed strong consensus among experts about the potential for promoting the transfer of skills from one health domain to another. Many elements were identified as important factors that facilitate the transfer of learning. Strategies for successful transfer were related to teaching methods, educational approaches, and consistency with the broader school curriculum. Findings suggest that the successful transfer of learning to a new health domain requires that educators recognize its importance and explicitly designate it as an educational aim.

Introduction

Typically, schools implement health promotion programs that focus on a single behavioral domain, and multiple related health topics may be addressed using separate interventions (1). This may occur because educators often view student health in the context of specific behaviors and outcomes; they also tend to hold beliefs about health promotion that are problem-specific rather than broad-based (2, 3). Moreover, when new problem behaviors are observed, there may be public pressure from families and local communities to address the issue promptly. Schools may then choose to implement a new prevention program that addresses the problem, but this may further increase an overlap in content areas across potentially similar interventions. For example, in recent years, new interventions have been developed to prevent youth gambling (4), reduce problem internet use (5), and promote mindful meditation (6), yet when new programs are adopted and implemented there may be little consideration given to whether the new content overlaps with existing school programming. Also, most evidence-based programs focus on specific health domains and their evaluations typically focus only on those specific behavioral outcomes. Crossover effects, in which researchers examine the impact of an intervention on similar behaviors with similar etiologies, are rarely tested. When overlapping interventions are implemented simultaneously, this may contribute to logistical problems and the inefficient use of limited resources. Every intervention requires funding, time, and capacity-building, and school staff can become overwhelmed with the numerous requests they receive to implement social and health-related interventions (7–12).

An important potential strategy to help reduce programmatic overlap may be to assist schools in identifying their specific needs for prevention and guide them in selecting the most appropriate interventions to implement (12). However, if only specific, highly prevalent problem behaviors are considered, this will fail to address the importance of primary and universal prevention and the promotion of healthy behaviors and well-being, which by definition should be implemented prior to any specific need. It is also important to consider that health behaviors are often related to one another, influence each other, and are related to people's lifestyles (13–16). Multiple unhealthy or risky behaviors often co-occur in both adolescents (17, 18) and adults (19). Furthermore, multiple health risk behaviors, including alcohol, drug use, and sexual risk taking, share a common set of risk and protective factors that include sociodemographic, interpersonal, school and family factors (20).

Thus, another strategy to reduce programmatic overlap may be to target multiple behaviors with the same intervention. In 2002, the Society of Behavioral Medicine formed a special interest group to contribute to the development of a science of multiple health behavior change (MHBC) for health promotion and disease management (19). MHBC has since become a developed area of research with theoretical and practical implications. Studies about the potential for long-term effects of broad-based prevention programs on behaviors that were not explicitly targeted as part of an MHBC intervention have been published and reviewed (8, 12, 21–24). A comprehensive and holistic approach, as opposed to one focusing on a single dimension of child health, has been supported for over 25 years by the whole school approach and the Health Promoting Schools (HPS) framework of the World Health Organization (WHO) (25–29).

Although the importance of the MHBC approach is recognized, there is little consensus on how to most effectively combine multiple health promotion programs with potentially overlapping content (8, 19, 30). An integrative approach is a promising framework for developing MHBC interventions (12, 13, 19, 23, 31–34). It aims to integrate intervention content based on links between health behaviors and their determinants (35, 36). Several health behavior theories suggest that distal psychosocial determinants are the same for a variety of different healthy and risky behaviors. Other theories identify proximal determinants which are often domain-specific but are generalizable constructs (e.g., self-efficacy, attitudes, and social influences).

An integrative strategy suitable for school settings is the transfer of learning approach, an approach based on extensive educational theory and research (37–43). Transfer of learning “occurs when learning in one context or with one set of materials impacts on performance in another context or with other materials” (42 p. 3). Transfer of learning is a core concept in education because it represents the final aim of education, that is, producing students with the capacity to apply acquired knowledge and skills to new situations and contexts (42). For example, students learn to interpret a textbook, then a romance novel or an essay, and in the long term, these literacy and critical thinking skills transfer to related activities and should enable them to read a newspaper critically. This transfer will also improve students' citizen skills and behaviors. Transfer of learning occurs across several dimensions that can be distinguished from one another (37). These include the specific context of the initial instruction, characteristics of the situation where this learning that takes place, the content of what was first learned and how it is transferred to a new context, the performance that is required in the original learning and new applied contexts, and the level of specificity and generality of the content and contexts. The contextual differences include several aspects: learning can be transferred across different knowledge domains or across varying social, physical, or temporal contexts. The change in these contexts can be small (e.g., at school for the same subject, for a different subject, or for a laboratory activity) or large (e.g., the transfer of learning from school to at home, or to applications when interacting with friends outside of school). These classifications show how powerful and widespread the transfer of learning can be.

A similar transfer of learning process is likely to occur as a result of participating in health promotion interventions, particularly when such programs are based on a positive youth development framework. Examining the transfer of learning in health promotion instruction is important yet understudied. Students may learn knowledge, attitudes, and skills in one context or health behavior domain and then be able to apply them in other contexts or domains (12, 44–46). For example, problem-solving or assertiveness skills can be developed in a drug prevention context and then be transferred to scenarios involving reducing sexual risk behaviors. Similarly, awareness about the influence of social norms or the adoption of healthy values may be also transferred from one health behavior domain to another (8, 12, 21, 22, 24). For school-based health promotion, a promising approach for MHBC is life skill education. This broad-based approach focuses on general skills building and enhancing resilience and has proven to be an effective prevention strategy across multiple health risk behaviors (47, 48). Life skills prevention approaches are based on broad theories, such as Problem Behavior Theory (49) or Social Learning Theory (50), that are applied to multiple behavioral domains. Methods and strategies focus on building and developing cognitive, emotional and behavioral skills. Action learning and cooperative learning methodologies are used and have been shown to be effective. The teaching methods are flexible and can be adapted for a variety of contexts or student characteristics. Life skill education efficacy increases when implemented through a whole school approach (51–53). Indeed, most of the programs included in reviews about multiple behavior approaches use a life skills education strategy (8). In practice, the transfer of context is a core concept for many health promotion and life skills education programs. Skills are taught to students prior to situations where they may be called upon to use them. For example, students' refusal skills are taught and reinforced from an early age so that students can later use these skills to refuse cigarette offers as adolescents.

A research area that needs more attention is the definition of the conditions that can promote the transfer of learning. Transfer of learning does not occur without the sustained attention by stakeholders to several necessary preconditions (12, 34, 42, 54, 55). Paulussen et al. identified three preconditions (34): behaviors are associated and have similar determinants, methods to modify these determinants are similar, and students are encouraged to apply what they have learned to different behaviors. Peters (12) underlines two transfer-promoting aspects. First, general principles or procedures relevant for carrying out behaviors must be explicitly addressed (e.g., rules about how to refuse), and students must be prompted to apply these general principles to multiple health behavior domains (e.g., tobacco and sex). Processes of contextualization (learning new skills in one context), decontextualization (deducing a general principle), and recontextualization (examining application in other contexts) should be both knowable and identifiable. Second, these general principles must be meaningful to students and relevant to their personal lives. Perkins and Salomon (42) outline several different conditions in which transfer of learning occurs, the mechanisms that underlie such transfer, and how strategies to promote transfer should be taught. What they refer to as the “low road of transfer” is based on repeated practice in different contexts, where practice activates specific scripts, action schema, and semi-automatic responses, and the transfer happens when stimulus conditions are similar to those in a prior context of learning and are able to trigger the same responses. What they call the “high road of transfer” is promoted by mindful abstraction, metacognition, and deliberate search of connections. The integration of these two approaches should be considered in developing a transfer of learning approach in the health promotion area.

Purpose

The present study focuses on the transfer of learning approach in health promotion. It aims to identify the conditions and factors that can facilitate the transfer of learning during the implementation of a life skills education program. In particular, the objectives are to identify:

• the characteristics of a life skill education program that can facilitate the transfer of learning to different health behaviors not explicitly targeted by the program,

• the necessary elements that should be reinforced to facilitate the transfer, and

• the strategies for achieving successful transfer.

The Delphi method was used to solicit input on these issues from a diverse group of experts. The Delphi method (56–59) is a consensus development technique used to obtain the most reliable consensus of a group of experts. In health research, Delphi methods have been used to identify core outcomes to measure in randomized controlled trials (RCTs), to identify research priorities in a specific setting, and to develop guidelines and specify program theory (60–64).

Method

Design

A two-round Delphi process was used in the present study as outlined by Schmidt (60, 65). First, potential experts from both the health and education sectors were identified and recruited to participate in the study. They were selected by health unit directors and HPS leaders of the Lombardy Region in the north of Italy. Inclusion criteria were that participants: (a) were experts in life skills education or the implementation of health promotion programs to reduce risky behaviors; (b) belonged to a Health Unit or a HPS; (c) actively participated in the regional group for the program adaptation. Then, a preliminary face-to-face study phase was designed to share with participants the objectives of the research and the topics being analyzed. During two face-to-face meetings, key elements of the life skill education program were summarized and an overview of the scientific literature on the transfer of learning was presented and discussed. The first-round of the Delphi procedure consisted of a brainstorming of ideas through open-ended questions. Each participant had the opportunity to write down ideas and thoughts without restrictions or limits. The authors then analyzed the answers obtained, and identified specific categories, characteristics, and motivations in participants' responses. The result of the first-round was a detailed draft list of key characteristics related to transfer of learning. A second round of the Delphi procedure was designed to expand upon and enrich the list, validate it, and choose the most important strategies for transfer of learning. The first-round results were presented to the experts involved and feedback and integration were requested. This second round also aimed to create group consensus on perspectives regarding key factors and strategies related to the transfer of learning. A third round was not necessary because data saturation was fully reached after the second round. All participants agreed with the first classification, and no new categories were added. The comments produced in the second-round provided further specification and refinement of content that emerged in the first-round. Figure 1 outlines the process of the study.

FIGURE 1

Figure 1. Delphi study process.

Both rounds used on-line questionnaires sent by e-mail. The study, procedure, and instruments were approved by the Regional Committee of Health Promoting School Network, and by the Regional Coordinators of the LifeSkills Training adaptation group, institutional review boards who reviewed the study for ethical standards.

Participants

Twenty-three potential experts were identified and recruited to participate in the study. Among them, 21 answered the first-round questionnaire (91% response rate): 14 were health professionals with different backgrounds (physicians, psychologists, health workers, and educators) and with expertise in different health behaviors (addiction, bullying, mental health, nutrition, physical activity, sexual health, and sexually transmitted diseases); seven were school staff (two principals, three teachers, and two office professionals). In the second-round, 19 of these professionals answered the questionnaire: 12 from the health sector and seven from the education sector. All approached individuals agreed to participate and provided written consent.

The Life Skill Education Program

The LifeSkills Training program (LST) (66) is a research-validated school-based prevention program proven to reduce alcohol, tobacco, drug abuse, and violence by targeting the major social and psychological factors that promote the initiation of substance use and other risky behaviors [for a review (67)]. The program provides adolescents with the confidence and skills necessary to handle challenging situations and succeed at the developmental tasks of adolescence. In particular, personal self-management skills (self-improvement, decision-making, problem-solving, coping with stress and anxiety, and managing anger), social skills (communication skills, resolving conflicts), and drug resistance skills (assertiveness, coping with peer pressure) are taught to students who receive the program. Moreover, anti-drug knowledge, attitudes against drug use, social norms awareness, and healthy values are promoted. Teaching strategies used in life skills education include traditional didactic teaching methods, facilitation and group discussion, and classroom demonstrations. A key teaching method is cognitive–behavioral skills training, which consists of instruction, demonstration, behavioral rehearsal (i.e., practice), feedback, social reinforcement (e.g., praise), and extended practice in the form of behavioral homework assignments.

In Lombardy, a region in the north of Italy, the LST Lombardia program was adapted to local culture and needs, and it was integrated with a whole school approach and HPS strategy (68, 69). Its efficacy was verified through a large-scale effectiveness study (70). It has been implemented with ~50,000 students in the region. The LST program represents an excellent case study to analyze transfer of learning because the program is recognized as a Model or Exemplary program by an array of government agencies, it is one of the most disseminated and well-known life skill education programs, and it has already been shown to be effective in changing behaviors that were not explicitly targeted (71, 72).

Questionnaires

During the first-round, participants were asked to: (a) review summary documents about core characteristics of the LST program and an accompanying literature review about conditions and strategies to promote the transfer of learning and (b) answer three open-ended questions: (1) “Which conditions and characteristics that promote the transfer of learning are included in the LST Lombardia program?;” (2) “Which conditions and characteristics that promote the transfer of learning are lacking in LST Lombardia and should be integrated?;” (3) “Which core elements of LST Lombardia can be leveraged to facilitate these integrations?” Participants were asked to both identify conditions and characteristics and to justify their answers. Participants' names and institutional affiliations were also collected.

The second-round asked participants to read the analysis of the first-round carefully and to answer a questionnaire designed to validate the results, integrate the list with new ideas, and identify the most important strategies to promote a transfer of learning approach through LST Lombardia. Validation was assessed with three questions, with response options on a 5-point Likert scale anchored by 1 (not at all) and 5 (a lot), examining the adequacy of the coding for the first-round answers, clarity of categories, and clarity of explanations of each category and subcategory. Additional suggestions were solicited with an open-ended question to improve and refine classification. Then, participants were asked to suggest additional ideas that they might not have considered initially related to the three themes of the first-round (characteristics included, characteristics lacking, and potential elements to leverage) and to identify elements suggested by other participants that they don't agree with. These four questions were open-ended. Based on the strategies identified in the first-round, participants were asked to select the most important strategies for adapting the LST program with a transfer-oriented approach, with item response options on a 10-point Likert scale anchored by 1 (Not important at all) and 10 (Extremely important). The aim of the ranking procedure was to identify and prioritize important shared strategies for transfer-of-learning. Finally, participants were asked to make additional suggestions for achieving the tasks above.

Data Analysis

The Delphi method establishes an iterative, multi-round process to collect systematically, aggregate, and present the individual and group's opinions or judgments on specific questions and issues related to the topic area of interest (60, 65). A two-round Delphi process was used in the present study, and results were analyzed with an integrative approach.

First-round answers were analyzed by the first author and revised and discussed with the second one. A qualitative content analysis was run (73). Answers were coded in meaningful units and then grouped under higher-order headings. Codes, subcategories, categories, and themes were identified. Both top-down and bottom-up analytic processes were used. The themes were coincident with the aims of the study. The categories, subcategories, and codes were defined by an abstraction process of answers and meaningful units. First, the units were condensed into brief descriptions close to the text of the manifest content representing the codes. The interpretation of the latent content and the abstraction process was used to define the subcategories and categories. Categories represent the program characteristics, or the strategy related to the transfer of learning. Subcategories described in-depth characteristics of specific elements related to the transfer of learning and why these elements are important.

Second-round answers were analyzed with both quantitative and qualitative methods. Qualitative content analysis was used for open-ended questions, and results were used to integrate or modify first-round analysis. Also in this case, answers were analyzed by the first author and revised and discussed with the second one. First, the answers were coded as in the first-round. Then, the new codes were compared to the first-round ones: duplicates were removed, and new codes were added and new subcategories created when necessary. No new categories emerged in the second-round, demonstrating excellent data saturation. This integrative analysis approach between the two rounds allowed the research team to aggregate the information collected during the entire study and relate responses to both individual and group perspectives. For the same reasons, the answers of experts from the health and educational sectors were analyzed together and the combination of the two perspectives were examined as a way to document consensus. This integrative approach produced a combined list of relevant factors to promote the transfer of learning. Descriptive analyses were run to validate the first-round classification and to rank strategies identified.

Results

Categories

Open-ended questions of the first-round were analyzed to classify the factors and strategies that facilitate the transfer of learning to different health behaviors not targeted by the LST program. The results were classified into three themes according to the study aims and 20 categories, 86 subcategories and 174 codes were identified. This classification of key characteristics related to transfer of learning was presented to participants in the second-round of the study. The experts had the opportunity to expand upon the list or to point out elements suggested by other participants that they did not agree with. No categories were added, showing excellent data saturation. The list was enriched with nine new subcategories and 17 new codes. No factors were considered as not relevant or not agreeable. Some integrations were suggested to better specify a few factors. The final list describes the conditions or characteristics of the program that can facilitate the transfer of learning to different health behaviors, the conditions or characteristics of the program that should be reinforced to facilitate the transfer, and the strategies to facilitate the transfer and adapt the program. Table 1 summarizes the number of themes, categories, subcategories and codes for each theme.

TABLE 1

Table 1. Codes and categories.

As shown in Table 2, four main categories of themes were identified in the two rounds regarding the conditions or characteristics of the LST program that can facilitate transfer of learning. The first category had to do with the instructional or learning methods used in the program, and several subcategories were identified that were seen as facilitating the transfer of learning. For example, the program's focus on active learning, the use of peer-to-peer practice, and the integration between practice and reflection were all viewed as key instructional methods that promote transfer of learning. The second category was related to teacher competencies that facilitated the transfer of learning. For example, the use of facilitated discussions, coaching, behavioral rehearsal, and providing positive feedback, and the use of these teaching methods across the entire curriculum were all viewed as key teacher competencies that promote transfer of learning. Also, the inclusion of questions at the end of each teaching unit to enhance students' awareness of learning outcomes and processes were considered particularly important. A third category was related to the program content and activities. Examples that facilitated the transfer of learning included the focus on life skills, the use of behavioral techniques taught to students, and the consistency with the broader school curriculum. The final category of themes identified included program characteristics that facilitated transfer of learning, and included subcategories related to the inclusion of booster sessions and the whole school approach.

TABLE 2

Table 2. Conditions or characteristics of the LST program that can facilitate the transfer of learning to different health behaviors.

As shown in Table 3, six categories of themes were identified in the two rounds regarding the conditions or characteristics of the program that should be reinforced to facilitate the transfer of learning. The first category was valuing transferability, such that the program is valued as a tool for the transfer of skills. For example, the broad-based nature of the program inherently values teaching general skills and promoting youth development, not simply preventing risk behaviors. Specific elements of the program (e.g., mindfulness in learning) as well as specific abilities or skills taught in the program (e.g., self-efficacy, flexibility) were seen as critical to reinforce to effectively promote the transfer of learning. The life skill approach was also valued and identified as a precondition to transfer learning to different health domains. Other categories that should be reinforced to facilitate transfer of learning included coherence within the broader school curriculum and the focus on student needs. Participants also stated that the degree of consistency between the health promotion program and the school's planning and integration of the program within the school curriculum was critical. Finally, participants considered it important that schools explicitly promote a transfer of learning approach and establish it as an essential goal for the school. Contents about specific behavior were cited. However, participants noted that any new content should not require a new intervention, rather it should be integrated into the regular school curriculum or introduced to students through recontextualization of skills and techniques already taught.

TABLE 3

Table 3. Conditions or characteristics of the LST program that should be reinforced to facilitate the transfer.

As shown in Table 4, several categories of themes were identified regarding the conditions to facilitate transfer of learning. These included the importance of integrating LST with daily teaching, school planning, and the larger school context. This might include integrating program terminology, methods, and competencies across the entire curriculum. Other strategies to facilitate transfer of learning included explicitly valuing transferability in the program units (e.g., focusing on transferability as a competency to develop). An additional important strategy to facilitate the transfer of learning was to adapt the provider training to emphasize the potentialities of transferability and explicitly incorporate training strategies to reinforce cross-competencies. Other strategies to facilitate the transfer of learning included proposing a focus on other domains such as practice (e.g., by obtaining input and suggestions from students regarding skills practice scenarios) and new content, such as introducing specific information about new health behaviors and to focus on specific attitudes and beliefs.

TABLE 4

Table 4. Strategies to facilitate the transfer of learning to different health behaviors and a ranking of their importance.

Validation of Categorization

All participants agreed with the list during the second-round: they reported that their own responses were well-categorized (100% rather or very well; mean 4.54; sd 0.51); that the categories were considered clear (100% rather or very well; mean 4.47; sd 0.51); and that the elements included were well-justified (95% rather or very well; mean 4.47; sd 0.61).

Ranking Relevant Factors

The strategies identified above were evaluated by participants, and then ranked by importance. The results are reported in Table 4. All strategies were considered important, with a minimum mean of 8.05 and a maximum of 9.74. However, the ranking confirmed what was already found during the classification task.

Discussion

In school health promotion, interventions and programs implemented often focus on single behavior domains. Typically, each health topic is addressed separately, potentially resulting in overlapping interventions. Furthermore, new health behavior problems often arise among students from year to year, requiring a response from school administrators. Rather than implementing multiple programs, it is likely to be more efficient and effective to target multiple behaviors with the same intervention or program. An integrative approach is a promising way to develop MHBC interventions, and in the present study we examined the transfer of learning approach as a strategy for expanding the scope of broad-based life skills education programs. The transfer of learning approach represents a core concept in education and is a major objective for educators and schools (44). We argue that the transfer of learning approach should be prioritized for school health promotion so that students can learn knowledge, attitudes, and skills in one health promotion context or behavioral domain and then learn how to apply them in other contexts or domains (12, 44, 45). The present study aimed to identify the characteristics of an evidence-based life skill education program that can facilitate the transfer of learning to different health behaviors not targeted by the program, the elements that should be reinforced to facilitate the transfer, and the strategies to do so successfully. A Delphi method was used involving experts in school health promotion, life skill education, and teaching. Results revealed strong consensus among experts about the potentialities of the LifeSkills Training program in promoting the transfer of skills from one health domain to another. Many elements already included in the program were identified as facilitators of transfer, and strategies to improve effective transfer were defined and clarified. These results confirm the strong potential of the transfer of learning approach in the health promotion area. Some studies have already been published (12, 44, 45), that verify the effectiveness of the transfer of learning from one health behavior to others. Life skill education has already been proven to be effective to prevent multiple health risk behaviors and promote healthy habits (47, 48), and many MHBC programs have used this strategy (8, 21, 22). However, the conditions required to effectively achieve transfer of learning have not been adequately defined and conceptualized in previous research. The findings from the present study demonstrate for the first time a consensus among experts in the field that life skills education is an adequate and promising approach for MHBC.

Most of the strategies that emerged in this study are related to teaching methods (e.g., reinforce and integrate open questions to conclude the unit or enhance teaching skills), educational approaches (e.g., explicitly valuing transferability in units), and educational consistency (e.g., integrate the life skill education program with daily teaching, school planning and school context). Contents about specific behaviors were cited, but participants suggested that they be integrated into the curriculum or addressed through recontextualization of skills and techniques already taught. Findings suggest that the successful transfer of learning to a new health domain requires that educators recognize its importance and explicitly designate it as an educational aim. These results confirmed that the transfer of learning process should consider not only the content of the program and the behavioral determinants targeted, as suggested by many authors (34–36), but also the implementation conditions. This idea is consistent with the implementation science approach which recognizes that health promotion strategies consist of complex interventions influenced by multifaceted contexts and dynamic conditions. So, the effectiveness of these strategies depends on how they are implemented, the contexts in which they are used, and the targets they reach (74, 75).

Previous studies about the transfer of learning conditions focused mainly on teaching methods and conditions (12, 34, 42, 54, 55). Their relevance was also recognized by the experts involved in this study. For example, the importance of contextualization, decontextualization, and recontextualization was confirmed (12, 34). The relevance of two paths to facilitate transfer was also recognized: the “low road” based on repeated practice and the “high road” promoted by mindful abstraction, metacognition, and deliberate search of connections (42). The study results also increase our knowledge about these teaching methods and conditions, and provide a more in-depth perspective on how teachers can implement them. For example, the “instructional or learning methods” and the “teachers competencies” already described in the LST program represent concrete strategies to promote both low road and high road paths to strengthen the transfer of learning. The “reinforce and integrate the questions to enhance students awareness on learning outcomes and process” strategy is also a practical teaching method to integrate the two paths.

Other factors and strategies identified, such as those related to educational consistency, underlined the importance of the contextual and organizational aspects to promote the transfer of learning. For example, the “coherence with curriculum” or the importance of “integrate LST with daily teaching, school planning, and school context” or of “organize LST units within the context of existing school schedules” were all related to the school organization and the curriculum definition. The categories and subcategories related to the educational approaches showed also the need to define a common and explicit educational perspective that values transferability. Again, the organizational and school community elements were highlighted by experts, which confirms the relevance of a whole-school approach to promote both educational and health outcomes promoted by the Health Promoting School approach (26–29).

There are several strengths in the present study. The use of a qualitative bottom-up approach and an integrative analysis based on the Delphi methodology helped to identify several codes and subcategories and define a rich and in-depth description of factors and strategies to promote the transfer of learning. Moreover, the involvement of experts in the health promotion area from both health educational and health sectors further enhances the elements identified and the consensus achieved among diverse experts enhances the validity of the findings. The present study has some limitations that could be addressed in future studies. A three rounds Delphi study would be more appropriate for better distinguishing the categorization phase from the validation and ranking phase. Moreover, validation could be requested for all subcategories identified. However, a preliminary face-to-face phase prepared and briefed all participants for the study, which served to increased first-round quality and facilitated participants' interactions in the second phase. Furthermore, the second-round of the study allowed all participants to comment and improve others' responses, and validation rates were high. Data saturation was reached in the second-round, and no categories were added, indicating high consensus. Another limitation was that the experts involved all belonged to the same intervention context. However, a broad range of participants from different disciplines, sectors (education and health promotion), organizations, and cultural background were selected.

Conclusions

The present study aimed to identify the conditions necessary to effectively promote transfer of learning in an evidence-based life skill education program to different health behaviors. The use of the Delphi method findings produced a consensus among experts in the health promotion area from both the educational and health sectors. The qualitative analysis demonstrated several key necessary conditions for the transfer of learning in a health promotion context and to define them in depth.

The study confirmed the importance of investigating the transfer of learning in the health promotion area. Future studies should focus on different life skill education programs to generalize results better. Moreover, new studies are needed to evaluate the concrete effectiveness of multi behavior life skill education programs and to verify the concrete transfer of learning from one domain to another. Other Delphi studies could analyze the differences between experts from the educational and health sectors. In this study, the authors decided to explore an integrative approach to reach consensus among different expert perspectives, but an investigation of the differences can also be useful to define effective collaborative strategies.

The results have also practical implications. The identification of elements to facilitate the transfer of learning offers a solution to find a balance between the importance of applying skills to a particular topic to be effective and the need to reinforce young people in different areas of their development (48). The factors identified in this study can be used to adapt several life skill education programs to MHBC or to design new ones. The theories that underlie the LST program and the life skills targeted by it are also observed in many similar programs. Life skills programs, although more holistic in nature, also focus on knowledge, attitudes, and skills for each task or goal and these general principles are required for certain behavioral skills such as problem-solving. Moreover, the conditions identified are related to general teaching methods, curriculum definition, and school organization and could be adapted to the different programs. For example, most life skill education programs reinforce skills through specific tasks or analysis. A transfer of learning approach requires identifying general principles and rules, decontextualizing learning, and improving metacognition skills and mindful abstraction. The results of the study identified specific strategies to integrate these perspectives and practices.

The use of a Delphi study also suggests strategies to adapt programs considering experts, stakeholders and community points of view and to integrate literature guidelines with practice (76–78). Considering that most elements identified were related to educational methods and strategies, the present results can also be used to improve teacher training by promoting actions aimed to promote the transfer of learning and raise awareness of its importance. These results also provide guidance on evidence-based program implementation and integration with a whole school approach. First, the analysis of participants responses illustrates how the perspectives of health and educational experts can be effectively integrated. School staff had the chance to explicitly state their educational perspective, and health professionals were able to integrate health promotion concepts with teaching and pedagogy. Then, results show the importance of the integration of an evidence-based program with organizational and contextual elements and with program providers' representations and beliefs (68, 79–81). To integrate the program into the curriculum and make explicit the educational strategies were found to be the most valued strategies. These factors should be considered when defining the role of health professionals in supporting a school in health promotion program implementation or the HPS approach.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

The studies involving human participants were reviewed and approved by the Regional Committee of Health Promoting School Network, by the Regional Coordinators of the LifeSkills Training adaptation group, institutional review boards who reviewed the study for ethical standards. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

VV managed all the phases of the study. CC revised and discussed the questionnaire design and analysis. KG offered methodological support. Estensione LST group represents the coordinators of the study and the wider project. All authors contributed to the paper and involved in the study design.

Funding

The program LifeSkills Training Lombardia is an initiative of the Region Prevention Plan 2015-19 of the Lombardy Region, Program 2 Health Promoting Schools – Lombardian Health Promoting School Network and Program 7 Regional network of Addiction Prevention (Deliberation 17th of December 2018 – n. XI/1046). The publication fee is covered by the fund 2020-NOECO-0118 of Milano-Bicocca University.

Estensione LST Group

Estensione LST group is made up of the regional staff of the project and the representatives of each organization involved: Regional Staff: Corrado Celata, Veronica Velasco, Francesca Mercuri, Sandro Brasca, Maria Grazia Crispiatico; School: Tommaso Andreano, Patrizia Bestetti, Simona Boffelli, Mara Caenazzo, Maria Concetta De Salvo, Elisabetta Franchini, Luigi Galbiati, Marina Ghislanzoni, Giancarlo Gobbi Frattini, Viviana Malvicini, Nazarena Marinoni, Amelia Molteni, Nadia Mortoni, Margherita Parolini, Umberto Parolini, Elena Pera, Cristina Pirovano, Alessandra Roncoroni, Ileana Sala Tenna, Simona Sala Tenna, Cosimo Scaglione, Alessandra Schiatti, Maria Teresa Tiana; Health Units: Stefania Bellesi, Luca Biffi, Antonella Calaciura, Rossana Di Silvio, Valter Drusetta, Laura Ferretti, Giovanni Fioni, Lidia Frattallone, Elvira Gaia, Paola Ghidini, Nicola Iannaccone, Lisa Impagliazzo, Barbara Lamera, Alessandra Maffioletti, Silvia Maggi, M. Letizia Marchetti, Margherita Marella, Raffaele Pacchetti, Ornella Perego, Giuliana Rocca, Valentina Salinetti, Cinzia Simonetti, Uber Sossi, Stefania Vizzardi, and Paolo Zampiceni.

Conflict of Interest

KG is a consultant to National Health Promotion Associates, Inc., which markets materials for the LifeSkills Training prevention program.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

1. Biglan A, Brennan PA, Foster SL. Helping Adolescents at Risk Prevention of Multiple Problem Behaviors. New York, NY: Guilford Publications (2004).

Google Scholar

2. Davó-Blanes MC, García de la Hera M, La Parra D. Educación para la salud en la escuela primaria: Opinión del profesorado de la ciudad de Alicante. Gac Sanit. (2016) 30:31–6. doi: 10.1016/j.gaceta.2015.07.008

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Miglioretti M, Velasco V, Celata C, Vecchio L. Teachers' ideas about health: implications for health promotion at school. Health Educ J. (2013) 72:695–707. doi: 10.1177/0017896912460929

CrossRef Full Text | Google Scholar

4. Tani F, Ponti L, Ghinassi S, Smorti M. A gambling primary prevention program for students through teacher training: an evidence-based study. Int Gambl Stud. (2021) 2020:1861056. doi: 10.1080/14459795.2020.1861056

CrossRef Full Text | Google Scholar

5. Romero Saletti SM, Van den Broucke S, Chau C. The effectiveness of prevention programs for problematic internet use in adolescents and youths: a systematic review and meta-analysis. Cyberpsychology. (2021) 15:10. doi: 10.5817/CP2021-2-10

CrossRef Full Text | Google Scholar

6. Nelson L, Roots K, Dunn TJ, Rees A, Hull DD, Van Gordon W. Effects of a regional school-based mindfulness programme on students' levels of well-being and resiliency. Int J Spa Wellness. (2021) 2021:1–15. doi: 10.1080/24721735.2021.1909865

CrossRef Full Text | Google Scholar

7. Converso D, Cortini M, Guidetti G, Molinengo G, Sottimano I, Viotti S, et al. Organizational climate and teachers' morale: developing a specific tool for the school context – a research project in Italy. Front Psychol. (2019) 2019:2132. doi: 10.3389/fpsyg.2019.02132

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Hale DR, Fitzgerald-Yau N, Viner RM A. systematic review of effective interventions for reducing multiple health risk behaviors in adolescence. Am J Public Health. (2014) 104:19–42. doi: 10.2105/AJPH.2014.301874

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Velasco V, Miglioretti M, Celata C, Vecchio L. Il benessere degli insegnanti: il ruolo del supporto sociale e delle dimensioni organizzative. Psicol DELLA Salut. (2013) 2:52–70. doi: 10.3280/PDS2013-002005

CrossRef Full Text | Google Scholar

10. Hargreaves A. Teaching in the Knowledge Society : Education in the Age of Insecurity. Milton Keynes: Open University Press (2003).

PubMed Abstract | Google Scholar

11. Lindqvist P, Nordänger UK. Who dares to disconnect in the age of uncertainty? Teachers' recesses and ‘off-the-clock' work. Teach Teach. (2007) 12:623–37. doi: 10.1080/13540600601029637

CrossRef Full Text | Google Scholar

12. Peters LWH. Searching for Similarities: Transfer-Oriented Learning in Health Education at Secondary Schools. Amsterdam: University of Amsterdam (2012).

Google Scholar

13. Busch V, Van Stel HF, Schrijvers AJ, De Leeuw JR. Clustering of health-related behaviors, health outcomes and demographics in Dutch adolescents: a cross-sectional study. BMC Public Health. (2013) 13:1118. doi: 10.1186/1471-2458-13-1118

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Mawditt C, Sacker A, Britton A, Kelly Y, Cable N. The clustering of health-related behaviours in a British population sample: testing for cohort differences. Prev Med. (2016) 88:95–107. doi: 10.1016/j.ypmed.2016.03.003

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Vecchio L, Velasco VMM. Dai comportamenti di salute all'individuazione di tipologie di condotte. I dati della ricerca HBSC come supporto per la progettazione di interventi sulla salute. In: OReD, editor, Generazione 2010: comportamenti di salute, contesti di vita e livelli di benessere degli studenti lombardi Indagine Health Behaviour in School-aged Children (HBSC) Lombardia 2009-2010: stili di vita e salute degli studenti di 11, 13 e 15 anni Rapporto. Milan: Éupolis Lombardia (2013). p. 227–48.

16. Lawrence EM, Mollborn S, Hummer RA. Health lifestyles across the transition to adulthood: implications for health. Soc Sci Med. (2017) 193:23–32. doi: 10.1016/j.socscimed.2017.09.041

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Ahmadi-Montecalvo H. Adolescent Health Risk Behaviors: An Examination of the Co-occurrence of Risk Behaviors in a National Sample of U.S. High School Adolescents. West Virginia: University Morgantown (2016).

Google Scholar

18. Catalano RF, Berglund ML, Ryan JAM, Lonczak HS, Hawkins JD. Positive youth development in the united states: research findings on evaluations of positive youth development programs. Ann Am Acad Pol Soc Sci. (2004) 591:98–124. doi: 10.1177/0002716203260102

CrossRef Full Text | Google Scholar

19. Prochaska JJ, Spring B, Nigg CR. Multiple health behavior change research: an introduction and overview. Prev Med. (2008) 46:181–8. doi: 10.1016/j.ypmed.2008.02.001

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Hale DR, Viner RM. The correlates and course of multiple health risk behaviour in adolescence. BMC Public Health. (2016) 16:1–12. doi: 10.1186/s12889-016-3120-z

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Botvin GJ, Griffin KW. Toward the development of preventive interventions to reduce HRSB, HIV/AIDS, and multiple problem behaviors. Prev Sci. (2014) 15:81–3. doi: 10.1007/s11121-014-0461-7

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Reider EE, Robertson EB, Sims BE. Does early intervention prevent health-risking sexual behaviors related to HIV/AIDS? Prev Sci. (2014) 15:S1. doi: 10.1007/s11121-013-0455-x

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Meader N, King K, Wright K, Graham HM, Petticrew M, Power C, et al. Multiple Risk Behavior Interventions: meta-analyses of RCTs. Am J Prev Med. (2017) 53:e19–30. doi: 10.1016/j.amepre.2017.01.032

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Johnson SS, Paiva AL, Mauriello L, Prochaska JO, Redding CA, Velicer WF. Coaction in multiple behavior change interventions: consistency across multiple studies on weight management & obesity prevention. Health Psychol. (2014) 33:475. doi: 10.1037/a0034215

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Busch V, de Leeuw JRJ, de Harder A, Schrijvers AJP. Changing multiple adolescent health behaviors through school-based interventions: a review of the literature. J Sch Health. (2013) 83:514–23. doi: 10.1111/josh.12060

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Vilaça T, Darlington E, Miranda Velasco MJ, Martinis O, Masson J. SHE School Manual 2.0. A Methodological Guidebook to Become a Health Promoting School. Haderslev, DK: Schools for Health in Europe Network Foundation (2019).

Google Scholar

27. Bartelink N, Bessems K. Health Promoting Schools in Europe State of the Art. Haderslev: Schools for Health in Europe Network Foundation (2019).

28. Dadaczynski K, Jensen BB, Viig NG, Sormunen M, von Seelen J, Kuchma V, et al. Health, well-being and education: building a sustainable future. The Moscow statement on Health Promoting Schools. Health Educ. (2020) 120:11–9. doi: 10.1108/HE-12-2019-0058

CrossRef Full Text | Google Scholar

29. Safarjan E, Buijs G, de Ruiter S. SHE Online School Manual: 5 Steps to a Health Promoting School. Utrecht: CBO. (2013).

Google Scholar

30. Vandelanotte C, Reeves M, Brug J, De Bourdeaudhuij I. A randomized trial of sequential and simultaneous multiple behavior change interventions for physical activity and fat intake. Prev Med. (2008) 46:232–7. doi: 10.1016/j.ypmed.2007.07.008

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Catalano R, Hawkins J, Berglund M, Pollard J, Arthur M. Prevention science and positive youth development: competitive or cooperative frameworks? J Adolesc Health. (2002) 31(6Suppl.):230–9. doi: 10.1016/S1054-139X(02)00496-2

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Flay B. Positive youth development requires comprehensive health promotion programs. Am J Health Behav. (2002) 26:407–24. doi: 10.5993/AJHB.26.6.2

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Greenberg M, Weissberg R, O'Brien M, Zins J, Fredericks L, Resnik H, et al. Enhancing school-based prevention and youth development through coordinated social, emotional, and academic learning. Am Psychol. (2003) 58:466–74. doi: 10.1037/0003-066X.58.6-7.466

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Paulussen T, Panis R, Peters L, Buijs G, Wijnsma P. Stand van zaken schoolgezondheidsbeleid in Nederland: Een inventariserend onderzoek. Woerden: NIGZ Netherlands Institute for Health Promotion and Disease Prevention. (1998).

35. Peters LWH, Wiefferink CH, Hoekstra F, Buijs GJ, Ten Dam GTM, Paulussen TGWM, et al. review of similarities between domain-specific determinants of four health behaviors among adolescents. Health Educ Res. (2009) 24:198–223. doi: 10.1093/her/cyn013

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Wiefferink CH, Peters L, Hoekstra F, Ten Dam G, Buijs GJ, Paulussen TGWM. Clustering of health-related behaviors and their determinants: possible consequences for school health interventions. Prev Sci. (2006) 7:127–49. doi: 10.1007/s11121-005-0021-2

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Barnett SM, Ceci SJ. When and where do we apply what we learn? A taxonomy for far transfer. Psychol Bull. (2002) 128:612–37. doi: 10.1037/0033-2909.128.4.612

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Beach K. Consequential transitions: a sociocultural expedition beyond transfer in education. Rev Res Educ. (1999) 24:101. doi: 10.2307/1167268

CrossRef Full Text | Google Scholar

39. Campione J, Shapiro A, Brown A. Forms of transfer in a community of learners: flexible learning and understanding. In: McKeough A, Lupart J, Marini A, editors, Teaching for Transfer: Fostering Generalization in Learning. New Jersey: Lawrence Erlbaum (1995). p. 35–68.

Google Scholar

40. Lobato J. Alternative perspectives on the transfer of learning: history, issues, and challenges for future research. J Learn Sci. (2006) 15:431–49. doi: 10.1207/s15327809jls1504_1

CrossRef Full Text | Google Scholar

41. Marini A, Genereux R. The challenge of teaching for transfer. In: McKeough A, Lupart J, Marini A, editors, Teaching for Transfer: Fostering Generalization in Learning. New Jersey: Lawrence Erlbaum Associates (1995). p. 1–33.

Google Scholar

42. Perkins D, Salamon G. Transfer of learning. In: Husen T, Postlethwaite T, editors, International Encyclopedia of Education. 2nd ed. Oxford: Pergamon (1992). p. 1–13.

Google Scholar

43. Tuomi-Gröhn T, Engeström Y. Between School and Work: New Perspectives on Transfer and Boundary-Crossing. Boston, MA: Pergamon. (2003).

Google Scholar

44. Peters LH, Ten Dam GTM, Kocken PL, Buijs GJ, Dusseldorp E, Paulussen TGWM. Effects of transfer-oriented curriculum on multiple behaviors in the Netherlands. Health Promot Int. (2015) 30:291–309. doi: 10.1093/heapro/dat039

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Young I, St Leger LGB. School Health Promotion: Evidence for Effective Action. Background Paper SHE Factsheet 2. Utrecht: CBO (2013).

Google Scholar

46. Geller K, Lippke S, Nigg CR. Future directions of multiple behavior change research. J Behav Med. (2017) 40:194–202. doi: 10.1007/s10865-016-9809-8

PubMed Abstract | CrossRef Full Text | Google Scholar

47. World Health Organization. Life Skills Education in Schools. Geneva: World Health Organization (1994).

48. World Health Organization. Skills for Health. Geneva: World Health Organization (2003).

49. Jessor RJS. Problem Behavior and Psychosocial Development: a Longitudinal Study of Youth. New York, NY: Academic Press. (1977).

50. Bandura A. Social Learning Theory. New York, NY: Academic Press. (1977).

Google Scholar

51. World Health Organization. Global Action Plan for the Prevention and Control of NCDs 2013-2020. Geneva: World Health Organization (2013).

Google Scholar

52. World Health Organization. Health Promoting School: An Effective Approach for Early Action on NCD Risk Factors. Geneva: World Health Organization (2017).

53. World Health Organization. Life Skills Education School Handbook: Prevention of Noncommunicable Diseases. Geneva: World Health Organization (2020).

Google Scholar

54. Bransford J, Brown A, Cocking R. How People Learn—Brain, Mind, Experience, and School, Expanded Edition. Washington, DC: National Academies Press (2000).

Google Scholar

55. Säljö R. Epilogue: from transfer to boundary-crossing. In: Tuomi-Gröhn T, Engeström Y, editors, Between School and Work: New Perspectives on Transfer and Boundary Crossing. Amsterdam: Pergamon (2003). p. 311–21.

56. Brown B. Delphi Process: A Methodology Used for the Elicitation of Opinions of Experts. Santa Monica, CA: RAND Corporation (1968).

Google Scholar

57. Dalkey N, Helmer O. An experimental application of the DELPHI method to the use of experts. Manage Sci. (1963) 9:458–67. doi: 10.1287/mnsc.9.3.458

CrossRef Full Text | Google Scholar

58. Hasson F, Keeney S, McKenna H. Research guidelines for the Delphi survey technique. J Adv Nurs. (2000) 32:1008–15. doi: 10.1046/j.1365-2648.2000.t01-1-01567.x

PubMed Abstract | CrossRef Full Text | Google Scholar

59. McKenna HP. The Delphi technique: a worthwhile research approach for nursing? J Adv Nurs. (1994) 19:1221–5. doi: 10.1111/j.1365-2648.1994.tb01207.x

PubMed Abstract | CrossRef Full Text | Google Scholar

60. Okoli C, Pawlowski SD. The Delphi method as a research tool: an example, design considerations and applications. Inf Manag. (2004) 42:15–29. doi: 10.1016/j.im.2003.11.002

CrossRef Full Text | Google Scholar

61. Pollard C, Pollard R. Research priorities in educational technology: a Delphi study. J Res Technol Educ. (2004) 37:145–60. doi: 10.1080/15391523.2004.10782430

CrossRef Full Text | Google Scholar

62. Sinha IP, Smyth RL, Williamson PR. Using the delphi technique to determine which outcomes to measure in clinical trials: recommendations for the future based on a systematic review of existing studies. PLoS Med. (2011) 8:1000393. doi: 10.1371/journal.pmed.1000393

PubMed Abstract | CrossRef Full Text | Google Scholar

63. van Urk F, Grant S, Bonell C. Supplemental materials: involving stakeholders in programme theory specification: discussion of a systematic, consensus-based approach. Evid Policy A J Res Debate Pract. (2018) 12:1332. doi: 10.1332/174426415X14474260456850

CrossRef Full Text

64. Whitehead D. An international Delphi study examining health promotion and health education in nursing practice, education and policy. J Clin Nurs. (2008) 17:891–900. doi: 10.1111/j.1365-2702.2007.02079.x

PubMed Abstract | CrossRef Full Text | Google Scholar

65. Schmidt R, Lyytinen K, Keil M, Cule P. Identifying software project risks: an international Delphi study. J Manag Inf Syst. (2001) 17:5–36. doi: 10.1080/07421222.2001.11045662

CrossRef Full Text | Google Scholar

66. Botvin GJ, Baker E, Dusenbury L, Botvin EM, Diaz T. Long-term follow-up results of a randomized drug abuse prevention trial in a white middle-class population. J Am Med Assoc. (1995) 273:1106–12. doi: 10.1001/jama.273.14.1106

PubMed Abstract | CrossRef Full Text | Google Scholar

67. Botvin GJ, Griffin KW. Life skills training: a competence enhancement approach to tobacco, alcohol, and drug abuse prevention. In: Scheier LM, editor. Handbook of Adolescent Drug Use Prevention: Research, Intervention Strategies, and Practice. Washington, DC: American Psychological Association (2015). p. 177–96. doi: 10.1037/14550-011

CrossRef Full Text | Google Scholar

68. Velasco V, Griffin KW, Antichi M, Celata C A. large-scale initiative to disseminate an evidence-based drug abuse prevention program in Italy: lessons learned for practitioners and researchers. Eval Program Plann. (2015) 52:27–38. doi: 10.1016/j.evalprogplan.2015.03.002

PubMed Abstract | CrossRef Full Text | Google Scholar

69. Crispiatico MG, Bestetti P, Velasco V, Celata C, Coppola L, Estensione LST G. La progettazione scolastica orientata alla promozione della salute. Un percorso di dialogo intersettoriale per l'allineamento delle competenze chiave per l'apprendimento e life skill. J Educ Cult Psychol Stud. (2020) 2020:22. doi: 10.7358/ecps-2020-022-cris

CrossRef Full Text | Google Scholar

70. Velasco V, Griffin KW, Botvin GJ, Celata C, Velasco V, Antichi M, et al. Preventing adolescent substance use through an evidence-based program: effects of the italian adaptation of life skills training. Prev Sci. (2017) 18:394–405. doi: 10.1007/s11121-017-0776-2

PubMed Abstract | CrossRef Full Text | Google Scholar

71. Griffin KW, Botvin GJ, Nichols TR. Long-term follow-up effects of a school-based drug abuse prevention program on adolescent risky driving. Prev Sci. (2004) 5:207–12. doi: 10.1023/B:PREV.0000037643.78420.74

PubMed Abstract | CrossRef Full Text | Google Scholar

72. Griffin KW, Botvin GJ, Nichols TR. Effects of a school-based drug abuse prevention program for adolescents on HIV risk behavior in young adulthood. Prev Sci. (2006) 7:103–12. doi: 10.1007/s11121-006-0025-6

PubMed Abstract | CrossRef Full Text | Google Scholar

73. Graneheim UH, Lundman B. Qualitative content analysis in nursing research: concepts, procedures and measures to achieve trustworthiness. Nurse Educ Today. (2004) 24:105–12. doi: 10.1016/j.nedt.2003.10.001

PubMed Abstract | CrossRef Full Text | Google Scholar

74. Pawson R, Greenhalgh T, Harvey G, Walshe K. Realist review - a new method of systematic review designed for complex policy interventions. J Heal Serv Res Pol. (2005) 10(suppl.1):21–34. doi: 10.1258/1355819054308530

PubMed Abstract | CrossRef Full Text | Google Scholar

75. Rycroft-Malone J, McCormack B, Hutchinson AM, DeCorby K, Bucknall TK, Kent B, et al. Realist synthesis: illustrating the method for implementation research. Implement Sci. (2012) 7:1–10. doi: 10.1186/1748-5908-7-33

PubMed Abstract | CrossRef Full Text | Google Scholar

76. August GJ, Gewirtz A, Realmuto GM. Moving the field of prevention from science to service: integrating evidence-based preventive interventions into community practice through adapted and adaptive models. Appl Prev Psychol. (2010) 14:72–85. doi: 10.1016/j.appsy.2008.11.001

CrossRef Full Text | Google Scholar

77. Backer T. Finding the Balance: Program Fidelity and Adaptation in Substance Abuse Prevention: A State-of-the-Art Review. Rockville, MD: Center for Substance Abuse Prevention (2001).

78. Evans RE, Craig P, Hoddinott P, Littlecott H, Moore L, Murphy S, et al. When and how do “effective” interventions need to be adapted and/or re-evaluated in new contexts? The need for guidance. J Epidemiol Community Health. (2019) 73:481–2. doi: 10.1136/jech-2018-210840

PubMed Abstract | CrossRef Full Text | Google Scholar

79. Backer TE. The failure of success: challenges of disseminating effective substance abuse prevention programs. J Community Psychol. (2000) 28:363–73. doi: 10.1002/(SICI)1520-6629(200005)28:3<363::AID-JCOP10>3.0.CO;2-T;2-T

CrossRef Full Text | Google Scholar

80. Miller RL, Shinn M. Learning from communities: overcoming difficulties in dissemination of prevention and promotion efforts. Am J Community Psychol. (2005) 35:169–83. doi: 10.1007/s10464-005-3395-1

PubMed Abstract | CrossRef Full Text | Google Scholar

81. Wandersman A, Duffy J, Flaspohler P, Noonan R, Lubell K, Stillman L, et al. Bridging the gap between prevention research and practice: the interactive systems framework for dissemination and implementation. Am J Community Psychol. (2008) 41:171–81. doi: 10.1007/s10464-008-9174-z

PubMed Abstract | CrossRef Full Text | Google Scholar