Almost Everything We Need to Better Serve Children of the Opioid Crisis We Learned in the 80s and 90s

Opioid use disorder impedes dependent parents' abilities to care for their children. In turn, children may languish in unpredictability and persistent chaos. Societal responses to these children are often guided by a belief that unless the drug dependent parent receives treatment, there is little help for the child. While a preponderance of the drug dependence research is adult-centric, a significant body of research demonstrates the importance of not only addressing the immediate well being of the children of drug dependent caregivers but preventing the continuing cycle of drug dependence. The present commentary demonstrates through a brief review of the US history of drug dependence crises and research from the 1980s and 1990s, a range of “tried and true” family, school, and community interventions centered on children. We already know that these children are at high risk of maladjustment and early onset of drug dependence; early intervention is critical; multiple risk factors are likely to occur simultaneously; comprehensive strategies are optimal; and multiple risk-focused strategies are most protective. Where we need now to turn our efforts is on how to effectively implement and disseminate best practices, many of which we learned in the 1980s and 1990s. The greatest opportunity in both changing the nature of the opioid epidemic at scale and influencing rapid translation of existing research findings into policy and practice is not in asking what to do, but in asking how to do the right things well, and quickly.

Introduction

The United States has a long history of drug dependence dating back to its origin, including several opioid epidemics in the last two centuries (1, 2). While it is distinct in some ways, today's opioid epidemic is an echo of past illicit drug crises in that children and youth are vulnerable. In the current epidemic, babies are often born on opioids, resulting from their mother's opioid dependence. Anguished cries and difficulty sleeping and feeding/nursing in the crucial first days of life mark their own withdrawal, termed Neonatal Abstinence Syndrome (NAS) (3). In a study of 28 states, the incidence of NAS increased almost 300% from 1999–2013, rising from 1.5 to 6.0 cases per 1000 hospital births. But, the current situation in some states is much worse. For example, the incidence of NAS in WV is 51 per 1000 live births; at least two counties in WV reported more than 100 cases per 1000 live births (3).

Children of parents with opioid use disorder (OUD), when discovered, are often placed with grandparents, other family members, or enter the foster-care system. Children remain in the homes of their drug dependent parents or caregivers and are at great risk for neglect, abuse, or worse.(4, 5) In the 1970's opioid epidemic, we learned that many depressed and anxious children resort to silence, upholding a fundamental family rule: no one talks about the problem, not to each other, and especially not to outsiders (4). Living in the context of this perceived imperative, a majority of these children exist in emotional incarceration where they are unable to reach out to others for support (5, 6). It is not surprising that we don't grasp the magnitude and number of affected children. As stated by Carol Levine, a MacArthur Fellow and renowned expert on the traumatic impact of epidemics on children, “No one knows how many of these vulnerable children there are in the US because no one is counting. They remain hidden in families with addiction until a crisis erupts and law enforcement or child welfare agencies get involved” (7). Such family dynamics hold true in the current opioid epidemic.

Societal aid for affected children is often guided by a belief that unless the drug dependent parent receives treatment, there is little help for the child. This view is debilitating because most opioid users never seek treatment. In turn, an entire family system remains frozen in uncertainty until the addicted individual accepts treatment, is forced into treatment, becomes justice-involved, is hospitalized, or dies. To understand how we arrived at this place, it is important to revisit the history of drug dependence in the US.

The US is no stranger to drug epidemics (8, 9). In the 1970s it was heroin and hallucinogens. The 1980s drug crisis shifted to cocaine and crack, and in the 1990s crystal methamphetamine was the drug of focus. Today, despite comprising < 5% of the world's population, the US consumes over 80% of the global opioid supply (10) with 5% experiencing dependence (10). Historically, the US response to drug crises prioritizes policy, legal action and criminalization, adult treatment, and relapse prevention. These actions are manifest in the 1956 Narcotic Control Act, the Controlled Substances Act of 1970, the formation of the DEA as a part of our “War on Drugs” in 1973; and the first Tobacco Industry Lawsuit in 1988 (11). As drug dependence continued to increase in the 1970s, and throughout the 1980s and 1990s, research on the causes and consequences of dependence also increased. The National Institute of Drug Abuse (NIDA) formed in 1974 and became a formal part of National Institutes of Health (NIH) in 1992 (9). The federal government also created the Alcohol, Drug Abuse, and Mental Health Agency (ADAMHA; now the Substance Abuse and Mental Health Services Administration SAMHSA) in 1992 to increase availability of resources for prevention and treatment of substance use and mental health disorders (12).

While a preponderance of the drug dependence research is adult-centric, a significant body of research demonstrates the importance of not only addressing the immediate well being of the children of drug dependent caregivers but preventing the continuing cycle of drug dependence. In fact, as touted by NIDA, by the late 1980s scientists were well on their way to building “… the scientific base for effective drug dependence prevention programs by identifying individual, family, school, and neighborhood factors that place children at risk for drug dependence. Researchers have since developed a broad array of effective family, school, and community programs that target these factors” (9).

Discussion

What We Already Know About Children of Drug Dependent Parents

Children Are at High Risk of Maladjustment and Early Onset of Drug Dependence

These children are likely to develop affective, behavioral, cognitive, and interpersonal adverse childhood experiences (ACEs) that manifest at home, school, and in peer groups (13). See examples summarized in Table 1. Children of opioid dependent parents or caregivers experience diminished levels of self-esteem and use isolation or withdrawal to cope (4, 14, 15). Moreover, they tend to normalize the dysfunction in their homes, and display poor interpersonal and social skills, low social attachment, and favorable attitudes toward drug dependence (16–18). Evidence from previous opioid crises tells us that these children are at risk for a variety of psychiatric disorders, including eventual drug dependence (17, 19, 20). Recent findings from longitudinal studies on ACEs underscore the profound impact of parental drug dependence on children's behavior and mental health (21).

TABLE 1

Table 1. Examples multi-level risk factors (13).

Early Intervention Is Critical

Early detection and treatment is important to minimize the potential for enduring functional impairment and suffering (17, 19, 20). Childhood development research on resiliency (22, 23) shows interventions that reduce exposure to risk and adversity will: (1) improve competencies, personal resources, and coping skills; (2) prevent or reduce early onset of maladaptive behavior; and (3) target specific groups who show a higher probability of developing high risk behaviors than the general population (24). Applicable today, early intervention with children of parents with OUD can moderate their responses to known risk factors, eliminating or buffering later risk factors (22, 23).

Multiple Risk Factors Are Likely

The presence of one risk factor or stressor occurring in isolation may not increase the likelihood that a child develops adjustment problems (25). But, when two or more stressors co-occur, the chances are two to four times greater that the child will develop significant problems (25). Fitting within this framework of cumulative risks, effective prevention for these children requires a focus on multiple risk factors as it is unlikely that children of individuals with OUD experience to risk factors singularly (26, 27) Refer to Table 1.

Comprehensive Strategies Are Optimal

Many drug dependence prevention programs for children occur in general settings where large groups of youth congregate (e.g., school classrooms). We know that these universal programs will be minimally effective for children affected by parental OUD because they: (1) are didactic; (2) assume comparable levels of risk for all children; and (3) do not address psychological, behavioral, cognitive, and interpersonal risk factors characteristic of children of drug dependent parents (28). Information-focused sessions presume that children make decisions about health or risky behavior in a values-expectancy framework (i.e., if a child values health then they will engage in healthy behavior). This approach ignores the complexity of the social, physiological, developmental, and external realities of their lives. Additionally, universal programs typically give little consideration to other factors such as cultural and social perspectives, socioeconomic levels, or specific sub-populations (16, 29, 30). This assumption does not suggest that prevention should preclude the use of broad prevention interventions. Researchers and practitioners must follow the evidence showing the importance of selective or indicated interventions for children affected by OUD consisting of appropriate content, intensity, and dosage (26, 31). The most effective interventions will target the biological, psychological, or social risk factors.

Multiple Risk-Focused Strategies Are Most Protective

Comprehensive “integrated delivery systems, long-term continuity of services, coupling of child development and family services, parent education, and inclusion of prenatal care….” increase the chances of positive cumulative impact (p.7) (31, 32). A notion of a cumulative strategies approach for children of drug dependent parents aligns with selected or indicated interventions. More specifically, interventions should consider the co-occurring delivery of at least two of five basic prevention strategies (27) inclusive of affective information, social and life skills training, alternative play activities, academic support networks, and healthy lifestyle mentoring. A comprehensive approach for children affected by parental OUD suggests that effective prevention strategies will: (1) focus on younger children; (2) offer specific risk-focused rather than generalized interventions; (3) address a range of behavioral, psychological, and cognitive risk factors that are common to children from families coping with OUD, and (4) consist of multiple components that target two or more risk factors together (27).

Selective interventions developed and tested in the 1980s, 1990s, and into the 2000s show evidence in support of approaches designed to decrease risk and increase protective factors involving one or domains of individual, peer, family, school, and community (16). Several such models exist in the literature: The Family Management Model of Adolescent Substance Abuse (14); The Intensive Family Prevention Services Model (33); The Social Ecological Model of Adolescent Substance Abuse (34); Communities that Care (35); Life Skills (36, 37); Strengthening Families (34); Focus on Kids (38); Multisystemic Therapy (39), and PROSPER (40), to name a few. Many of these evidence-based programs appear in repositories such as the SAMHSA Evidence-Based Practice Resource Center (URL: https://www.samhsa.gov/ebp-resource-center), Social Programs that Work (URL: http://evidencebasedprograms.org), and the Coalition for Evidence-based Policy (URL: http://toptierevidence.org). Bearing utility for practitioners, including pediatricians and family medicine physicians working with children of the current opioid crisis, these websites provide information in lay-person language appropriate for local—or regional level tailoring and implementation, including key outcomes, effectiveness across a variety of settings, and costs for implementation.

As with the epidemics of the 1980s and 1990s, schools are the primary place where children receive prevention services (41). Evidence shows that school counselors can have integral roles in the lives of children of drug dependent parents. Important for children affected by parental OUD, professional school counselors are better trained to identify and intervene than they were in the previous century, in large part due to the standardization of training through accreditation standards by the Council on the Accreditation of Counseling and Related Educational Programs (CACREP, URL: http://www.cacrep.org/wp-content/uploads/2018/05/2016-Standards-with-Glossary-5.3.2018.pdf). CACREP standards require training in “the theories and etiology of addictions and addictive behaviors” and “evidence-based counseling strategies and techniques for prevention and intervention.”

What We Don't Know and How the Gaps Can Inform Rapid Results Research

Where we are falling short is understanding of how to implement and disseminate our best practices, many of which we learned in the 1980s and 1990s. Perhaps the greatest opportunity in both changing the nature of the opioid epidemic at scale and influencing rapid translation of research into policy and practice is not in asking what to do, but in asking how to do the right things well, and quickly. There are many effective tools to fight the epidemic as explained by Mathis et al. in a study outlining effective approaches all along the continuum of OUD (42). It is clear that effective prevention programming pays the most dividends for return on investment, with as many as $15 saved for every dollar spent on primary prevention for drug dependence (43). Part of the challenge is in how we compel people to adopt these tools. Since children are typically in school, school-based programming is an obvious starting point.

Several key questions should be considered. First, how do we persuade educational institutions to prioritize evidence-based prevention programming delivered by trained facilitators when federal and state pressures on school systems are largely focused on academics and not health? Second, how do we make large scale prevention programs sustainable after grant funding ends? Primary prevention programming is often subject to the vagaries of grant funding. Unfortunately, grants do not always go to the places that need them the most. For prevention programming to be sustainable, the stakeholders must find clear value in the programming, and there must be funding mechanisms to support it (44). Third, how do we tailor interventions to be scalable and efficient? Effective primary prevention programming is ideally theory-based, made up of myriad skill-developing activities with peers, and with a high expectation that the facilitator will deliver the intervention as intended (i.e., fidelity) (45). Fourth, how does adoption of excellent prevention programming become routinized over creation and evaluation of new programs, which is costly, time consuming and uncertain on efficacy? Entire academic careers can focus on crafting novel and effective interventions. But these same interventions, despite the strength of evidence of effectiveness, are not disseminated to the populations that need them the most. Rotheram-Borus and colleagues evoke a disruption in intervention dissemination, calling for simpler and better programs that are more market-oriented and responsive to the public's needs than current practices (46). Finally, how will NAS-affected children experience development and what will their experience be in school? An emerging literature on this topic indicates that NAS-affected school-age children present with significant problems that will affect their development and academic progress, and will cause significant deficits in learning and knowledge by the time they enter high school (47). Further, the areas of the country most affected by the long-standing prescription drug abuse epidemic that now includes other opiates like heroin and fentanyl, are already under-resourced for children that may need individualized education plans. These issues also beg for the engagement of pediatricians and family medicine providers in prevention interventions as early as possible. For instance, providers can receive training to provide screening and brief intervention among their patients, particularly adolescent and young adult female patients, as reproductive-age women are increasingly a focal point of the opioid crisis (10). Trained providers are also in positions to detect NAS symptoms when they may have been otherwise missed. At a minimum, pediatricians and family medicine providers have opportunities unique other providers to jointly educate children and families about OUD, including safe storage practices. All of these primary and secondary prevention efforts have demonstrated efficacy (13).

Having knowledge about the effectiveness of these programs or interventions does not mean that we shouldn't seek new knowledge. Of course, we should. But we should also focus on the best methods to improve the Reach, Adoption, Effectiveness, Implementation, and Maintenance (REAIM) of evidence-based practices and approaches (48). This simple but powerful tool, outlining key tenets of implementation science, can help guide our work and improve health outcomes for locally and nationally (49).

In conclusion, as evidenced by the rich citations in this commentary, deliberately cited from the 1980s and 1990s, there is much to gain by seeking knowledge from the archives and putting it to work for these children now. We simply don't have time to reinvent the wheel. In fact, there is not a moment to lose in this grand challenge.

Author Contributions

KA conceived the topic of focus and wrote significant portions of the manuscript. RP, RT, and GL contributed in writing to sections of the manuscript. All authors provided final review and approval of the full manuscript.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

References

1. Hari J. Chasing the Scream: The First and Last Days of the War on Drugs. 1st US. ed. New York, NY: Bloomsbury (2015).

2. Tracy SW, Acker CJ. Altering American Consciousness: The History of Alcohol and Drug Use in the United States, 1800-2000. Amherst, MA: UMass Press (2004).

Google Scholar

3. Ko JY, Patrick SW, Tong V, Patel R, Lind JN, Barfield WD. Incidence of Neonatal Abstinence Syndrome - 28 States, 1999–2013. MMWR Morb Mortal Wkly Rep. (2016). doi: 10.15585/mmwr.mm6531a2

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Anderson EE, Quast W. Young children in alcoholic families: a mental health needs-assessment and an intervention/prevention strategy. J Prim Prev. (1983). doi: 10.1007/BF01325438

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Black C. Alcohol education: children of alcoholics. Prairie Rose (1980) 59:15.

PubMed Abstract | Google Scholar

6. Black C, Bucky SF, Wilder-Padilla S. The interpersonal and emotional consequences of being an adult child of an alcoholic. Int J Addict. (1986). doi: 10.3109/10826088609063451

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Levine C. The Statistics Don't Capture the Opioid Epidemic's Impact on Children. STAT (2018).

8. Bukoski WJ. A framework for drug abuse prevention research. NIDA Res Monogr. (1991).

PubMed Abstract | Google Scholar

9. DuPont RL. Reflections on the early history of National Institute on Drug Abuse (NIDA): implications for today. J Drug Issues (2009). doi: 10.1177/002204260903900101

CrossRef Full Text | Google Scholar

10. Terplan M. Women and the opioid crisis: historical context and public health solutions. Fertil Steril. (2017). doi: 10.1016/j.fertnstert.2017.06.007

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Substance Abuse and Mental Health Services Administration. Office of Applied Studies. Drug Abuse Warning Network Annual Medical Examiner Data, in Drug Abuse Warning Network Series D. Rockville, MD: Department of Health and Human Services (1997).

12. Substance Abuse and Mental Health Services Administration. In: Substance Abuse Treatment and Domestic Violence. Rockville, MD: Substance Abuse and Mental Health Services Administration (1997). p. 5.

13. Centers for Disease Control and Prevention (CDC). Adverse Childhood Experiences (ACEs), Journal Articles by Topic Area (2018). Available online at: https://www.cdc.gov/violenceprevention/acestudy/journal.html (Accessed August 29, 2018).

14. Dishion TJ, Patterson GR, Reid JR. Parent and peer factors associated with drug sampling in early adolescence: implications for treatment. NIDA Res Monogr. (1988).

PubMed Abstract | Google Scholar

15. Goldman BM, Rossland S. Young children of alcoholics: a group treatment model. Soc Work Health Care (1992). doi: 10.1300/J010v16n03_06

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Hawkins JD, Catalano RF, Miller JY. Risk and protective factors for alcohol and other drug problems in adolescence and early adulthood: implications for substance abuse prevention. Psychol Bull. (1992). doi: 10.1037/0033-2909.112.1.64

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Weintraub SA. Children and adolescents at risk for substance abuse and psychopathology. Int J Addict. (1990).

PubMed Abstract | Google Scholar

18. Bays J. Substance-abuse and child-abuse - impact of addiction on the child. Pediatr Clin N Am. (1990). doi: 10.1016/S0031-3955(16)36941-3

PubMed Abstract | CrossRef Full Text | Google Scholar

19. de Cubas MM, Field T. Children of methadone-dependent women: developmental outcomes. Am J Orthopsychiatry (1993). doi: 10.1037/h0079429

CrossRef Full Text | Google Scholar

20. Wilens TE, Biederman J, Kiely K, Bredin E, Spencer TJ. Pilot study of behavioral and emotional disturbances in the high-risk children of parents with opioid dependence. J Am Acad Child Adolesc Psychiatry (1995). doi: 10.1097/00004583-199506000-00019

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Choi NG, DiNitto DM, Marti CN, Choi BY. Association of adverse childhood experiences with lifetime mental and substance use disorders among men and women aged 50+ years. Int Psychogeriatr (2017). doi: 10.1017/S1041610216001800

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Bernard PM, Lindsey CR. Child and adolescent psychiatry placement in specialist psychiatric training: a regional study. Med Educ. (1995). doi: 10.1111/j.1365-2923.1995.tb02822.x

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Takanishi R, DeLeon PH. A head start for the 21st century. Am Psychol. (1994). doi: 10.1037/0003-066X.49.2.120

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Rosenberg MS, Reppucci ND. Primary prevention of child abuse. J Consult Clin Psychol. (1985). doi: 10.1037/0022-006X.53.5.576

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Rutter M. Protective factors in children's responses to stress and disadvantage. Ann Acad Med Singapore (1979).

PubMed Abstract | Google Scholar

26. Substance, Abuse and Mental Health Services Administration. Risk and Protective Factors. Available online at: https://www.samhsa.gov/capt/practicing-effective-prevention/prevention-behavioral-health/risk-protective-factors (Accessed July 31, 2018).

27. Horn K, Kolbo JR. Application of a cumulative strategies model for drug abuse prevention: exploring choices for high risk children. J Drug Educ. (2000). doi: 10.2190/UDC0-6ECT-EQCA-WVTT

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Dielman TE, Butchart AT, Shope JT. Structural equation model tests of patterns of family interaction, peer alcohol use, and intrapersonal predictors of adolescent alcohol use and misuse. J Drug Educ (1993). doi: 10.2190/8YXM-K9GB-B8FD-82NQ

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Elmquist DL. A systematic review of parent-oriented programs to prevent children's use of alcohol and other drugs. J Drug Educ. (1995). doi: 10.2190/XAW9-JV18-J480-4VD7

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Hawkins JD, Catalano RF. Broadening the vision of education: schools as health promoting environments. J Sch Health (1990). doi: 10.1111/j.1746-1561.1990.tb05433.x

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Substance Abuse and Mental Health Services Administration. Reauthorization Issues Facing the Substance Abuse and Mental Health Services Administration. Rockville, MD (1997).

32. Van Hasselt VB, Hersen M, Null JA, Ammerman RT, Bukstein OG, McGillivray J, et al. Drug abuse prevention for high-risk African American children and their families: a review and model program. Addict Behav. (1993). doi: 10.1016/0306-4603(93)90051-A

CrossRef Full Text | Google Scholar

33. Hawkins JD. Lishner DM, Catalano RF. Childhood predictors and the prevention of adolescent substance abuse. NIDA Res Monogr. (1985).

34. Kumpfer KL, Turner CW. The social ecology model of adolescent substance abuse: implications for prevention. Int J Addict (1990).

PubMed Abstract | Google Scholar

35. Hawkins JD. Catalano RF, Arthur MW. Promoting science-based prevention in communities. Addict Behav. (2002). doi: 10.1016/S0306-4603(02)00298-8

CrossRef Full Text

36. Botvin GJ, Griffin KW. Life skills training as a primary prevention approach for adolescent drug abuse and other problem behaviors. Int J Emerg Ment Health (2002).

PubMed Abstract

37. Renick NL, Botvin GJ. Enhancing junior high school students' life skills as a smoking prevention strategy. Health Educ. (1985). doi: 10.1080/00970050.1985.10614471

PubMed Abstract | CrossRef Full Text | Google Scholar

38. ETR Associates. Focus on Kids: Adolescent HIV Risk Prevention. Santa Cruz, CA: ETR Associates (2005). p. 244.

39. Brown TL, Swenson CC, Cunningham PB, Henggeler SW, Schoenwald SK, Rowland MD. Multisystemic treatment of violent and chronic juvenile offenders: bridging the gap between research and practice. Adm Policy Ment Health (1997). doi: 10.1023/A:1022247207249

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Spoth R, Greenberg M, Bierman K, Redmond C. PROSPER community-university partnership model for public education systems: capacity-building for evidence-based, competence-building prevention. Prev Sci. (2004). doi: 10.1023/B:PREV.0000013979.52796.8b

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Kempf C, Llorca PM, Pizon F, Brousse G, Flaudias V. What's new in addiction prevention in young people: a literature review of the last years of research. Front Psychol. (2017) 8:1131. doi: 10.3389/fpsyg.2017.01131

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Mathis SM., Hagemeier N, Hagaman A, Dreyzehner J, Pack RP. A dissemination and implementation science approach to the epidemic of opioid use disorder in the United States. Curr HIV/AIDS Rep. (2018). doi: 10.1007/s11904-018-0409-9

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Miller T. Substance Abuse Prevention Dollars and Cents: A Cost-Benefit Analysis. Rockville, MD: Center for Substance Abuse Prevention, Substance Abuse and Mental Health Services Administration (2008).

Google Scholar

44. Centers for Disease Control and Prevention (CDC). Sustainability Planning Guide (2018). Available online at: https://www.cdc.gov/nccdphp/dch/programs/healthycommunitiesprogram/pdf/sustainability_guide.pdf (Accessed July 31, 2018).

45. Chambers DA. Vinson CA, Norton WE. Advancing the Science of Implementation Across the Cancer Continuum. New York, NY: Oxford University Press (2018).

46. Rotheram-Borus MJ, Swendeman D, Chorpita BF. Disruptive innovations for designing and diffusing evidence-based interventions. Am Psychol. (2012). doi: 10.1037/a0028180

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Oei JL, Melhuish E, Uebel H, Azzam N, Breen C, Burns L, et al. Neonatal abstinence syndrome and high school performance. Pediatrics (2017) 139:e20162651. doi: 10.1542/peds.2016-2651

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health (1999). doi: 10.2105/AJPH.89.9.1322

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Kessler RS, Purcell EP, Glasgow RE, Klesges LM, Benkeser RM, Peek CJ. What does it mean to “employ” the RE-AIM model? Eval Health Prof. (2013). doi: 10.1177/0163278712446066

PubMed Abstract | CrossRef Full Text | Google Scholar