This article is a commentary on:
Adapting and Operationalizing the RE-AIM Framework for Implementation Science in Environmental Health: Clean Fuel Cooking Programs in Low Resource Countries
Ana A. Baumannby Quinn, A. K., Neta, G., Sturke, R., Olopade, C. O., Pollard, S. L., Sherr, K., et al. (2019). Front. Public Health 7:389. doi: 10.3389/fpubh.2019.00389
The field of implementation science has seen an accumulation of theories, models and frameworks in the past years. However, few empirical studies are informed by them (1), and when informed, few clearly describe how they applied the frameworks in the study (2). The study by Quinn et al. (3) provides an exception to this rule and gives us an example of how to use the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) (4) framework in their study of a consortium of 11 sites in low middle income countries (LMIC). Instead of focusing on one study at a time, a consortium can advance the field by having common metrics across different settings, providing an unique opportunity for theory testing [e.g., (5, 6)].
Testing the Boundaries of Construct Definitions
Quinn et al.' study (3) developed a checklist and case studies to evaluate household energy interventions. The results showed that the constructs effectiveness and adoption needed more adaptation in their definitions compared to the other constructs of the framework. Effectiveness, defined as “the impact of an intervention on important outcomes” (7), was hard to gather in their context because health is considered a co-benefit of the programs, and therefore health outcomes and measures of air pollutions are not usually readily available. To address this challenge, the authors adapted the definition to capture “potential” health impact of the stove/fuel, relying on estimated data accrued from stove emissions. We need more empirical studies in different contexts to continue to refine the definition of effectiveness.
Adoption, a construct used to capture the proportion and representativeness of organizations willing to adopt a program (7), was challenging because usually clean fuel cooking programs do not involve an intermediary organization. To address this issue, the authors re-defined the construct to encompass factors at society level (e.g., description of supply chain) as well as household/community factors (e.g., household use of technology). The adoption construct in this case was also difficult because it should refer not only to the uptake of something (i.e., adding a stove) but also to the discontinuation of older stoves who are health-damaging. This discussion is timely as the field starts to understand the unique aspects of de-implementation and how to define them. Accordingly, Prusaczyk et al. (8) suggest expanding the adoption concept to include de-adoption, defined as the intention or initial decision to stop a practice.
The results from Quinn et al. (3) showed that, while Reach was the easier construct to gather data across sites, the definition of reach is challenging in the context of public health programs. As Quinn et al. (3) comment: it is difficult to evaluate “Reach” of an intervention that improves sidewalks. Gaglio et al. (9) recognize the challenges in the definition, which has also been adapted to refer to awareness of a program (10). Finally, Maintenance was hard to capture because the sites were at the beginning of implementation the program. It will be interesting to see how this consortia captures maintenance later on.
Context and LMIC
When stakeholders were asked about their perceived ease and usefulness of employing RE-AIM on their project, they mentioned the challenges in capturing context using RE-AIM, particularly the political and social aspects of the studies. In fact, as May et al. (11) state: “context is a problem in implementation science.” Let me explain.
Quinn et al. (3) mention that a solution to capture contextual outcomes could be using the Practical, Robust Implementation, and Sustainability Model (PRISM) framework, which is an expansion of RE-AIM. In fact, PRISM's constructs of External Environment, Intervention, Implementation and Sustainability, Infrastructure, and the Recipients align well with the RE-AIM constructs (12) and could be a great fit for Quinn et al.'s project.
However, as is shown by Quinn et al.'s data (3), we need to be careful about our assumptions that frameworks and constructs developed in high income countries (HIC) would fit in LMIC without any adaptation. This is because often contextual factors, such as health system structures, resource availability, cultural, and political norms and values are different in HIC compared to LMIC (13). In fact, the issues with fitting definitions of the implementation science constructs in LMICs are not unique to RE-AIM. In a systematic review of papers and authors survey, Means et al. (14) also identified challenges with some of the Consolidated Framework for Implementation Research (CFIR) (15) constructs. For example, similar to Quinn et al., their stakeholders also asked for more system level constructs, as they had difficulties applying the construct patient needs with interventions that took place at district or national levels. The examples of Quinn et al. and of Means et al. highlight the necessity of being humble with our frameworks, and to examine carefully our definitions to avoid the ethnocentric bias of implementation studies (16).
Several of us have written about the challenges of working in LMIC including issues with: (a) defining the evidence of the intervention (e.g., the fact that one intervention is proven efficacious in HIC, does not mean that it is efficacious in a LMIC), (b) measurement (i.e., issues of validity, availability of data), and (c) mechanisms of action (which may differ depending on context) (17, 18). As we continue to define our implementation constructs and outcomes, and better understand the theories and conceptual approaches, we should incorporate the testing of the boundaries of our implementation science frameworks in LMICs, as the majority of the frameworks and measures were developed in HICs. Perhaps now it is time for us to consider how is implementation being conceptualized (19). That is, in addition to adapting the definitions of the constructs of our frameworks, we should also have an explicit conversation about what is context and how context defines the boundaries of these definitions, our evidence, and who judges the usefulness of the frameworks and theories in which context. I look forward to more empirical studies so that we can continue to “theorize” (2) and contribute to the advancement of the field of implementation science.
Author Contributions
The author confirms being the sole contributor of this work and has approved it for publication.
Funding
AB was funded by P50 CA-19-006, 3U01HL13399403S1, UL1TR002345, NCT 03380351, and 5U24HL136790.
Disclaimer
The content is solely the responsibility of the author and does not necessarily represent the official view of the NIH.
Conflict of Interest
The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
References
1. Damschroder LJ. Clarity out of chaos: use of theory in implementation research. Psychiatry Res. (2020) 283:112461. doi: 10.1016/j.psychres.2019.06.036
2. Kislov R, Pope C, Martin GP, Wilson PM. Harnessing the power of theorising in implementation science. Implement Sci. (2019) 14:103. doi: 10.1186/s13012-019-0957-4
3. Quinn AK, Neta G, Sturke R, Olopade CO, Pollard SL, Sherr K, et al. Adapting and operationalizing the RE-AIM framework for implementation science in environmental health: clean fuel cooking programs in low resource countries. Front Public Health. (2019) 7:389. doi: 10.3389/fpubh.2019.00389
4. Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. (1999) 89:1322–7. doi: 10.2105/AJPH.89.9.1322
5. Knight DK, Belenko S, Wiley T, Robertson AA, Arrigona N, Dennis M, et al. Juvenile Justice—Translational Research on Interventions for Adolescents in the Legal System (JJ-TRIALS): a cluster randomized trial targeting system-wide improvement in substance use services. Implement Sci. (2016) 11:57. doi: 10.1186/s13012-016-0423-5
6. DiMartino LD, Baumann AA, Hsu LL, Kanter J, Gordeuk VR, Glassberg J, et al. The sickle cell disease implementation consortium: translating evidence-based guidelines into practice for sickle cell disease. Am J Hematol. (2018) 93:E391–5. doi: 10.1002/ajh.25282
7. Glasgow RE, Harden SM, Gaglio B, Rabin B, Smith ML, Porter GC, et al. RE-AIM planning and evaluation framework: adapting to new science and practice with a 20-year review. Front Public Health. (2019) 7:54. doi: 10.3389/fpubh.2019.00064
8. Prusaczyk B, Swindle T, Curran G. Defining and conceptualizing outcomes for de-implementation: key distinctions from implementation outcomes. Implement Sci Commun. (2020) 1:43. doi: 10.1186/s43058-020-00035-3
9. Gaglio B, Shoup JA, Glasgow RE. The RE-AIM framework: a systematic review of use over time. Am J Public Health. (2013) 103:e38–46. doi: 10.2105/AJPH.2013.301299
10. Jones A, Thow AM, Mhurchu CN, Sacks G, Neal B. The performance and potential of the Australasian Health Star Rating system: a four-year review using the RE-AIM framework. Aust N Z J Public Health. (2019) 43:355–65. doi: 10.1111/1753-6405.12908
11. May CR, Johnson M, Finch T. Implementation, context and complexity. Implement Sci. (2016) 11:141. doi: 10.1186/s13012-016-0506-3
12. McCreight MS, Rabin BA, Glasgow RE, Ayele RA, Leonard CA, Gilmartin HM, et al. Using the Practical, Robust Implementation and Sustainability Model (PRISM) to qualitatively assess multilevel contextual factors to help plan, implement, evaluate, and disseminate health services programs. Transl Behav Med. (2019) 9:1002–11. doi: 10.1093/tbm/ibz085
13. Bergström A, Skeen S, Duc DM, Blandon EZ, Estabrooks C, Gustavsson P, et al. Health system context and implementation of evidence-based practices—development and validation of the Context Assessment for Community Health (COACH) tool for low- and middle-income settings. Implement Sci. (2015) 10:120. doi: 10.1186/s13012-015-0305-2
14. Means AR, Kemp CG, Gwayi-Chore M-C, Gimbel S, Soi C, Sherr K, et al. Evaluating and optimizing the consolidated framework for implementation research (CFIR) for use in low- and middle-income countries: a systematic review. Implement Sci. (2020) 15:17. doi: 10.1186/s13012-020-0977-0
15. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. (2009) 4:50. doi: 10.6/1748-5908-4-50
16. Ridde V, Pérez D, Robert E. Using implementation science theories and frameworks in global health. BMJ Glob Health. (2020) 5:e002269. doi: 10.1136/bmjgh-2019-002269
17. Baumann AA, Cabassa LJ. Reframing implementation science to address inequities in healthcare delivery. BMC Health Serv Res. (2020) 20:190. doi: 10.1186/s12913-020-4975-3
18. Baumann AA, Mejia A, Lachman JM, Parra-Cardona R, López-Zerón G, Amador Buenabad NG, et al. Parenting programs for underserved populations in low- and middle-income countries: issues of scientific integrity and social justice. Glob Soc Welfare. (2018). doi: 10.1007/s40609-018-0121-0
Keywords: adaptation, implementation, low-middle income countries, framework, RE-AIM (reach, effectiveness, adoption, implementation and maintenance)
Citation: Baumann AA (2020) Commentary: Adapting and Operationalizing the RE-AIM Framework for Implementation Science in Environmental Health: Clean Fuel Cooking Programs in Low Resource Countries. Front. Public Health 8:218. doi: 10.3389/fpubh.2020.00218
Received: 06 May 2020; Accepted: 12 May 2020;
Published: 09 June 2020.
Edited by:
Matthew Lee Smith, Texas A&M University, United StatesReviewed by:
Caroline Diane Bergeron, Bexar County Community Health Collaborative, United StatesCopyright © 2020 Baumann. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Ana A. Baumann, abaumann@gwbmail.wustl.edu