Advancing rapid adaptation for urgent public health crises: Using implementation science to facilitate effective and efficient responses

Responding rapidly to emerging public health crises is vital to reducing their escalation, spread, and impact on population health. These responses, however, are challenging and disparate processes for researchers and practitioners. Researchers often develop new interventions that take significant time and resources, with little exportability. In contrast, community-serving systems are often poorly equipped to properly adopt new interventions or adapt existing ones in a data-driven way during crises' onset and escalation. This results in significant delays in deploying evidence-based interventions (EBIs) with notable public health consequences. This prolonged timeline for EBI development and implementation results in significant morbidity and mortality that is costly and preventable. As public health emergencies have demonstrated (e.g., COVID-19 pandemic), the negative consequences often exacerbate existing health disparities. Implementation science has the potential to bridge the extant gap between research and practice, and enhance equity in rapid public health responses, but is underutilized. For the field to have a greater “real-world” impact, it needs to be more rapid, iterative, participatory, and work within the timeframes of community-serving systems. This paper focuses on rapid adaptation as a developing implementation science area to facilitate system responses during public health crises. We highlight frameworks to guide rapid adaptation for optimizing existing EBIs when responding to urgent public health issues. We also explore the economic implications of rapid adaptation. Resource limitations are frequently a central reason for implementation failure; thus, we consider the economic impacts of rapid adaptation. Finally, we provide examples and propose directions for future research and application.

Introduction

Public health emergencies are health-related events for which the timing, scale, and unpredictability threaten the capability of clinical, community, and public health systems to effectively manage them (1). In response to these urgent issues, a range of treatment and prevention interventions across a variety of settings are utilized to stem their onset and escalation. But how interventions are implemented is as important as what interventions are implemented (2). Implementation science has significant and underdeveloped potential to facilitate the rapid adaptation of evidence-based interventions (EBIs; which can include programs, practices, and/or policies) to meet population needs and minimize morbidity and mortality in response to a crisis (3).

The COVID-19 pandemic highlighted the critical role of dissemination and implementation (D&I) science in facilitating the rapid and effective uptake of EBIs during urgent or emerging public health events. The D&I of interventions often involves adaptations to existing practices, policies, workflows, and priorities; this is even more challenging under the time- and resource-constraints related to urgent issues (4). The consequences of not incorporating a D&I focus in addressing urgent issues can include delayed services, wasted resources, inequities in service access and delivery, and ultimately poor public health outcomes (4).

Responding to emergent public health issues are challenging and disparate processes for researchers and practitioners. Traditional research-to-practice translation approaches that are both time- and resource-intensive are inadequate to address urgent public health crises (3). Researchers often develop new interventions using the traditional, linear research process that takes years and significant resources, generally with little exportability or external validity once the initial efficacy trials are completed (5). Yet, developing and testing new interventions for each emerging crisis is unlikely to meet population needs, fit health organizations' timeframes, and likely to result in a widening of the research-to-practice gap. Practitioners, health clinics, schools, and other systems, in contrast, are often poorly equipped to identify and adopt EBIs for rapid dissemination during a crisis (2). Research evidence is infrequently communicated in a way that is accessible and pragmatic for practitioners and systems to apply (6). For D&I science to have a greater “real-world” impact it needs to be more rapid, iterative, and work collaboratively within the timeframes and capacities of systems that serve communities (7, 8).

Health equity in D&I research and rapid adaptation

As the COVID-19 pandemic and other public health emergencies have demonstrated, marginalized and disadvantaged populations often suffer the negative consequences disproportionately (9). During the pandemic, we observed elevated morbidity and mortality and secondary consequences such as exacerbating mental health and substance use issues among marginalized populations who have the least access to treatment and prevention services (9, 10). Researchers suggest that those who could most benefit from EBIs may also be the least likely to receive them as intended -referred to as the “inverse prevention law (11).” This may be especially true with urgent public health issues. Rapid responses are often needed for all populations, but with attention to specific gaps that may contribute to inequities. Health equity may be the “central indicator of success” for implementation research, but this requires a greater explicit focus on meeting the needs of higher-risk populations (11, 12).

In the current paper, our purpose is to advance the application of D&I science to address urgent public health issues through rapid adaptations. We address the following objectives: (1) summarize recent work on rapid adaptations within implementation science, including specific theories and frameworks; (2) provide examples and identify strengths and limitations of this work, and (3) discuss future directions for research and practice on rapid adaptations.

D&I science and rapid adaptation

Structured approaches to rapid adaptations, based on conceptual models or research design principles, are needed. Rapid research methods, used to guide rapid adaptations, are not unique to D&I science; fields such as human factors engineering and frameworks such as human-centered design have long embraced rapid-cycle research to iteratively improve (i.e., adapt) products and processes to effectively and efficiently meet end-user needs (13–16). D&I science is learning from these areas, and while not comprehensive, we highlight several D&I approaches for rapid responses (also see Figure 1). We summarize common steps and demonstrate their application in the case examples and discuss collaborator engagement; setting-related factors; and economic implications as central in the rapid, iterative nature of the process.

FIGURE 1

Figure 1. Common steps across the rapid adaptation process: collaborator engagement, setting-related factors, and economic implications are central as are the cyclical, rapid, iterative nature of the processes.

Selected frameworks to inform rapid responses

After action review

An After Action Review (AAR) is a learning-driven constructive review of actions taken in preparation for, during, and following a public health event, to inform changes to effectively address impending, ongoing, and future crises (17). AARs have been successfully used by the World Health Organization (WHO) and others to systematically learn from both emergent events [e.g., natural disasters; (18, 19)] and planned events [e.g., quality improvement initiatives; (20, 21)]. AAR culminates in an actionable report of (1) what is expected to happen under existing procedures, (2) what is actually happening, (3) what is going well and why, and (4) what can be improved and how. AAR can rapidly identify needed changes to an EBI under crises, primarily through employing collaborator engagement procedures (e.g., constructive, improvement-focused facilitated discussions) that efficiently and systematically gather multiple perspectives to assess the issue, identify solutions, review their impact, and make further adjustments, as depicted in Figure 1. Specifically, AAR leads to methodical delineation of (i) current planned tasks for EBI delivery, (ii) tasks actually being conducted, (iii) planned tasks achieving stated objectives to retain as-is, and (iv) tasks that can be improved via rapid adaptations that allow effective intervention delivery under crises.

Rapid cycle research

Rapid Cycle Research (RCR), according to the Agency for Healthcare Research and Quality (AHRQ), is a process in which urgent problems are identified and addressed using incremental, contextually informed methods [see steps and context, Figure 1; (22)]. As defined at a recent meeting sponsored by the National Cancer Institute building on this initial work (23), rapid-cycle research (RCR) has six characteristics: iterative design; proximal outcome focus; partner engagement emphasis; setting and context focus; consider data sources, and incorporate appropriate rigor. While some of these characteristics are found in other research areas, their combination constitutes RCR. RCR studies, importantly, may place greater or lesser emphasis on each of these characteristics. The rigor characteristic is intended to address potential objections that rapid research necessarily sacrifices rigor or methodology standards. Although not listed as a key characteristic, RCR also focuses on the efficiency of research, that is maximizing outcomes while minimizing resource use (see economic implications section).

Iterative RE-AIM

The RE-AIM (Reach Effectiveness Adoption Implementation and Maintenance) framework has been widely used to evaluate implementation and plan programs. Iterative RE-AIM provides a conceptually and data-based approach to rapid research (24). In Iterative RE-AIM, periodically throughout implementation partners and implementation teams assess progress on the various RE-AIM outcome dimensions and current priorities across these dimensions (see Figure 1). In the Iterative component, team members determine current priorities and the gap between priority and progress on each RE-AIM dimension. They then collaboratively discuss outcome targets and decide upon adaptations for the next time period.

Economic implications of D&I science for rapid adaptation

Implementing EBIs in response to crises has numerous and immediate economic implications. Economic evaluation methods used to assess costs and outcomes of public health systems interventions (25) can also inform whether rapid adaptations represent a cost-effective use of limited resources. Important costs related to rapid responses include intervention costs, implementation costs, including implementation strategies, and downstream costs (26). Implementation strategy costs to engage in rapid adaptation (solution identification, Figure 1) may include resources for intervention tailoring (i.e., promote adaptability), conducting ongoing training, developing an implementation blueprint, and resources dedicated to ongoing quality monitoring of the intervention to ensure its safety and effectiveness (evaluation tools, iterative assessments, Figure 1), and potentially downstream costs.

Whether rapid implementation strategies are making health organizations more efficient is another key consideration. Efficiency may include, for example, technical efficiency which measures the quantity of outputs produced relative to inputs used. Allocative efficiency examines whether an organization uses inputs, given their prices, in a way that minimizes total costs (27). Relative efficiency considers the diversity of inputs and outputs used in healthcare and the range of implicit valuation on various intervention components placed by organizations and systems. Prospective economic modeling can inform health practitioners and policymakers planning to deploy strategies for rapid adaptation and scale-up EBIs by projecting the expected value and impacts of various levels of efficiency.

Case examples

AAR to rapidly adapt residential treatment programs' responses to COVID-19

We applied the AAR framework to learn from residential treatment programs' COVID-19 responses and identify changes to inform subsequent waves of the pandemic (28). We examined two Mental Health Residential Rehabilitation and Treatment Programs of the United States Department of Veterans Affairs [VA; (29)]. These programs have around-the-clock staffing for residents in need of behavioral health care and/or experiencing homelessness. The AAR included five phases (Design, Prepare, Conduct, Debrief, and Follow-up) conducted over seven months, approximately 2 months of which involved engaging program personnel in improvement-focused discussions and rapidly identifying context-specific adaptations. The other preparatory and follow-up months were for initially establishing AAR procedures and iteratively pursuing continuous improvement, respectively, which can be expedited for future AAR applications and undertaken in parallel with operationalizing identified adaptations.

For the Design phase, we devised involving program staff and residents to conduct small-group virtual discussions of four to seven individuals per group. For the Prepare phase, our semi-structured guide included: What operating procedures were established/revised for COVID-19? What cooperation with other organizations occurred? What physical/mental health issues arose more/less frequently since COVID-19? What policies worked well or need revision? For the Conduct phase, we incorporated additional probing questions aligned with the AAR framework: What was planned? What actually happened? What went well and why? What can be improved and how? For the Debrief phase, we summarized and shared our findings with program and health care system leadership. Recommended adaptations identified included (i) conveying reasons for COVID-related precautions/changes to residents, (ii) keeping COVID-related information sharing and recovery-oriented programming separate, (iii) providing “how to use technology” training during program orientation, and (iv) developing procedures for safe family interactions and off-site activities. For the Follow-up phase, rapid adaptations identified in the AAR discussions included (i) providing details for COVID-related precautions/change during all-resident community meetings, (ii) de-emphasizing COVID-related information during treatment groups, (iii) consolidating all remote programming under one technology platform, and (iv) when COVID prevalence is low, granting family visit passes.

Rapid adaptation of a physical activity intervention during COVID-19

InPACT (Interrupting Prolonged sitting with ACTivity) is a school-based intervention for children and youth focused on increasing physical activity levels with short bursts of structured activity breaks throughout the day (30). InPACT was originally developed based on principles of designing for dissemination to support implementing core functions (e.g., PA: physically active time) and permit flexibility to meet the unique needs and resources of the setting; InPACT includes a compendium of implementation strategies to support flexibility and uptake (31, 32).

The COVID-19 pandemic posed new challenges to youth PA, with low-resource communities disproportionately impacted (33, 34). The widespread shutdowns, including schools, imposed significant barriers to exercise opportunities; consequently, PA among youth plummeted, especially among racial/ethnic minorities in low-resource environments (35). The Vice President of Michigan's Board of Education created a PA dissemination task force and chose InPACT to rapidly adapt and disseminate to mitigate this urgent public health issue. The 3-month process of rapidly adapting InPACT from school to the home was guided by Rapid-Cycle Research and Iterative RE-AIM. The details of the adaptation process are described elsewhere (36), but are briefly described here.

The steps included: Step 1: identifying partner organizations aligned with the goal of improving PA; this included PE teachers, state agencies, professional organizations (e.g., principals' association), school health coordinators, and professional sports teams. Steps 2 and 3: engaging in problem and knowledge exploration. The research team hosted community forms with parents, teachers, administrators, and community members to aid in understanding the scope of the problem. Steps 4 and 5: initiating solution development and testing. The task force identified InPACT as the simplest and most scalable solution. The intervention development team adapted InPACT for home delivery (e.g., creating asynchronous PA videos) and used iterative RE-AIM to assess progress. Partner organizations/teams met periodically throughout the implementation process and assessed progress on their identified RE-AIM outcome dimensions given each team's priorities and objectives. For example, the InPACT development team assessed Reach by the number of partner organizations that were actively disseminating InPACT through various channels (e.g., public TV). Iterative RE-AIM offered each team member the opportunity to capitalize on their strengths and make adjustments based on the most important dimensions of their efforts. Step 6: utilizing coordinated, active dissemination strategies across collaborators (e.g., press releases, promotion through partner organizations) to enhance the reach of InPACT@home.

Limitations of implementation science and rapid adaptation

Although rapid adaptation may be essential for responding to emerging public health crises, it also has the potential to create adverse sequelae and unintended consequences. In response to surging COVID-19 cases, hospitals around the world rapidly revised and implemented new visitation policies (37–39). The swift enactment of adapted visitation policies significantly affected patients and their family members. For example, in maternal-infant health settings, many of these rapidly adapted policies restricted support persons (e.g., partners or doulas) during labor (40), and, specifically in neonatal intensive care settings, the separation of mothers and/or fathers from their infants (41). Emerging literature highlights the negative consequences of these adapted infection prevention policies on patient outcomes (39).

The example of rapidly adapted policies highlights two critical points as we consider advancing rapid adaptation in D&I science: The need to (1) identify and assess the potential for immediate and long-term adverse effects before engaging in rapid adaptation and (2) prepare to address the occurrence and magnitude of potential downstream adverse effects. Additionally, iterative and rapid assessments, as with iterative RE-AIM or other audit and feedback processes (see Figure 1), are vital so practices or policies can be modified or abandoned if proven harmful (4). Involving community partners in planning for and guiding rapid adaptations (see collaborator engagement, Figure 1), and curating available resources and supports should reduce the likelihood and/or magnitude of adverse or subsequent effects resulting from rapid adaptation.

Creating synergy: D&I science and related fields

There is increasing dialogue and synergy between D&I science and related fields that can especially be leveraged to fuel rapid adaptation-related developments. The first is with improvement science. These fields share purpose, scope, and methods and are at similar stages of scientific discipline development (42, 43). This synergy is expected to grow as D&I's focus on improving EBI implementation extends to demonstrating successful impact on care quality, value, and safety (i.e., foci of improvement science) and as improvement science integrates principles of both implementation science and quality improvement. The synergy can provide fertile ground for adopting well-established iterative quality improvement approaches and concepts for use by implementation science for rapid adaptations. The second area is human-centered design, which focuses on shaping products (e.g., EBIs) to be grounded in the people and settings who will use the innovation (44). Efforts to apply human-centered design approaches for EBI implementation (45–47) emphasize iteratively updated contextual needs assessments and updated interventions and implementation strategies. As implementation science continues to draw on human-centered design, the latter's in-depth and constant focus on user needs, prototype testing, and contextual alignment can undoubtedly aid with rapid adaptations that enable better implementation of EBIs into target settings.

Discussion

Several cross-cutting issues emerge from the rapid adaptation approaches and examples discussed (see Table 1):

TABLE 1

Table 1. Cross-cutting issues in rapid adaptation.

(1) The suitability of a rapid adaptation application: Not all issues lend themselves to rapid adaptation. For example, if the costs of being wrong are substantial or if the only data available are significantly lagging behind implementation, this may not be the best approach. (2) Pragmatic data sources and evaluation: There are recent examples of developing valid, close-to real-time data (48). However, obtaining and checking data for accuracy can itself be time-consuming. Organizations and systems may need to create an infrastructure, for example, to curate reliable electronic health record (EHR) data (49). Given truncated timeframes, prioritizing data that are central to decision-making is also important when considering data sources and evaluation. (3) Collaborator engagement and “going slow to go fast.” Developing trust and working across different partners takes time, but once these relationships are established, research can proceed more quickly. (4) Allocating and leveraging resources: The need to leverage existing resources includes creating ways to deliver programs and evaluations using available resources and staff, or routinely collected secondary data. A less frequently recognized rapid adaptation need is relying on institutional memory, especially how similar EBIs adaptations have worked in the past. (5) Reduce risks and optimize benefits: Another cross-cutting issue is the need to prevent and address potential risks to rapid adaptations. Unintended consequences, especially adversely impacting health and equity, need to be considered. Such outcomes can be mitigated by strategies including partner engagement that ensures representation from impacted groups, rapid participatory modeling-including costs and benefits, and including measures of health disparity as key outcomes. Rapid iteration along with continual evaluation may be the single best way to address potential harms as well as other challenges such as limited experimental rigor. (6) Equity impacts: Although sample sizes are often small in rapid research, settings and participants can be purposefully selected to include diversity and characteristics especially important for generalization (e.g., including low-resource settings that have high staff turnover; participants with social health challenges or having experienced health inequities). Ensuring that given adaptations achieve equitable impact will support a greater focus on how to meet the needs of populations at high risk of experiencing the negative impacts of public health events (12).

As argued by Chambers and Norton, adaptations to the EBI itself may be required to better fit the context in which implementation occurs but must retain its core elements to achieve the intended benefit (50). Similarly, implementation strategies may also need modification to suit the context and in response to evolving challenges (51, 52). Moreover, to effectively address a crisis, several different EBIs and implementation strategies may warrant being employed simultaneously, and this, too, may require tailoring to the context. For example, there are multiple COVID-19 prevention measures including vaccines, masks, and physical distancing, and how they are relatively prioritized and implemented alongside one another may need to be modified based on cultural norms, available resources, and other contextual characteristics of different settings.

As adaptations are central to implementation, Wiltsey Stirman et al. (53) developed the Framework for Reporting Adaptations and Modifications to Evidence-based interventions (FRAME) to advance adaptation measurement. Although FRAME was not developed exclusively for rapid adaptations, elements of it and the companion FRAME-IS for evaluating adaptations to implementation strategies (54) can be applied to rapid research. Reporting and measuring modifications to EBIs and implementation strategies is critically important, and examples, recommendations, and frameworks are available (53, 55).

The COVID-19 pandemic has underscored the urgency of advancing D&I science to guide effective, rigorous, and efficient rapid adaptations. We conclude that making science more rapid is vital to reducing morbidity and mortality during public health crises. We acknowledge that the relative contribution and costs of rapid adaptations need to be carefully considered and monitored to ensure they achieve desired objectives. In this paper, we provide preliminary guidance on rapid adaptation based on data and theory from D&I and related disciplines. As an emerging area in D&I science, rapid adaptation has notable potential to support conceptual and data-driven decision-making during crises to minimize negative public health impacts.

Data availability statement

The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author/s.

Author contributions

All authors listed have made a substantial, direct, and intellectual contribution to the work and approved it for publication.

Funding

Preparation for this manuscript was in part supported by K01DA044279, PI: Eisman, National Institute on Drug Abuse, National Institutes of Health. Publication fees are supported by Wayne State University.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

1. Nelson C, Lurie N, Wasserman J, Zakowski S. Conceptualizing and defining public health emergency preparedness. Am J Public Health. (2007) 97:S9–11. doi: 10.2105/AJPH.2007.114496

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Eccles M, Mittman B. Welcome to implementation science. Implement Sci. (2006) 1:1. doi: 10.1186/1748-5908-1-1

CrossRef Full Text | Google Scholar

3. Wensing M, Sales A, Armstrong R, Wilson P. Implementation science in times of Covid-19. Implement Sci. (2020) 15:42. doi: 10.1186/s13012-020-01006-x

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Yousefi Nooraie R, Shelton RC, Fiscella K, Kwan BM, McMahon JM. The pragmatic, rapid, and iterative dissemination and implementation (PRIDI) cycle: adapting to the dynamic nature of public health emergencies (and beyond). Health Res Policy Sys. (2021) 19:110. doi: 10.1186/s12961-021-00764-4

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Balas E, Boren S. Managing clinical knowledge for health care improvement. Yearb Med Inform. (2000) 65–70. doi: 10.1055/s-0038-1637943

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Grimshaw J, Eccles M, Lavis J, Hill S, Squires J. Knowledge translation of research findings. Implementation Sci. (2012) 7:1–17. doi: 10.1186/1748-5908-7-50

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Glasgow R, Chambers D. Developing robust, sustainable, implementation systems using rigorous, rapid and relevant science. Clin Transl Sci. (2012) 5:48–55. doi: 10.1111/j.1752-8062.2011.00383.x

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Peek CJ, Glasgow RE, Stange KC, Klesges LM, Purcell EP, Kessler RS. The 5 R's: An emerging bold standard for conducting relevant research in a changing world. Ann Fam Med. (2014) 12:447–55. doi: 10.1370/afm.1688

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Laajaj R, Webb D, Aristizabal D, Behrentz E, Bernal R, Buitrago G, et al. Understanding how socioeconomic inequalities drive inequalities in COVID-19 infections. Sci Rep. (2022) 12:8269. doi: 10.1038/s41598-022-11706-7

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Calvano C, Engelke L, Di Bella J, Kindermann J, Renneberg B, Winter SM. Families in the COVID-19 pandemic: parental stress, parent mental health and the occurrence of adverse childhood experiences—results of a representative survey in Germany. Eur Child Adolesc Psychiatry. (2022) 31:1–13. doi: 10.1007/s00787-021-01739-0

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Lorenc T, Petticrew M, Welch V, Tugwell P. What types of interventions generate inequalities? evidence from systematic reviews. J Epidemiol Community Health. (2013) 67:190–3. doi: 10.1136/jech-2012-201257

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Brownson RC, Kumanyika SK, Kreuter MW, Haire-Joshu D. Implementation science should give higher priority to health equity. Implement Sci. (2021) 16:28. doi: 10.1186/s13012-021-01097-0

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Daddato AE, Staton EW, Bettcher BM, Shanbhag P, Lum HD. Human-centered design of an advance care planning group visit for mild cognitive impairment. Gerontologist. (2021) gnab181. doi: 10.1093/geront/gnab181. [Epub ahead of print].

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Austin EJ, LeRouge C, Lee JR, Segal C, Sangameswaran S, Heim J, et al. A learning health systems approach to integrating electronic patient-reported outcomes across the health care organization. Learn Health Syst. (2021) 5:e10263. doi: 10.1002/lrh2.10263

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Kurtz CE, Peng Y, Jesso M, Sanghavi H, Kuehl DR, Parker SH. Using a human factors-centric approach to development and testing of a face shield designed for health care workers: a COVID-19 case study for process and outcomes. Am J Infect Control. (2022) 50:306–11. doi: 10.1016/j.ajic.2021.10.033

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Sonney J, Cho EE, Zheng Q, Kientz JA. Refinement of a parent-child shared asthma management mobile health app: human-centered design study. JMIR Pediatr Parent. (2022) 5:e34117. doi: 10.2196/34117

PubMed Abstract | CrossRef Full Text | Google Scholar

17. World Health Organization. Guidance for After Action Review (AAR). Geneva (2019). Report No.: Licence: CC BY-NC-SA 3.0 IGO. Available online at: https://www.euro.who.int/__data/assets/pdf_file/0009/393651/who-guid-aar-eng.pdf

Google Scholar

18. Knox CC. Analyzing after-action reports from hurricanes andrew and katrina: repeated, modified, and newly created recommendations. J Emerg Manag. (2013) 11:160–8. doi: 10.5055/jem.2012.0135

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Stoto MA, Nelson C, Piltch-Loeb R, Mayigane LN, Copper F, Chungong S. Getting the most from after action reviews to improve global health security. Global Health. (2019) 15:58. doi: 10.1186/s12992-019-0500-z

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Rowe R. After-Action Review: A Process for Improvement. Pensacola, FL: University of West Florida (2006). p. 5. Available online at: https://www1.appa.org/files/PDFs/After-Action%20Revew.pdf

Google Scholar

21. Salem-Schatz S, Ordin D, Mittman B. Guide to the After Action Review. VA Center for Implementation Practice Research Support (2010). p. 12. Available online at: https://www.cebma.org/wp-content/uploads/Guide-to-the-after_action_review.pdf

Google Scholar

22. Johnson K, Gustafon D, Ewigman B, Provist L, Roper R. Using Rapid-Cycle Research to Reach Goals: Awareness, Assessment, Adaptation, Acceleration. Rockville, MD: Agency for Healthcare Research and Quality U.S. Department of Health and Human Services (2015). Report No.: AHRQ Publication No. 15-0036.

Google Scholar

23. Division of Cancer Control Population Sciences. Rapid Cycle Research in Cancer Care. Bethesda, MD (2022). Available online at: https://events.cancer.gov/nci/rapidcycleresearch

Google Scholar

24. Glasgow R, Battaglia C, McCreight M, Ayele R, Rabin B. Making implementation science more rapid: use of the re-aim framework for mid-course adaptations across five health services research projects in the veterans health administration. Front Public Health. (2020) 8:194. doi: 10.3389/fpubh.2020.00194

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Owens DK, Qaseem A, Chou R, Shekelle P, Clinical Guidelines Committee of the American College of Physicians. High-value, cost-conscious health care: concepts for clinicians to evaluate the benefits, harms, and costs of medical interventions. Ann Intern Med. (2011) 154:174–80. doi: 10.7326/0003-4819-154-3-201102010-00007

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Gold HT, McDermott C, Hoomans T, Wagner TH. Cost data in implementation science: categories and approaches to costing. Implement Sci. (2022) 17:1–12. doi: 10.1186/s13012-021-01172-6

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Sickles R, Zelenyuk V. Measurement of Productivity Efficiency. Cambridge University Press (2019). Available online at: https://econpapers.repec.org/bookchap/cupcbooks/9781107036161.htm (accessed May 13, 2022).

Google Scholar

28. Kim B, Petrakis BA, McInnes DK, Gifford AL, Sliwinski SK, Smelson DA. Applying after action review to examine residential treatment programs' responses to COVID-19. Int J Health Plann Manage. (2022) 37:2461–67. doi: 10.1002/hpm.3475

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Veterans Health Administration. Definitions of MH RRTPs. Available online at: https://www.va.gov/homeless/docs/DCHV_Definitions_of_MHRRTPs.pdf (accessed April 4, 2020).

Google Scholar

30. Beemer LR, Ajibewa TA, Vance US, Colabianchi N, Hasson RE. Feasibility of the InPACT intervention to enhance movement and learning in the classroom. Translat J Am Acad Sprot Med. (2018) 3:136-51. doi: 10.1249/TJX.0000000000000069

CrossRef Full Text | Google Scholar

31. Brownson R, Jacobs J, Tabak R, Hoehner C, Stamatakis K. Designing for dissemination among public health researchers: findings from a national survey in the United States. Am J Public Health. (2013) 103:1693–9. doi: 10.2105/AJPH.2012.301165

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Hasson R, Beemer L, Ajibewa T, Eisman A. Adapting the InPACT intervention to enhance implementation fidelity and flexibility. Prev Sci. (2021) 22:324–33. doi: 10.1007/s11121-020-01199-z

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Bates LC, Zieff G, Stanford K, Moore JB, Kerr ZY, Hanson ED, et al. COVID-19 Impact on behaviors across the 24-hour day in children and adolescents: physical activity, sedentary behavior, and sleep. Children. (2020) 7:E138. doi: 10.3390/children7090138

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Moore SA, Faulkner G, Rhodes RE, Brussoni M, Chulak-Bozzer T, Ferguson LJ, et al. Impact of the COVID-19 virus outbreak on movement and play behaviours of Canadian children and youth: a national survey. Int J Behav Nutr Phys Act. (2020) 17:85. doi: 10.1186/s12966-020-00987-8

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Hasson R, Sallis JF, Coleman N, Kaushal N, Nocera VG, Keith N. COVID-19: Implications for physical activity, health disparities, and health equity. Am J Lifestyle Med. (2021) 16:420–33. doi: 10.1177/15598276211029222

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Hasson R, Eisman A, Wassman A, Martin S, Pugh P, Robinson L, et al. Rapid cycle adaptation of the InPACT program to promote equity in access to youth physical activity opportunities. Translat Behav Med. (In press).

Google Scholar

37. Fiest KM, Krewulak KD, Hiploylee C, Bagshaw SM, Burns KEA, Cook DJ, et al. An environmental scan of visitation policies in Canadian intensive care units during the first wave of the COVID-19 pandemic. Can J Anaesth. (2021) 68:1474–84. doi: 10.1007/s12630-021-02049-4

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Liu YA, Hsu YC, Lin MH, Chang HT, Chen TJ, Chou LF, et al. Hospital visiting policies in the time of coronavirus disease 2019: a nationwide website survey in Taiwan. J Chin Med Assoc. (2020) 83:566–70. doi: 10.1097/JCMA.0000000000000326

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Vance AJ, Duy J, Laventhal N, Iwashyna TJ, Costa DK. Visitor guidelines in US children's hospitals during COVID-19. Hosp Pediatr. (2021) 11:e83–9. doi: 10.1542/hpeds.2020-005772

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Shuman CJ, Morgan ME, Chiangong J, Pareddy N, Veliz P, Peahl AF, et al. “Mourning the experience of what should have been”: experiences of peripartum women during the COVID-19 pandemic. Matern Child Health J. (2022) 26:102–9. doi: 10.1007/s10995-021-03344-8

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Darcy Mahoney A, White RD, Velasquez A, Barrett TS, Clark RH, Ahmad KA. Impact of restrictions on parental presence in neonatal intensive care units related to coronavirus disease 2019. J Perinatol. (2020) 40:36–46. doi: 10.1038/s41372-020-0753-7

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Check DK, Zullig LL, Davis MM, Davies L, Chambers D, Fleisher L, et al. Improvement science and implementation science in cancer care: identifying areas of synergy and opportunities for further integration. J Gen Intern Med. (2021) 36:186–95. doi: 10.1007/s11606-020-06138-w

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Leeman J, Rohweder C, Lee M, Brenner A, Dwyer A, Ko LK, et al. Aligning implementation science with improvement practice: a call to action. Implement Sci Commun. (2021) 2:99. doi: 10.1186/s43058-021-00201-1

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Norman D, Draper S editors. User centered system design : new perspectives on human-computer interaction. Hillsdale, NJ: Lawrence Erlbaum Assoc. (1986). p. 526. doi: 10.1201/b15703

CrossRef Full Text | Google Scholar

45. Dopp AR, Parisi KE, Munson SA, Lyon AR. Aligning implementation and user-centered design strategies to enhance the impact of health services: results from a concept mapping study. Implement Sci Commun. (2020) 1:17. doi: 10.1186/s43058-020-00020-w

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Graham AK, Wildes JE, Reddy M, Munson SA, Barr Taylor C, Mohr DC. User-centered design for technology-enabled services for eating disorders. Int J Eat Disord. (2019) 52:1095–107. doi: 10.1002/eat.23130

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Lyon AR, Dopp AR, Brewer SK, Kientz JA, Munson SA. Designing the future of children's mental health services. Adm Policy Ment Health. (2020) 47:735–51. doi: 10.1007/s10488-020-01038-x

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Curtis JR, Foster PJ, Saag KG. Tools and methods for real-world evidence generation: pragmatic trials, electronic consent, and data linkages. Rheum Dis Clin North Am. (2019) 45:275–89. doi: 10.1016/j.rdc.2019.01.010

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Madhavan S, Bastarache L, Brown JS, Butte AJ, Dorr DA, Embi PJ, et al. Use of electronic health records to support a public health response to the COVID-19 pandemic in the United States: a perspective from 15 academic medical centers. J Am Med Inform Assoc. (2020) 28:393–401. doi: 10.1093/jamia/ocaa287

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Chambers DA, Norton WE. The adaptome: advancing the science of intervention adaptation. Am J Prev Med. (2016) 51:S124–31. doi: 10.1016/j.amepre.2016.05.011

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Kilbourne AM, Almirall D, Eisenburg D, Waxmonsky J, Goodrich DE, Fortney JC, et al. Adaptive implementation of effective programs trial (ADEPT): cluster randomized SMART trial comparing a standard versus enhanced implementation strategy to improve outcomes of a mood disorders program. Implement Sci. (2014) 9:132. doi: 10.1186/s13012-014-0132-x

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Kilbourne AM, Smith SN, Choi SY, Koschmann E, Liebrecht C, Rusch A, et al. Adaptive School-based Implementation of CBT (ASIC): clustered-SMART for building an optimized adaptive implementation intervention to improve uptake of mental health interventions in schools. Implement Sci. (2018) 13:119. doi: 10.1186/s13012-018-0808-8

PubMed Abstract | CrossRef Full Text | Google Scholar

53. Wiltsey Stirman S, Baumann AA, Miller CJ. The FRAME: an expanded framework for reporting adaptations and modifications to evidence-based interventions. Implement Sci. (2019) 14:58. doi: 10.1186/s13012-019-0898-y

PubMed Abstract | CrossRef Full Text | Google Scholar

54. Miller CJ, Barnett ML, Baumann AA, Gutner CA, Wiltsey-Stirman S. The FRAME-IS: a framework for documenting modifications to implementation strategies in healthcare. Implement Sci. (2021) 16:36. doi: 10.1186/s13012-021-01105-3

PubMed Abstract | CrossRef Full Text | Google Scholar

55. Escoffery C, Lebow-Skelley E, Haardoerfer R, Boing E, Udelson H, Wood R, et al. A systematic review of adaptations of evidence-based public health interventions globally. Implement Sci. (2018) 13:125. doi: 10.1186/s13012-018-0815-9

PubMed Abstract | CrossRef Full Text | Google Scholar