Can theory-driven implementation interventions help clinician champions promote opioid stewardship after childbirth? Protocol for a pragmatic implementation study

Background: Our objective is to determine the effect of a new national clinical practice guideline (CPG) for pain management after childbirth, as implemented with less vs. more intensive implementation support, on postpartum opioid prescribing.

Methods: A quasi-experimental analysis will measure the impact of post-childbirth pain management guidelines on opioid prescribing in a statewide hospital collaborative, overall and among key patient subgroups at risk for inequitable care and outcomes. We will also use a randomized, non-responder design and mixed-methods approaches to evaluate the effects of Replicating Effective Programs (REP), a theory-driven, scalable implementation intervention, and Enhanced REP (E-REP; i.e., REP augmented with facilitation, which is individualized consultation with site champions to overcome local barriers) on the uptake of the CPG. The study will include hospitals within the Obstetrics Initiative (OBI), a perinatal collaborative quality initiative funded by Blue Cross Blue Shield of Michigan that includes 68 member hospitals serving more than 120,000 postpartum people, over approximately 15 months. Hospitals not initially responding to REP—defined by performance <15th percentile of all OBI hospitals for (a) inpatient order for opioid-sparing postpartum pain management (e.g., scheduled acetaminophen and non-steroidal anti-inflammatory drugs when not contraindicated), or (b) amount of opioid prescribed at discharge—will be allocated via block randomization to continue REP or to E-REP. Using interrupted time series analyses, the primary analysis will evaluate the rate of postpartum opioid-sparing prescribing metrics at the time of discharge (primary outcome) and opioid prescription refills and high-risk prescribing (secondary outcomes) before and after CPG implementation with REP. We will evaluate inequities in outcomes by patient, procedure, prescriber, and hospital factors. Exploratory analyses will examine temporal trends in patient-reported outcomes and the effects of continued REP vs. E-REP among slower-responder sites. We will evaluate implementation outcomes (e.g., acceptability, feasibility, costs, needed REP and E-REP adaptations) using clinician and patient surveys and qualitative methods (ClinicalTrials.gov identifier: NCT06285123).

Discussion: Findings will inform refinements to the REP and E-REP interventions and add to the literature on the effectiveness of facilitation to promote uptake of evidence-based clinical practices in maternity care.

1 Introduction

Annually in the United States (U.S.), there are nearly four million birthing people who require pain management and comfort after childbirth. Postpartum opioid prescribing in the U.S. is widely variable at time of discharge from the childbirth hospitalization, putting individuals at risk of both insufficient pain management and the risks of opioid medications (1, 2). Inadequate pain management after childbirth can adversely affect maternal wellbeing, infant care and bonding, breastfeeding continuation, and risk of depression and chronic pain (3–7). Simultaneously, excess opioid prescribing confers risks of persistent opioid use and associated harms (8–12) and adverse health outcomes in infants (13). The peripartum period is also fraught with significant inequities in pain management experiences (14–20). In response, our team led a rigorous, multidisciplinary process to develop the Creating Optimal pain Management FOR Tailoring care (COMFORT) clinical practice guideline (CPG) for pain management after childbirth, which aims to mitigate peripartum opioid-related risks and existing inequities in pain management after childbirth.

When new clinical guidelines emerge, standard dissemination approaches, (i.e., publication and announcement) are often insufficient to change clinical practice. Intentional implementation efforts that focus on changing provider behavior and addressing organizational barriers such as local culture, leadership buy-in, and provider training and capacity are far more resource-intensive, but may more effectively encourage use of new CPGs.

Replicating Effective Programs (REP) is one theory-based (21–25) implementation intervention that has demonstrated effectiveness in accelerating clinical change compared to routine dissemination alone (23, 26). REP consists of four core implementation strategies: (1) User-friendly “packaging” of the CPG; (2) Structured provider training; (3) Performance feedback (e.g., reports with quarterly prescribing data); and 4) Brief technical assistance [e.g., group-based educational sessions for quality improvement (QI) champions]. REP advantageously requires no specialized implementation expertise, so it can be readily deployed by local implementers.

However, REP alone may be insufficient for ensuring effective adoption of the COMFORT CPG by all sites and uniform uptake in best practices by all clinicians within sites. In prior studies, REP-based approaches were effective in only 10%–15% of sites (27–36). To address heterogeneity of treatment effects, REP may be enhanced with facilitation [Enhanced REP (E-REP)] (37, 38). Facilitation is a process of interactive problem-solving within a supportive consultative relationship, with a goal of addressing local barriers and unanticipated implementation challenges and thereby enhancing clinician uptake of evidence-based practices (39–41). Facilitation is delivered by an expert who meets regularly with site QI (or other appropriate) leaders. In contrast to the brief, standardized technical assistance in REP, facilitation is intensive, individualized consultation that generates highly customized solutions that can potentially be sustained by local clinicians (37, 40, 42, 43). Moreover, E-REP may reveal the mechanisms that underlie differences in CPG implementation among marginalized patients, and potential strategies to close these gaps, given the opportunity to deeply engage with local partners. Though E-REP can be effective for sites not responding to REP alone (35, 44), facilitation is more resource-intensive and difficult to scale (45). Therefore, identifying healthcare settings most in need of facilitation is highly significant but has not been evaluated in maternity contexts to date (35, 46).

1.1 Primary aim 1

The primary aim of this study is to evaluate clinical outcomes before and after CPG implementation with REP. We hypothesize that REP initiation will be associated with greater reductions in rates and amounts of postpartum opioid prescribing at discharge from the childbirth hospitalization (primary outcome), via an increase in opioid-sparing pain management approaches.

1.2 Exploratory aim 1

To characterize temporal trends in patient-reported outcomes (i.e., pain intensity in the first week after childbirth, opioid consumption after discharge from the childbirth hospitalization, and satisfaction with pain management after discharge from the childbirth hospitalization). We hypothesize that opioid prescribing (primary outcome) will decrease without adverse effects on patient-reported outcomes.

1.3 Exploratory aim 2

To determine the effects of REP alone vs. REP with added facilitation (E-REP) among slower-responder sites. We hypothesize that, for slower-responders, adding facilitation (E-REP) is more effective in increasing opioid-sparing prescribing than continuing REP alone.

1.4 Exploratory aim 3

To evaluate for heterogeneity of treatment effects of REP and E-REP. Specifically, we will determine whether implementation intervention effectiveness is moderated by hospital-level factors (e.g., early positive change in COMFORT practices, perceived hospital leadership support for COMFORT adoption). Results of these analyses may be used to construct a more deeply tailored adaptive intervention that further improves uptake of the COMFORT CPG.

1.5 Exploratory aim 4

To describe CPG implementation using REP and E-REP at the site level, including feasibility and acceptability to patients and providers, needed REP and E-REP adaptations, and costs of these implementation interventions.

The result of this study will be an optimized, potentially adaptive implementation intervention that could inform QI efforts in maternity units—and other acute pain settings—to improve pain management, outcomes, and outcome inequities.

2 Methods and analysis

2.1 Study design and duration

We will use a quasi-experimental design to evaluate temporal trends in clinical outcomes before and after CPG implementation with REP. A non-responder randomized trial (47, 48) will also be conducted subsequently to test the effects of REP and E-REP on clinical outcomes—overall and among key subgroups. In addition to clinical outcomes, we will also measure implementation outcomes, (e.g., adoption, acceptability, feasibility, and appropriateness of REP and E-REP; needed REP/E-REP adaptations; costs of delivering E-REP) to guide REP and E-REP enhancements. The unit of randomization will be the hospital. Figure 1 outlines the study design and timeline, resulting in a group of responder sites that continue to receive REP, a slower-responder arm that receives only REP, and a slower-responder arm that receives E-REP. It is anticipated that the study will take approximately 15 months to complete. Because our QI interventions involve provider education and consultation in use of guidelines that meet or exceed standards of practice, this study was deemed exempt in its use of survey and qualitative data collection and secondary analysis of existing data by the University of Michigan institutional review board (HUM00248235 and HUM00248331). Our study is reported according to SPIRIT guidelines.

Figure 1

Figure 1. Study design. R, randomization; REP, Replicating Effective Programs; E-REP, Enhanced Replicating Effective Programs.

2.2 Study setting and population

The study setting is the Obstetrics Initiative (OBI), a Blue Cross Blue Shield of Michigan (BCBSM)-funded collaborative of Michigan hospitals dedicated to maternity care quality improvement. OBI's 68 member hospitals constitute diverse maternity practice settings ranging from urban to rural, academic to community, and high- to low-volume maternity units. Each site has a designated QI champion team, including at least one physician champion, one nurse/midwife champion, and one clinical data abstractor. Each hospital's clinical data abstractor reviews hospital medical record data and enters initiative-specific data into an OBI clinical registry workstation. OBI hospitals receive site-specific and collaborative-wide data on a variety of maternity quality indicators. Site champions attend OBI's twice annual in-person collaborative-wide meetings, as well as virtual sessions throughout the year for education, best practice sharing, and peer mentorship. Finally, OBI sites receive Pay for Performance (P4P) points for participation in OBI activities and quality metrics obtained, with associated payments to OBI hospitals through BCBSM's Value-Based Partnerships program.

In January 2024, OBI launched a new QI initiative, “Bringing Our Patients COMFORT,” to promote adoption of the COMFORT CPG across member hospitals. To support this initiative, all OBI hospitals are eligible for P4P points for COMFORT CPG-concordant clinical behaviors—specifically, ordering scheduled acetaminophen and scheduled oral non-steroidal anti-inflammatory drugs (NSAIDs) for eligible patients and discharge opioid prescribing within CPG-concordant ranges. All hospitals will receive REP- or E-REP-based support and tools from the OBI Coordinating Center. To the degree that OBI clinical champions share these resources on their units, maternity clinicians staffing OBI hospitals will be exposed to these activities. To the degree that OBI hospitals adopt the COMFORT CPG, birthing populations at these sites will be exposed to guideline-concordant care.

2.3 Evidence-based practice

The COMFORT CPG was created using the RAND/UCLA Appropriateness Method (49), a rigorous, modified eDelphi process that incorporates published evidence (collated via a systematic review) and expert opinion to arrive at clinical recommendations. The COMFORT CPG development process explicitly involved public members and considered health equity in all stages of CPG development, informed by best practices collated by the GRADE Working Group (50–53) and a recent scoping review (54). Key components of the COMFORT CPG are highlighted in Figure 2.

Figure 2

Figure 2. Key components of the COMFORT CPG. CPG, clinical practice guidelines; OUD, opioid use disorder.

2.4 Implementation interventions

We designed REP and E-REP implementation interventions informed by prior qualitative work involving individual semi-structured interviews with 31 patients and 22 clinicians and six focus groups with 43 maternity care QI champions (55). We specify the REP and E-REP interventions briefly in Table 1, and fully elsewhere (55).

Table 1

Table 1. Brief implementation intervention specification.

2.5 Trial procedures

Pre-intervention (Month -3 to 0): All OBI hospital champions are notified about the COMFORT CPG using standard dissemination techniques (e.g., email, announcements at collaborative-wide meetings, postings on collaborative website). First-stage intervention (Month 1–9): All hospitals receive the REP implementation intervention, including: a web-based toolkit with the CPG and related implementation resources; structured clinician training offered to all maternity clinicians through a virtual educational session with an asynchronous viewing option; performance feedback about opioid prescribing and other key outcome metrics; and peer mentorship opportunities (e.g., didactic content at OBI's in-person meetings). Determination of non-response: Hospital response to REP is assessed at Month 9 (at which time Month 6 OBI registry data will be available for analysis). Performance in Months 1–6 will be used to determine responder status. Response will be defined based on hospital concordance with specific CPG recommendations; specifically: a) provision of opioid-sparing pain management during the childbirth hospitalization, and b) amount of opioid prescribing at time of discharge. OBI hospitals with the highest 15th percentile performance on both indicators (estimated n = 10 sites) will be considered “Rapid Responder” sites. The remaining 58 sites will be deemed “Slower-Responders,” enabling capture of the part of the performance distribution that is least compliant with CPG recommendations. We designated these a priori counts of hospitals a) based on prior work showing about 15% of sites respond to REP alone (27–36), and b) due to OBI Coordinating Center capacity to offer facilitation to up to 28 sites concurrently. If needed, we will modify these thresholds based on observed comparative performance of OBI sites. Second-stage intervention for slower-responders (Month 10–15): Hospitals that respond to REP will continue it. Hospitals that do not respond will be randomized 1:1 by the study data analyst to continue REP (as some hospitals simply require more time to respond) or to E-REP. Randomization will be stratified by rural vs. urban status, delivery volume (above vs. below 1,500 births annually), and site-average rate of opioid-sparing interventions in Months 4–6 to ensure that intervention groups are balanced for site variables that may highly correlate with outcomes. The study analyst will generate stratified permuted-block random allocation lists (block sizes of 2, 4, and 6) using a computer program such as PROC PLAN in SAS. A site will be considered randomized once the study analyst informs the study coordinator of each site's random assignment. For hospitals randomized to E-REP, facilitation will be based on the COMFORT facilitation guide and delivered by a trained facilitator, with support from OBI Coordinating Center Outreach and Engagement Coordinators, who have strong relationships with OBI member hospitals. The facilitator will offer to meet (virtually, or in person at the site's request) with the site's QI team to review performance trends, identify key barriers and facilitators to improvement, select improvement strategies to execute, and specify improvement targets. The facilitator will then offer recurring virtual meetings with site QI teams to promote CPG uptake, with meeting cadence adjusted based on site needs and preferences.

2.6 Ensuring fidelity to REP and E-REP

Fidelity monitoring will be used to assess whether each site is receiving the core components of each implementation intervention (REP and E-REP). Fidelity to REP will be defined based on study team documentation of dates when trainings are offered and materials are uploaded to the OBI website, and ascertained via routine OBI administrative data. Fidelity to E-REP will be defined using the following criteria: (a) number of sessions offered and number of sessions completed by the facilitator with champions at each site; (b) number of documented barriers, facilitators, and specific goals to enhance CPG uptake; (c) documentation of site champion team strengths and available opportunities to influence site activities and overcome barriers; and (d) number of facilitation core components delivered [using tracking tools deployed in other large-scale implementation trials (56) and adapted for this study]. Data to determine E-REP fidelity will come from facilitation meeting logs completed by study staff. Throughout the trial, the study team will also collect data on adaptations made to REP and E-REP using the FRAME and FRAME-IS tools (57, 58).

2.7 Data sources

OBI has access to multiple data streams including a clinical registry, claims data from BCBSM and Medicaid plans (representing over 75% of market share in Michigan), and prescription drug monitoring program data with an approximately 90-day lag from the index procedure (59). The OBI clinical registry includes a subset of births (approximately 25,000 individuals in the nulliparous, term, singleton, vertex population are abstracted annually), and trial outcome measures will occur in this registry cohort as a signal of the unit's practices across its entire birthing population. Eligible births are abstracted to the OBI registry within 90 days by on-site clinical data abstractors at each OBI hospital. Our analytic sample will exclude those with opioid use disorder and complex pain, as these populations have different pain management needs than the broader birthing population.

Variables are collected and attributed to the hospital level to enable outcome adjustment for patient complexity (e.g., age, race and ethnicity, chronic conditions); procedural factors (e.g., mode of delivery); hospital characteristics (e.g., annual delivery volume, payer mix); and provider characteristics (e.g., specialty) (Table 2).

Table 2

Table 2. Collaborative data and covariates in the analysis.

2.8 Effectiveness outcomes

We will measure CPG effectiveness as implemented by REP or E-REP using prescribing outcomes and patient-reported outcomes (Table 3). Our primary outcomes include: (1) the proportion of patients receiving an opioid prescription at discharge; and (2) the median opioid prescription size at discharge reported in oral morphine equivalents (OMEs)—overall, by hospital, and among key subgroups. Secondary outcomes include refilled prescriptions (i.e., additional opioid prescriptions filled within 30 days of discharge after childbirth) and high-risk prescribing (i.e., prescription >50 OMEs/day, overlapping benzodiazepine and opioid prescriptions, or overlapping opioid prescriptions—all within 30 days of discharge after childbirth), which may elevate risk of opioid-related harms (60–62). Exploratory effectiveness outcomes include patient-reported pain intensity in the first week after childbirth, opioid consumption after discharge from the childbirth hospitalization, and satisfaction with pain management after discharge from the childbirth hospitalization—all captured by validated survey measures on surveys sent to eligible patients abstracted into the OBI registry (63, 64). We will examine all effectiveness outcomes overall and by procedure type (e.g., vaginal births without additional procedures, cesarean births).

Table 3

Table 3. Measures of CPG effectiveness as implemented with REP or E-REP.

2.9 Primary analysis of the association between REP and effectiveness outcomes

Interrupted time series analysis will compare monthly rates of each outcome (e.g., opioid prescribing rate and amount) at the hospital level, before and after CPG implementation with REP. This quasi-experimental approach allows us to estimate the causal effect of the CPG as implemented with REP and identify the attributes of hospitals that tend to respond to REP alone. All analyses will be performed at the hospital level in an intent-to-treat manner among all OBI hospitals (groups A, B, and C in Figure 1), regardless of fidelity to REP and E-REP. We will use aggregated outcomes within each procedure type in our primary analysis. Time series plots will be used to visually inspect the effect of the intervention and presence of trends, cyclical patterns, and outliers. Serial autocorrelation, non-stationarity, and seasonality will be assessed using Cumby-Huizinga tests, Dickey-Fuller unit root test, and visual inspection of residual plots. If autocorrelation is present, we can employ robust Newey-West errors with the appropriate number of lags.

2.9.1 Extensions of the analysis

We will construct mixed-effects logistic models for each effectiveness outcome (Table 3), incorporating time to detect trend and all covariates in Table 2 to explore heterogeneity in response to REP. The clustering effects of patients cared for by the same provider within a hospital will be accounted for to allow for REP's effects to vary at the provider/hospital level.

2.9.2 Sample size and power calculation

In our preliminary analyses, OBI's average hospital rate of opioid prescribing at discharge among pregnant individuals was estimated to be 26% and hospital-level intraclass correlation coefficient (ICC) regarding the prescribing patterns was 0.25. We conservatively estimate 68 hospitals participating in OBI throughout the study period, each with an average of 100 deliveries in three months. We have sufficient power (0.823) to detect a decline as small as 2% (from 26% to 24%) in the opioid prescribing rate. Based on these estimates, we have >0.81 power to detect even small declines in opioid prescribing across 68 hospitals that receive the COMFORT CPG implemented with REP. When stratified by mode of delivery, we estimated a power of 0.864 to detect a 3% reduction in prescribing rate (from 88% to 85%) among cesarean births and a power of 0.843 to detect a 1.2% reduction in prescribing rate (from 6% to 4.8%) among vaginal births.

2.10 Exploratory analysis of the effects of REP vs. E-REP among slower-responders

While our primary analysis uses an observational design to evaluate the CPG as implemented by REP, an immediately publicly available and scalable implementation intervention, our exploratory analyses will identify how best to remediate sites that do not rapidly respond to the CPG as implemented by REP alone. Analyses will include all registry patients undergoing childbirth at the randomized slower-responder hospitals during the second intervention stage and compare effectiveness outcomes after exposure to the CPG implemented by REP vs. E-REP. Mixed-effect logistic regression models will be used to analyze each effectiveness outcome, accounting for clustering of individuals within hospitals (i.e., a 3-level model that includes fixed effects for the intercept, provider, and site-specific baseline opioid prescribing rate). A random effect will be included for provider and site. We will cluster standard errors in a way that accounts for clustering of patients within providers within hospitals.

Exploratory Aim 3 analyses will assess whether the implementation intervention effectiveness is moderated by baseline or time-varying hospital factors, to capitalize on hospital-level heterogeneity to inform the potentially adaptive implementation intervention. From prior literature, we have identified several candidate variables for testing moderation of the effects of REP vs. E-REP. Specifically, we will assess whether implementation intervention effectiveness is moderated by early positive change in COMFORT practices (observed in trial months 1–6); higher proportion of attendings who are private practice physicians; perceived hospital administrator support for adoption of COMFORT (as reported by the QI team leading COMFORT implementation); and number of hospital barriers to COMFORT (as reported at first measurement post-randomization). Among rapid responders, we hypothesize that few, if any, hospitals will demonstrate worsening performance after determination of response status, so we do not have pre-specified moderator analyses for this group.

2.10.1 Power

An estimated 58 hospitals will be deemed slower-responders and randomized to either continue REP or to E-REP. With at least 100 patients per site, a mixed-effects logistic regression, provider-level ICC of 0.05, hospital-level ICC of 0.20, and two-sided alpha = 0.05, we have 85% power to detect an 8% change in compliance for opioid sparing prescribing, from 82% to 90%, by Month 6 post-randomization—a smaller difference than in our prior pilot work (65) that improved adherence from 38% to 61%. Among cesarean births, we estimated 0.838 power in detecting a 10% increase in compliance (from 80% to 90%), and among vaginal births, a power of 0.806 in detecting an 8% increase (from 82% to 90%).

2.11 Evaluation of implementation

We selected outcomes using the RE-AIM framework, which includes a comprehensive set of theory-based outcomes to evaluate implementation interventions (Table 4) (66). We will assess Reach, the number and proportion of patients receiving CPG-concordant care (defined by our primary analysis of effectiveness, examined pre/post CPG implementation, and our exploratory slower-responder analysis, examined post-implementation among slower-responder sites receiving REP vs. EREP), including by historically marginalized groups (e.g., populations marginalized by race or ethnicity), with a goal of identifying inequities in care delivery. Adoption metrics will characterize variation in hospital and provider uptake of CPGs and their attributes. Implementation outcomes: We will examine providers’ perceived feasibility, acceptability, and appropriateness of REP and E-REP (via validated measures on provider surveys and qualitative methods) (67, 68). The study team will also estimate the OBI Coordinating Center costs of creating REP and delivering E-REP facilitation using time-driven, activity-based costing methods (69, 70). We will estimate maintenance outcomes (for effectiveness, adoption, and reach metrics) at Month 24.

Table 4

Table 4. Implementation outcomes by RE-AIM domain.

Finally, we will conduct robust qualitative work to elucidate the mechanisms underlying observed response to REP and E-REP and effects of the CPG. We will purposively sample eight hospitals (one high-performer and one low-performer hospital from Group A and one high- and two low-performer hospitals from Group B and Group C in Figure 1, with performance defined by our primary effectiveness analyses). We will invite site champions and up to five designated key informants from these hospitals to participate in semi-structured interviews or focus groups. The diversity in OBI practice settings will allow for maximum flexibility in sampling of facilities based on CPG implementation performance and by any organizational determinants noted to be significant drivers of implementation. We will also conduct semi-structured interviews with patients (n = 3 per site), with participants purposively sampled from consenting patient-reported outcome survey respondents.

3 Discussion

This study will provide the first prospective evaluation of the effectiveness of the COMFORT CPG on postpartum pain management in a statewide collaborative quality initiative. To date, this is one of the only prospective studies to design a potentially adaptive site-level implementation intervention (REP) in real-world maternity settings in the U.S. This study is also one of the first to test the augmentation of REP with facilitation (E-REP) among maternity units that exhibit slower response after six months, as well as one of the first large-scale, prospective evaluations of implementation interventions designed to promote opioid stewardship within collaborative quality initiatives. This study will elucidate how best to adapt REP to the unique context of maternity care and whether augmentation of REP is needed, or whether in some circumstances, withholding augmentation may result in delayed implementation effect of REP alone among slower-responder maternity units.

Improving the quality of maternity care is a national priority in the U.S. and many other countries, but there is a paucity of rigorous data on effective maternity care QI interventions. There have been few rigorous trials of implementation interventions in real-world maternity settings (71). A recent systematic review including 15 large-scale improvement programs focused on intrapartum care in the English National Health Service identified weak or absent evaluation in most studies, suggesting an urgent need to improve the evaluation of improvement initiatives in order to optimize their impact and ensure accountability for how improvement resources are utilized and how lessons learned are aggregated to improve clinical outcomes (72). Without sound evaluation, resources may be spent on ineffective QI interventions, misdirected to sites that do not need additional support, or geared toward interventions or implementation strategies that do not benefit the most at-risk, marginalized populations and that perpetuate inequities in maternity care and outcomes.

REP has documented success in real-world healthcare settings, but QI leaders may face organizational barriers (e.g., lack of leadership buy-in, competing priorities, provider resistance) that are beyond the scope of REP's key components (i.e., QI toolkits, structured clinician training, performance feedback, and group mentorship). Addressing these complex barriers may require more sustained strategic thinking, problem-solving, and organizational alliances. Facilitation may provide the interpersonal support that equips and motivates site champions to generate highly customized, local solutions to these barriers—and, ultimately, more effective CPG implementation. Facilitation may also uncover barriers in the effectiveness and implementation of CPGs across certain groups of hospitals, providers, or patients and may support more focused strategies to close these gaps and achieve equity in care delivery. This study will help determine the value of REP and E-REP and their effects on COMFORT CPG uptake in maternity units and patient outcomes.

While this study has numerous strengths, including the use of a novel implementation study design within a real-world perinatal quality collaborative, there are limitations that should be noted. Planned facilitation fidelity measures do not include direct observations of sessions to support implementation at the site level due to resource constraints. There is also a chance of contamination between REP and E-REP, because OBI sites have the option to request additional support from Coordinating Center staff as needed. This risk is unlikely, as site-initiated requests are rare. In addition, while we have robust mechanisms to capture opioid prescribing, opioid consumption, and pain management, our capture of non-prescribed analgesic use (opioids/NSAIDS/acetaminophen) post-discharge is limited. Finally, although OBI hospitals represent a range of practice settings and patient populations served, they may not be representative of all practices and populations nationwide.

Nearly 4 million individuals undergo childbirth annually in the U.S. The COMFORT CPG—if effectively implemented—holds promise to reduce variation in postpartum pain management practices, reduce opioid risks, and promote more equitable care and outcomes. The results of this study will inform implementation of the COMFORT CPG in Michigan, as well as how to conduct practical implementation studies in perinatal quality collaboratives. This study will also enable those leading perinatal QI initiatives to determine the potential added value of more intensive implementation strategies and to tailor QI resource allocation across sites that vary in need of support for adoption of evidence-based maternity practices.

4 Ethics and dissemination

The QI interventions in this study involve provider education and consultation in use of guidelines that meet or exceed standards of practice. This study was reviewed by the University of Michigan institutional review board and deemed exempt in its use of survey and qualitative data collection and secondary analysis of existing data (HUM00248235 and HUM00248331).

Ethics statement

This study involving humans was approved by the University of Michigan Institutional Review Board and will be conducted in accordance with institutional requirements. Written informed consent for participation will not be not required, in accordance with institutional requirements.

Author contributions

MM: Conceptualization, Data curation, Investigation, Methodology, Writing – original draft, Writing – review & editing. AK: Methodology, Writing – review & editing. AP: Methodology, Writing – review & editing. JW: Data curation, Investigation, Methodology, Writing – review & editing. SC: Methodology, Writing – review & editing. CS: Methodology, Writing – review & editing. LK: Methodology, Writing – review & editing. MB: Methodology, Writing – review & editing. ME: Data curation, Investigation, Methodology, Writing – review & editing. MS: Data curation, Investigation, Methodology, Writing – review & editing. VG: Methodology, Writing – review & editing. AB: Methodology, Writing – review & editing. MH: Methodology, Writing – review & editing. CM: Methodology, Writing – review & editing. SS: Methodology, Writing – review & editing.

Funding

The author(s) declare that financial support was received for the research and/or publication of this article. Drs. MM, AK, AP, JW, LK, ME, SC, and MB, and Ms. CS and CM, received salary support from FDA Grant 1U01FD007803-01, as well as P50 DA054039-01, NIH NCATS 1UM1 TR004404, and UG3 HL154280. This project is supported by grant 1U01FD007803-01 from the Food and Drug Administration (FDA) of the U.S. Department of Health and Human Services (HHS) as part of a financial assistance award [FAIN] totaling $1,999,980.00, with 100 percent funded by FDA/HHS. The contents are those of the author(s) and do not necessarily represent the official views of, nor an endorsement, by FDA/HHS, or the U.S. Government. Support for the Obstetrics Initiative is provided by Blue Cross Blue Shield of Michigan and Blue Care Network as part of the BCBSM Value Partnerships program. Although Blue Cross Blue Shield of Michigan and OBI work collaboratively, the opinions, beliefs and viewpoints expressed by the author do not necessarily reflect the opinions, beliefs and viewpoints of BCBSM or any of its employees. Authors MM, AP, LK, MS, AB, CM, and SS receive salary support for their roles in OBI.

Acknowledgments

The authors gratefully acknowledge Sarah Evilsizer, Kimberly Brown, and Marisa Wetmore for their important contributions to the development of the REP package.

Conflict of interest

AP is a consultant for Maven and Mirvie and receives research support from Pulsenmore.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Generative AI statement

The author(s) declare that no Generative AI was used in the creation of this manuscript.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Abbreviations

CPG, clinical practice guideline; COMFORT, creating optimal pain management for tailoring care; QI, quality improvement; REP, replicating effective programs; E-REP, enhanced REP; OBI, obstetrics initiative; BCBSM, blue cross blue shield of michigan; NSAIDS, non-steroidal anti-inflammatory drugs; ICC, intraclass correlation coefficient; RE-AIM, reach, effectiveness, adoption, implementation, and maintenance.

References

1. National Academies of Sciences Engineering Medicine, Health and Medicine Division, Board on Health Care Services, Committee on Evidence-Based Clinical Practice Guidelines for Prescribing Opioids for Acute Pain. Evaluating Clinical Practice Guidelines for Prescribing Opioids for Acute Pain. Washington, D.C.: The National Academies Press (2019).

Google Scholar

2. National Academies of Sciences Engineering Medicine, Health and Medicine Division, Board on Health Care Services, Committee on Evidence-Based Clinical Practice Guidelines for Prescribing Opioids for Acute Pain. Framing Opioid Prescribing Guidelines for Acute Pain: Developing the Evidence. Washington, D.C.: The National Academies Press (2020).

Google Scholar

3. Niklasson B, Georgsson Ohman S, Segerdahl M, Blanck A. Risk factors for persistent pain and its influence on maternal wellbeing after cesarean section. Acta Obstet Gynecol Scand. (2015) 94:622–8. doi: 10.1111/aogs.12613

PubMed Abstract | Crossref Full Text | Google Scholar

4. Karlström A, Engström-Olofsson R, Norbergh KG, Sjöling M, Hildingsson I. Postoperative pain after cesarean birth affects breastfeeding and infant care. J Obstet Gynecol Neonatal Nurs. (2007) 36:430–40. doi: 10.1111/j.1552-6909.2007.00160.x

PubMed Abstract | Crossref Full Text | Google Scholar

5. Mo J, Ning Z, Wang X, Lv F, Feng J, Pan L. Association between perinatal pain and postpartum depression: a systematic review and meta-analysis. J Affect Disord. (2022) 312:92–9. doi: 10.1016/j.jad.2022.06.010

PubMed Abstract | Crossref Full Text | Google Scholar

6. Kinsey CB, Baptiste-Roberts K, Zhu J, Kjerulff KH. Birth-related, psychosocial, and emotional correlates of positive maternal-infant bonding in a cohort of first-time mothers. Midwifery. (2014) 30:e188–94. doi: 10.1016/j.midw.2014.02.006

PubMed Abstract | Crossref Full Text | Google Scholar

7. Eisenach JC, Pan PH, Smiley R, Lavand'homme P, Landau R, Houle TT. Severity of acute pain after childbirth, but not type of delivery, predicts persistent pain and postpartum depression. Pain. (2008) 140:87–94. doi: 10.1016/j.pain.2008.07.011

PubMed Abstract | Crossref Full Text | Google Scholar

8. Brummett CM, Waljee JF, Goesling J, Moser S, Lin P, Englesbe MJ, et al. New persistent opioid use after minor and major surgical procedures in US adults. JAMA Surg. (2017) 152:e170504. doi: 10.1001/jamasurg.2017.0504

PubMed Abstract | Crossref Full Text | Google Scholar

9. Peahl AF, Morgan DM, Dalton VK, Zivin K, Lai YL, Hu HM, et al. New persistent opioid use after acute opioid prescribing in pregnancy: a nationwide analysis. Am J Obstet Gynecol. (2020) 223:566.e1–e13. doi: 10.1016/j.ajog.2020.03.020

PubMed Abstract | Crossref Full Text | Google Scholar

10. Brummett CM, Evans-Shields J, England C, Kong AM, Lew CR, Henriques C, et al. Increased health care costs associated with new persistent opioid use after major surgery in opioid-naive patients. J Manag Care Spec Pharm. (2021) 27:760–71. doi: 10.18553/jmcp.2021.20507

PubMed Abstract | Crossref Full Text | Google Scholar

11. McCabe SE, West BT, Boyd CJ. Leftover prescription opioids and nonmedical use among high school seniors: a multi-cohort national study. J Adolesc Health. (2013) 52:480–5. doi: 10.1016/j.jadohealth.2012.08.007

PubMed Abstract | Crossref Full Text | Google Scholar

12. Brat GA, Agniel D, Beam A, Yorkgitis B, Bicket M, Homer M, et al. Postsurgical prescriptions for opioid naive patients and association with overdose and misuse: retrospective cohort study. Br Med J. (2018) 360:j5790. doi: 10.1136/bmj.j5790

PubMed Abstract | Crossref Full Text | Google Scholar

13. Hale TW, Krutsch K. Opioid use in breastfeeding mothers and neonatal risks. Clin Pharmacol Ther. (2021) 109:573–5. doi: 10.1002/cpt.2118

PubMed Abstract | Crossref Full Text | Google Scholar

14. Johnson JD, Asiodu IV, McKenzie CP, Tucker C, Tully KP, Bryant K, et al. Racial and ethnic inequities in postpartum pain evaluation and management. Obstet Gynecol. (2019) 134:1155–62. doi: 10.1097/AOG.0000000000003505

PubMed Abstract | Crossref Full Text | Google Scholar

15. Wiles A, Korn E, Dinglas C, Bentley B, Rosne J, Rahimi S. Disparities in post cesarean section pain management. J Clin Gynecol Obstet. (2022) 11:27–32. doi: 10.14740/jcgo786

Crossref Full Text | Google Scholar

16. Vedam S, Stoll K, Taiwo TK, Rubashkin N, Cheyney M, Strauss N, et al. The giving voice to mothers study: inequity and mistreatment during pregnancy and childbirth in the United States. Reprod Health. (2019) 16:77. doi: 10.1186/s12978-019-0729-2

PubMed Abstract | Crossref Full Text | Google Scholar

17. Johnson JD, Cooper S, Badreldin N, Green C. How can we get to equitable and effective postpartum pain control? Clin Obstet Gynecol. (2022) 65:577–87. doi: 10.1097/GRF.0000000000000731

PubMed Abstract | Crossref Full Text | Google Scholar

18. Felder L, Cao CD, Konys C, Weerasooriya N, Mercier R, Berghella V, et al. Enhanced recovery after surgery protocol to improve racial and ethnic disparities in postcesarean pain management. Am J Perinatol. (2022) 39:1375–82. doi: 10.1055/a-1799-5582

PubMed Abstract | Crossref Full Text | Google Scholar

19. Matthews KC, White RS, Ewing J, Abramovitz SE, Kalish RB. Enhanced recovery after surgery for cesarean delivery: a quality improvement initiative. Am J Perinatol. (2024) 41:e362–e8. doi: 10.1055/s-0042-1754405

PubMed Abstract | Crossref Full Text | Google Scholar

20. Green CA, Johnson JD, McKenzie C, Stuebe AM. Standardized order sets do not eliminate racial or ethnic inequities in postpartum pain management. Health Equity. (2023) 7:685–91. doi: 10.1089/heq.2022.0180

PubMed Abstract | Crossref Full Text | Google Scholar

21. Kilbourne AM, Neumann MS, Pincus HA, Bauer MS, Stall R. Implementing evidence-based interventions in health care: application of the replicating effective programs framework. Implement Sci. (2007) 2:42. doi: 10.1186/1748-5908-2-42

PubMed Abstract | Crossref Full Text | Google Scholar

22. Neumann MS, Sogolow ED. Replicating effective programs: HIV/AIDS prevention technology transfer. AIDS Educ Prev. (2000) 12:35–48.11063068

PubMed Abstract | Google Scholar

23. Kelly JA, Somlai AM, DiFranceisco WJ, Otto-Salaj LL, McAuliffe TL, Hackl KL, et al. Bridging the gap between the science and service of HIV prevention: transferring effective research-based HIV prevention interventions to community AIDS service providers. Am J Public Health. (2000) 90:1082–8. doi: 10.2105/AJPH.90.7.1082

PubMed Abstract | Crossref Full Text | Google Scholar

24. Bandura A. Social Learning Theory. Englewood Cliffs, NJ: Prentice-Hall (1977).

Google Scholar

25. Rogers EM. Diffusion of Innovations. 5th ed New York, NY: Simon & Schuster (2003).

Google Scholar

26. Kelly JA, Heckman TG, Stevenson LY, Williams PN, Ertl T, Hays RB, et al. Transfer of research-based HIV prevention interventions to community service providers: fidelity and adaptation. AIDS Educ Prev. (2000) 12:87–98.11063072

PubMed Abstract | Google Scholar

27. Kilbourne AM, Goodrich DE, Lai Z, Almirall D, Nord KM, Bowersox NW, et al. Reengaging veterans with serious mental illness into care: preliminary results from a national randomized trial. Psychiatr Serv. (2015) 66:90–3. doi: 10.1176/appi.ps.201300497

PubMed Abstract | Crossref Full Text | Google Scholar

28. Casalino LP, Elster A, Eisenberg A, Lewis E, Montgomery J, Ramos D. Will pay-for-performance and quality reporting affect health care disparities? Health Aff (Millwood). (2007) 26:w405–14. doi: 10.1377/hlthaff.26.3.w405

PubMed Abstract | Crossref Full Text | Google Scholar

29. Jha AK, Orav EJ, Epstein AM. The effect of financial incentives on hospitals that serve poor patients. Ann Intern Med. (2010) 153:299–306. doi: 10.7326/0003-4819-153-5-201009070-00004

PubMed Abstract | Crossref Full Text | Google Scholar

30. Bardach NS, Wang JJ, De Leon SF, Shih SC, Boscardin WJ, Goldman LE, et al. Effect of pay-for-performance incentives on quality of care in small practices with electronic health records: a randomized trial. JAMA. (2013) 310:1051–9. doi: 10.1001/jama.2013.277353

PubMed Abstract | Crossref Full Text | Google Scholar

31. Ryan AM, Blustein J, Doran T, Michelow MD, Casalino LP. The effect of phase 2 of the premier hospital quality incentive demonstration on incentive payments to hospitals caring for disadvantaged patients. Health Serv Res. (2012) 47:1418–36. doi: 10.1111/j.1475-6773.2012.01393.x

PubMed Abstract | Crossref Full Text | Google Scholar

32. Joynt KE, Jha AK. Characteristics of hospitals receiving penalties under the hospital readmissions reduction program. JAMA. (2013) 309:342–3. doi: 10.1001/jama.2012.94856

PubMed Abstract | Crossref Full Text | Google Scholar

33. Lindenauer PK, Remus D, Roman S, Rothberg MB, Benjamin EM, Ma A, et al. Public reporting and pay for performance in hospital quality improvement. N Engl J Med. (2007) 356:486–96. doi: 10.1056/NEJMsa064964

PubMed Abstract | Crossref Full Text | Google Scholar

34. Damberg CL, Shortell SM, Raube K, Gillies RR, Rittenhouse D, McCurdy RK, et al. Relationship between quality improvement processes and clinical performance. Am J Manag Care. (2010) 16:601–6.20712393

PubMed Abstract | Google Scholar

35. Smith SN, Almirall D, Prenovost K, Liebrecht C, Kyle J, Eisenberg D, et al. Change in patient outcomes after augmenting a low-level implementation strategy in community practices that are slow to adopt a collaborative chronic care model: a cluster randomized implementation trial. Med Care. (2019) 57:503–11. doi: 10.1097/MLR.0000000000001138

PubMed Abstract | Crossref Full Text | Google Scholar

36. Eisman AB, Hutton DW, Prosser LA, Smith SN, Kilbourne AM. Cost-effectiveness of the adaptive implementation of effective programs trial (ADEPT): approaches to adopting implementation strategies. Implement Sci. (2020) 15:109. doi: 10.1186/s13012-020-01069-w

PubMed Abstract | Crossref Full Text | Google Scholar

37. Kilbourne AM, Abraham KM, Goodrich DE, Bowersox NW, Almirall D, Lai Z, et al. Cluster randomized adaptive implementation trial comparing a standard versus enhanced implementation intervention to improve uptake of an effective re-engagement program for patients with serious mental illness. Implement Sci. (2013) 8:136. doi: 10.1186/1748-5908-8-136

PubMed Abstract | Crossref Full Text | Google Scholar

38. Kilbourne AM, Goodrich DE, Nord KM, Van Poppelen C, Kyle J, Bauer MS, et al. Long-term clinical outcomes from a randomized controlled trial of two implementation strategies to promote collaborative care attendance in community practices. Adm Policy Ment Health. (2015) 42:642–53. doi: 10.1007/s10488-014-0598-5

PubMed Abstract | Crossref Full Text | Google Scholar

39. Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, et al. A refined compilation of implementation strategies: results from the expert recommendations for implementing change (ERIC) project. Implement Sci. (2015) 10:21. doi: 10.1186/s13012-015-0209-1

PubMed Abstract | Crossref Full Text | Google Scholar

40. Berta W, Cranley L, Dearing JW, Dogherty EJ, Squires JE, Estabrooks CA. Why (we think) facilitation works: insights from organizational learning theory. Implement Sci. (2015) 10:141. doi: 10.1186/s13012-015-0323-0

PubMed Abstract | Crossref Full Text | Google Scholar

41. Kilbourne AM, Geng E, Eshun-Wilson I, Sweeney S, Shelley D, Cohen DJ, et al. How does facilitation in healthcare work? Using mechanism mapping to illuminate the black box of a meta-implementation strategy. Implement Sci Commun. (2023) 4:53. doi: 10.1186/s43058-023-00435-1

PubMed Abstract | Crossref Full Text | Google Scholar

42. Harvey G, Kitson A. PARIHS revisited: from heuristic to integrated framework for the successful implementation of knowledge into practice. Implement Sci. (2016) 11:33. doi: 10.1186/s13012-016-0398-2

PubMed Abstract | Crossref Full Text | Google Scholar

43. Kilbourne AM, Neumann MS, Waxmonsky J, Bauer MS, Kim HM, Pincus HA, et al. Public-academic partnerships: evidence-based implementation: the role of sustained community-based practice and research partnerships. Psychiatr Serv. (2012) 63:205–7. doi: 10.1176/appi.ps.201200032

PubMed Abstract | Crossref Full Text | Google Scholar

44. Smith SN, Almirall D, Choi SY, Koschmann E, Rusch A, Bilek E, et al. Primary aim results of a clustered SMART for developing a school-level, adaptive implementation strategy to support CBT delivery at high schools in Michigan. Implement Sci. (2022) 17:42. doi: 10.1186/s13012-022-01211-w

PubMed Abstract | Crossref Full Text | Google Scholar

45. Ritchie MJ, Parker LE, Edlund CN, Kirchner JE. Using implementation facilitation to foster clinical practice quality and adherence to evidence in challenged settings: a qualitative study. BMC Health Serv Res. (2017) 17:294. doi: 10.1186/s12913-017-2217-0

PubMed Abstract | Crossref Full Text | Google Scholar

46. Hamilton AB, Farmer MM, Moin T, Finley EP, Lang AJ, Oishi SM, et al. Enhancing mental and physical health of women through engagement and retention (EMPOWER): a protocol for a program of research. Implement Sci. (2017) 12:127. doi: 10.1186/s13012-017-0658-9

PubMed Abstract | Crossref Full Text | Google Scholar

47. Almirall D, Nahum-Shani I, Wang L, Kasari C. Experimental designs for research on adaptive interventions: singly and sequentially randomized trials. In: Collins LM, Kugler KC, editors. Optimization of Behavioral, Biobehavioral, and Biomedical Interventions. Cham, Switzerland: Springer International Publishing AG (2018). p. 89–120.

Google Scholar

48. Swindle T, Rutledge JM, Selig JP, Painter J, Zhang D, Martin J, et al. Obesity prevention practices in early care and education settings: an adaptive implementation trial. Implement Sci. (2022) 17:25. doi: 10.1186/s13012-021-01185-1

PubMed Abstract | Crossref Full Text | Google Scholar

49. Fitch K, Bernstein SJ, Aguilar MD, Burnand B, LaCalle JR, Lazaro P, et al. The RAND/UCLA Appropriateness Method User’s Manual. Santa Monica, CA: RAND Corporation (2001).

Google Scholar

50. Welch VA, Akl EA, Guyatt G, Pottie K, Eslava-Schmalbach J, Ansari MT, et al. GRADE equity guidelines 1: considering health equity in GRADE guideline development: introduction and rationale. J Clin Epidemiol. (2017) 90:59–67. doi: 10.1016/j.jclinepi.2017.01.014

PubMed Abstract | Crossref Full Text | Google Scholar

51. Akl EA, Welch V, Pottie K, Eslava-Schmalbach J, Darzi A, Sola I, et al. GRADE equity guidelines 2: considering health equity in GRADE guideline development: equity extension of the guideline development checklist. J Clin Epidemiol. (2017) 90:68–75. doi: 10.1016/j.jclinepi.2017.01.017

PubMed Abstract | Crossref Full Text | Google Scholar

52. Welch VA, Akl EA, Pottie K, Ansari MT, Briel M, Christensen R, et al. GRADE equity guidelines 3: considering health equity in GRADE guideline development: rating the certainty of synthesized evidence. J Clin Epidemiol. (2017) 90:76–83. doi: 10.1016/j.jclinepi.2017.01.015

PubMed Abstract | Crossref Full Text | Google Scholar

53. Pottie K, Welch V, Morton R, Akl EA, Eslava-Schmalbach JH, Katikireddi V, et al. GRADE equity guidelines 4: considering health equity in GRADE guideline development: evidence to decision process. J Clin Epidemiol. (2017) 90:84–91. doi: 10.1016/j.jclinepi.2017.08.001

PubMed Abstract | Crossref Full Text | Google Scholar

54. Shaver N, Bennett A, Beck A, Skidmore B, Traversy G, Brouwers M, et al. Health equity considerations in guideline development: a rapid scoping review. CMAJ Open. (2023) 11:E357–E71. doi: 10.9778/cmajo.20220130

PubMed Abstract | Crossref Full Text | Google Scholar

55. Sharif L, Cocroft S, Smith SN, Benincasa C, Peahl AF, Low LK, et al. Development of an implementation intervention to promote adoption of the COMFORT clinical practice guideline for peripartum pain management: a qualitative study. Implement Sci Commun. (2025) 6:1–17. doi: 10.1186/s43058-024-00687-5

PubMed Abstract | Crossref Full Text | Google Scholar

56. Kilbourne AM, Almirall D, Eisenberg D, Waxmonsky J, Goodrich DE, Fortney JC, et al. Protocol: adaptive implementation of effective programs trial (ADEPT): cluster randomized SMART trial comparing a standard versus enhanced implementation strategy to improve outcomes of a mood disorders program. Implement Sci. (2014) 9:132. doi: 10.1186/s13012-014-0132-x

PubMed Abstract | Crossref Full Text | Google Scholar

57. Miller CJ, Barnett ML, Baumann AA, Gutner CA, Wiltsey-Stirman S. The FRAME-IS: a framework for documenting modifications to implementation strategies in healthcare. Implement Sci. (2021) 16:36. doi: 10.1186/s13012-021-01105-3

PubMed Abstract | Crossref Full Text | Google Scholar

58. Strayer TE, Spalluto LB, Burns A, Lindsell CJ, Henschke CI, Yankelevitz DF, et al. Using the framework for reporting adaptations and modifications-expanded (FRAME) to study adaptations in lung cancer screening delivery in the veterans health administration: a cohort study. Implement Sci Commun. (2023) 4:5. doi: 10.1186/s43058-022-00388-x

PubMed Abstract | Crossref Full Text | Google Scholar

59. Michigan Department of Health & Human Services. Vital Statistics. Lansing, MI: Michigan Department of Health & Human Services (2025). Available online at: https://www.michigan.gov/mdhhs/inside-mdhhs/statisticsreports/vitalstats (Accessed May 15, 2024)

Google Scholar

60. Delaney LD, Gunaseelan V, Rieck H, Dupree J, Hallstrom BR, Waljee JF. High-risk prescribing increases rates of new persistent opioid use in total hip arthroplasty patients. J Arthroplasty. (2020) 35:2472–9.e2. doi: 10.1016/j.arth.2020.04.019

PubMed Abstract | Crossref Full Text | Google Scholar

61. Levy B, Paulozzi L, Mack KA, Jones CM. Trends in opioid analgesic-prescribing rates by specialty, U.S., 2007–2012. Am J Prev Med. (2015) 49:409–13. doi: 10.1016/j.amepre.2015.02.020

PubMed Abstract | Crossref Full Text | Google Scholar

62. Paulozzi LJ, Strickler GK, Kreiner PW, Koris CM. Centers for disease C, prevention. Controlled substance prescribing patterns–prescription behavior surveillance system, eight states, 2013. MMWR Surveill Summ. (2015) 64:1–14. doi: 10.15585/mmwr.ss6409a1

PubMed Abstract | Crossref Full Text | Google Scholar

63. Macones GA, Caughey AB, Wood SL, Wrench IJ, Huang J, Norman M, et al. Guidelines for postoperative care in cesarean delivery: enhanced recovery after surgery (ERAS) society recommendations (part 3). Am J Obstet Gynecol. (2019) 221:247.e1–e9. doi: 10.1016/j.ajog.2019.04.012

PubMed Abstract | Crossref Full Text | Google Scholar

64. Smith SN, Lai Z, Almirall D, Goodrich DE, Abraham KM, Nord KM, et al. Implementing effective policy in a national mental health reengagement program for veterans. J Nerv Ment Dis. (2017) 205:161–70. doi: 10.1097/NMD.0000000000000592

PubMed Abstract | Crossref Full Text | Google Scholar

65. Howard R, Hallway A, Santos-Parker J, Vu J, Waljee J, Brummett CM, et al. Optimizing postoperative opioid prescribing through quality-based reimbursement. JAMA Netw Open. (2019) 2:e1911619. doi: 10.1001/jamanetworkopen.2019.11619

PubMed Abstract | Crossref Full Text | Google Scholar

66. Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. (1999) 89:1322–7. doi: 10.2105/AJPH.89.9.1322

PubMed Abstract | Crossref Full Text | Google Scholar

67. Proctor EK, Landsverk J, Aarons G, Chambers D, Glisson C, Mittman B. Implementation research in mental health services: an emerging science with conceptual, methodological, and training challenges. Adm Policy Ment Health. (2009) 36:24–34. doi: 10.1007/s10488-008-0197-4

PubMed Abstract | Crossref Full Text | Google Scholar

68. Weiner BJ, Lewis CC, Stanick C, Powell BJ, Dorsey CN, Clary AS, et al. Psychometric assessment of three newly developed implementation outcome measures. Implement Sci. (2017) 12:108. doi: 10.1186/s13012-017-0635-3

PubMed Abstract | Crossref Full Text | Google Scholar

69. Ninerola A, Hernandez-Lara AB, Sanchez-Rebull MV. Improving healthcare performance through activity-based costing and time-driven activity-based costing. Int J Health Plann Manage. (2021) 36:2079–93. doi: 10.1002/hpm.3304

PubMed Abstract | Crossref Full Text | Google Scholar

70. Cidav Z, Mandell D, Pyne J, Beidas R, Curran G, Marcus S. A pragmatic method for costing implementation strategies using time-driven activity-based costing. Implement Sci. (2020) 15:28. doi: 10.1186/s13012-020-00993-1

PubMed Abstract | Crossref Full Text | Google Scholar

71. Callaghan-Koru JA, Moniz MH, Hamm RF. Prioritize implementation research to effectively address the maternal health crisis. Am J Obstet Gynecol. (2021) 225:212–3. doi: 10.1016/j.ajog.2021.02.005

PubMed Abstract | Crossref Full Text | Google Scholar

72. McGowan J, Attal B, Kuhn I, Hinton L, Draycott T, Martin GP, et al. Quality and reporting of large-scale improvement programmes: a review of maternity initiatives in the English NHS, 2010–2023. BMJ Qual Saf. (2024) 33:704–15. doi: 10.1136/bmjqs-2023-016606

PubMed Abstract | Crossref Full Text | Google Scholar