An external facilitation intervention to increase uptake of an adverse drug event reporting intervention

Background: Adverse drug events (ADEs) are a leading cause of emergency department visits and hospital admissions in Canada. ActionADE prevents repeat ADEs by enabling clinicians to document and communicate standardized ADE information across care settings. We used an external facilitation intervention to promote the uptake of ActionADE in four hospitals in British Columbia, Canada. This study examined whether, how and in what context external facilitation influenced the uptake of ActionADE.

Methods: In this convergent-parallel mixed-methods study, an external facilitator used a four-step iterative process to support site champions using context-specific implementation strategies to increase the ADE reporting rate at their sites. We extracted archival data to assess implementation determinants before and after the implementation of the external facilitation and implementation strategies. We also retrieved data on the mean monthly counts of reported ADEs for each user from the ActionADE server. Zero-inflated Poisson models were used to examine changes in mean monthly counts of reported ADEs per user between pre-intervention (June 2021 to October 2021) and intervention (November 2021 to March 2022) periods.

Results: The external facilitator and site champions co-created three functions: (1) educate pharmacists about what and how to report in ActionADE, (2) educate pharmacists about the impact of ActionADE on patient outcomes, and (3) provide social support for pharmacists to integrate ADE reporting into clinical workflows. Site champions used eight forms to address the three functions. Peer support and reporting competition were the two common strategies used by all sites. Sites’ responses to external facilitation varied. The rate of mean monthly counts of reported ADEs per user significantly increased during the intervention period compared to the pre-intervention period at LGH (RR: 3.74, 95% CI 2.78 to 5.01) and RH (RR: 1.43, 95% CI 1.23 to 1.94), but did not change at SPH (RR: 0.68, 95% CI: 0.43 to 1.09) and VGH (RR: 1.17, 95% CI 0.92 to 1.49). Leave of absence of the clinical pharmacist champion and failure to address all identified functions were implementation determinants that influenced the effectiveness of external facilitation.

Conclusion: External facilitation effectively supported researchers and stakeholders to co-create context-specific implementation strategies. It increased ADE reporting at sites where clinical pharmacist champions were available, and where all functions were addressed.

Background

Adverse drug events (ADEs)—harmful and unintended events related to medication use—are a leading cause of patient harm, and a burden on health systems (1–4). One in nine adult visits to the emergency department is caused by an ADE. Of those visits, one in three are repeat events (5). Repeat ADEs occur because clinicians may be unaware of patients' ADE histories when prescribing. Different health settings, such as hospitals, long-term care facilities and clinics, often use different clinical information systems that do not automatically exchange ADE information, leading to information discontinuity (5). Effective system-level interventions are needed to address this communication gap (6).

ActionADE is software that enables healthcare providers to document and share ADE information using standardized terminologies in a user-friendly electronic format (7–9). ActionADE (8) has been integrated with British Columbia's provincial medication dispensing database, PharmaNet, to automatically share ADE information documented in hospitals, where patients with severe and acute ADEs commonly seek care. This allows care providers in other health sectors (e.g., community clinics and pharmacies) across the province who have access to PharmaNet to access ADE information. Through systems integration, PharmaNet presents community pharmacists with standardized ADE alerts if they attempt to re-dispense a medication or medication of the same class for which the patient has an ADE recorded in PharmaNet. Preliminary data shows that ActionADE prevents repeat ADEs in 10.8% of patients with reports shared to PharmaNet (10), supporting the preliminary effectiveness of ActionADE in preventing re-exposure to culprit medication”.

Noteworthy, valuable clinical interventions scarcely implement themselves. The use of effective strategies to implement evidence-based interventions into clinical practice is necessary to ensure that patients receive the benefit (11). Implementation strategies are methods or techniques used to improve adoption, implementation, sustainment, and scale-up of interventions (12). The field of implementation science has made significant progress to generate evidence for implementation strategies in the past two decades, with published reviews and taxonomies describing over 70 strategies, such as audit and feedback and educational outreach (12–16). Selecting the most appropriate implementation strategies for clinical interventions requires thorough understanding of implementation determinants (i.e., barriers and enablers) across multiple levels of stakeholders and settings in the dynamic and complex healthcare system (15, 17, 18) However, the literature offers limited evidence on methods for doing so effectively (17, 19, 20).

External facilitation offers a promising approach to align implementation strategy with determinants. External facilitation is a multi-faceted process whereby external implementation experts work with stakeholders to promote interactive problem-solving and knowledge exchange that supports the adoption and use of an evidence-based practice (21–23). Key components of external facilitation include assessing the contexts, assisting teams in identifying problems and developing implementation strategies, monitoring, and providing feedback around the change efforts (24–26). External facilitation has been effective in improving the uptake of various health interventions such as antenatal care (27), postpartum care, peer specialist service (28), opioid use disorder treatment (29), and psychosocial intervention for homelessness (30). There is growing evidence suggesting that external facilitation is effective in improving health intervention implementation (29). However, most studies did not provide clear and explicit descriptions of the facilitation process, which prevented others from repeating and adapting this approach. A systematic review synthesized evidence from 195 facilitation studies to identify the role and characteristics of facilitation, and found only six studies explicitly described the actual process (21). Moreover, we know little regarding context-specific effectiveness, particularly within multi-site interventions (25, 31). Previous multi-site studies found that the effects of external facilitation on intervention uptake were variable across sites, but the factors contributing to such variations has yet to be identified (28, 29, 32).

The objectives of this study were to examine whether, how and in which context external facilitation influenced the uptake of ActionADE. We aimed to address four research questions:

1. What were the implementation determinants that influenced uptake of ActionADE before the external facilitation?

2. What implementation strategies were used by each site to promote ActionADE uptake?

3. What were the effects of external facilitation on the mean monthly counts of reported ADEs per user?

4. What were the implementation determinants that influenced uptake of ActionADE during external facilitation?

Methods

Study design

We used a convergent-parallel mixed-methods design. We collected quantitative data from archival data and qualitative data from meeting notes during the study period. We analyzed quantitative and qualitative results separately and then triangulated the findings when interpreting the results (33).

Setting

Since December 2020, nine hospitals have adopted ActionADE, with four engaging in active change management to onboard new users and sustain reporting. After a 10-month pilot implementation (January to October 2021) to secure stakeholder buy-in, we initiated an external facilitation intervention to increase the uptake of ActionADE among frontline providers. We presented the characteristics of the four participating hospitals in Table 1. The four participating hospitals were Lions Gate Hospital (LGH), Richmond Hospital (RH), St Paul's Hospital (SPH) and Vancouver General Hospital (VGH). All are in the Greater Vancouver area within the Vancouver Coastal Health Authority. All are acute care hospitals serving urban areas, and each had two pharmacists coordinating the implementation of ActionADE. LGH and RH are smaller urban community hospitals with fewer emergency department visits, fewer clinical areas covered by pharmacists, and fewer onsite clinical pharmacists. SPH and VGH are tertiary and quaternary urban teaching hospitals, respectively. All sites were involved in developing ActionADE, with pilot testing occurring at VGH.

TABLE 1

Table 1. Site characteristics.

External facilitation intervention

We conducted external facilitation between November 2021 and March 2022. External facilitation aimed to increase the use of ActionADE by supporting site champions to develop and use implementation strategies that fit their contexts. We hypothesized that the external facilitation process would lead to the use of context-specific strategies and increase mean monthly counts of reported ADEs per user.

A research team member (EL) with training in implementation science and knowledge about the implementation settings served as the external facilitator (EF) to provide strategic and methodological support to site champions. EF used a four-step iterative process guided by previous facilitation studies (Figure 1) (29, 34).

FIGURE 1

Figure 1. The four iterative steps of the external facilitation intervention for promoting uptake of ActionADE.

Step 1: formative evaluation

The formative evaluation aimed to identify implementation determinants (i.e., factors that influence implementation success or failure) (35) influencing pharmacists' use of ActionADE before the external facilitation intervention. To identify implementation determinants, the EF analyzed meeting minutes during the implementation planning and pilot implementation phases. The EF then categorized the identified implementation determinants according to the Consolidated Framework for Implementation Research (CFIR) (36). Next, the EF met with champions at each site to refine the list of identified implementation determinants.

Step 2: co-create implementation strategies

To develop implementation strategies, the EF met with site champions to co-create a list of strategies targeting determinants identified in step 1. They then operationalized the strategies by specifying the name (10), purposes, action (the specific activities or processes that need to be enacted), the actors (who acts the strategy), and action target (target population of the actions) (37).

Step 3: generate an implementation plan

The EF developed, discussed, and refined an implementation plan with all site champions, which outlined the context, purpose, scope, timeline, target outcomes, and implementation strategies to increase the uptake of ActionADE. The plan was a living document for the EF to provide updates on ActionADE usage and document changes in implementation strategies and contextual factors that influence implementation at each site. The EF shared the plan with the site champions electronically.

Step 4: execute, monitor, and evaluate

Site champions executed the implementation plan, while the EF and research team monitored the process and evaluated outcomes through bi-weekly emails and monthly meetings. The EF met with site champions monthly to review utilization statistics, revisit implementation determinants, and modify the implementation strategies, if needed. During the post-intervention period, the EF met with champions at each site to obtain feedback for the external facilitation intervention and discuss determinants identified at pre-intervention or that emerged during the intervention.

Outcome measures

Qualitative outcomes were implementation determinants reported before and after implementation of the external facilitation intervention, the functions and forms of the implementation strategies, and contextual factors influencing the uptake of ActionADE during the intervention. We extracted data on the implementation determinants before the intervention from meeting minutes documented between December 2020 to October 2021 (ActionADE pilot implementation period). The research team used a template for recording meeting minutes. A research team member recorded the date, time, purposes, and attendees of the meeting. The note-taker also recorded key discussion points, decisions, and action items. We extracted data on the implementation strategies used at each site and contextual factors from meeting notes documented between November 2021 and May 2022 (during the external facilitation intervention). These meeting notes captured opinions from research team members, patient partners, pharmacists at the participating hospitals, and site champions. Monthly meetings embedded within the external facilitation intervention offered a conducive environment for site champions to recall their implementation strategies. This approach was ideal for obtaining frequent feedback and specific perspectives on time-sensitive issues.

The main quantitative outcome was the mean monthly count of reported ADEs per user. We retrieved data on the mean monthly counts of reported ADEs for each individual user from the ActionADE server between June 2021 and March 2022 (10 months). We included data from pharmacists who registered for an ActionADE account before 1 June 2021 and held active employment without leaves at the same hospital throughout the study period.

Data analysis

We analyzed qualitative data by thematically summarizing the meeting minutes. We coded the implementation determinants and the contextual factors according to the Consolidation Framework for Implementation Research (CFIR) qualitative data codebook (38). To describe implementation strategies, we drew upon the concepts of functions and forms, a crucial concept to guide the development of complex, adaptable and scalable innovations (39, 40). Functions are the purpose of a set of activities, why it matters and how it produces changes in the expected outcomes. Forms are a set of activities used to meet the functions (37, 39, 41). For example, in the context of ActionADE, a function could be educating pharmacists on what to report in ActionADE. The form for the first site could be delivering information about ActionADE reporting criteria to pharmacists in a group presentation, while the form for the second site could be delivering the same information through a user manual. The EF shared a summary of the qualitative findings with research team members, and the team subsequently reached a consensus about the implementation strategies and determinants through discussion. Qualitative analyses were conducted using NVivo 11 qualitative data analysis software (QSR International).

For quantitative data, we used descriptive statistics to calculate the means and standard deviations. We measured the effects of the external facilitation on the mean monthly counts of reported ADEs per user using zero-inflated Poisson models. We selected this model because exploratory analyses showed that the distribution of participants' mean monthly counts of reported ADEs was overdispersed (i.e., mean and variance differ significantly) and contained an excess of zeros created by non-adopters (42). To account for these issues, the model optimizes the estimations by creating two regression equations: the logit component for predicting excess zero counts and the typical Poisson component for predicting differences in the occurrence of the count (42, 43).

Given the heterogeneity of site characteristics and implementation strategies, we stratified the analysis by site. The model included the mean monthly counts of reported ADEs per user between June 2021 to March 2022 as the dependent variable and time as the independent variable. We treated time as a categorical variable, with 0 indicating the pre-intervention period (June 2021 to October 2021) and 1 for the intervention period (November 2021 to March 2022). We also tested a random effect term to account for repeated measurements nested within users. The random effects were not statistically significant in models for LGH, SPH and VGH. The model did not converge for RH's model likely due to a small sample size. Therefore, we removed the random effect term in the final models for RH. We validated the model by plotting the predicted and observed residual values from the models. The level of significance was set at p < 0.05. We conducted quantitative statistical analyses using SAS 9.4 (SAS).

The model produced two sets of estimates: a logistic component that yielded the odds ratios predicting the odds of having zero monthly counts of reported ADEs per user, a Poisson component that yielded the rate ratios (RRs) of the mean monthly counts of reported ADEs per user between the pre and during the intervention period after adjusting for excess zeroes by the logistic component (43). With a focus on the effects of the external facilitation on the mean monthly counts of reported ADEs per user, hereinafter, we presented and interpreted the RRs from the Poisson component only.

Results

Research question 1: what were the implementation determinants that influenced uptake of ActionADE before the external facilitation?

The formative evaluation identified four categories of implementation determinants that were common across sites: available resources, compatibility with workflow, relative priority and providers' knowledge and belief.

Available resources

All site champions noted lack of dedicated staff time as a major barrier to implementing ActionADE. They noted that staff shortages and turnover impacted reporting. Site champions at RH and LGH stated that they were smaller hospitals with fewer resources per patient compared to other sites.

Compatibility

Site champions stated that pharmacists had difficulties fitting ActionADE into their existing workflows. At the time of the study, pharmacists were unable to directly access ActionADE in the health information system being used without searching for it or receive visual reminders for ADE reporting through their local electronic medical records systems. Without streamlining the process, site champions felt that pharmacists were uncertain about the stage during care provision they should integrate ADE reporting into their workflow. When a patient transitioned between care areas (e.g., from the emergency department to an in-patient ward) there was no mechanism to support the handover of patients' ADE information across service locations.

Relative priority

At the time of the intervention new initiatives, such as COVID-19 vaccinations and training of new hires (due to the high staff turnover rate), competed with ActionADE implementation activities. With staff shortages, pharmacists were stressed, and experienced burnout and change fatigue. In this context, the site champions noted that pharmacists might have been less likely to prioritize ADE reporting.

Providers' knowledge and belief

Site champions noted that some pharmacists had questions about the types of ADEs to report (e.g., non-adherence, refuted allergy) and about specific data fields. Site champions noted that some pharmacists had not yet seen the impact of ADE reporting on patient care. Site champions suggested that these perceptions may explain why some pharmacists were reluctant to adopt the intervention.

Research question 2: what implementation strategies were used by each site to promote ActionADE uptake?

During step 2 of the external facilitation process, the EF and site champions co-created functions and forms for the implementation strategies based on the implementation determinants identified in step 1. The EF and site champions recognized the complexity of addressing implementation determinants related to available resources, relative priority and compatibility of ActionADE with other health information systems. Increasing the number of pharmacists and changing organizational priorities for ADE reporting were not feasible functions, and beyond the capacity of the research team. Similarly, more fulsome integration into other health information systems requires infrastructure from multisectoral collaboration (e.g., data standards, data privacy regulations and technological infrastructure), which could not be accomplished over the course of five months. Due to these constraints, site champions suggested improving pharmacists' education around the clinical impact that ActionADE could have on patient outcomes to motivate them to prioritize time for ADE reporting and providing social support for pharmacists to integrate ActionADE into clinical workflow. Table 2 describes the three functions co-created by the EF and site champions: (1) educate pharmacists about what and how to report in ActionADE, (2) educate pharmacists about the impact of ActionADE on patient outcomes and (3) provide social support for pharmacists to integrate ActionADE into clinical workflow. We operationalized social support as supports accessible to an individual through social ties to other individuals and groups, such as encouragement from a co-worker (44).

TABLE 2

Table 2. Implementation strategies (functions and forms) used by each site during the external facilitation.

Site champions developed and used eight distinct forms to address the three functions (Table 2). Noteworthy, LGH, RH and SPH delivered forms meeting all three functions, while VGH addressed functions 1 and 3 only. All sites employed two common forms: peer support and reporting competitions. Peer support included site champions providing reminders, verbal encouragement and troubleshooting to pharmacists at their sites. Reporting competitions consisted of one individual-based and two team-based challenges in which pharmacists competed for prizes awarded to the top three reporters across sites individually or with a team of 2 to 3 members from the same site. Winners of the reporting competitions received gift cards to redeem for merchandise. Both forms encouraged pharmacists to integrate ActionADE into their clinical workflow by creating a social milieu for ADE reporting. Each site used slightly different forms to address the functions to fit their contexts. For instance, LGH employed educational meetings and materials to address function 1, while VGH used educational materials and 1-on-1 follow-up.

Each site operationalized the same form slightly differently. All sites used peer support but targeted different sub-groups. LGH focused on pharmacists in the emergency department; RH targeted pharmacists at different service locations; SPH targeted all clinical pharmacists, and VGH focused on less frequent users. Pharmacists were the action targets for forms. Site champions were the primary actors for most forms, with the research team assisting in the deployment. For instance, three site champions identified the need to develop and re-distribute ActionADE educational materials. The site champions were the ones who decided the content, format, and distribution channels of the educational materials. The research team supported them by sharing existing educational materials or tailoring new materials as requested.

Research question 3: what were the effects of external facilitation on the mean monthly counts of reported ADEs per user?

The analytical sample included 146 pharmacist users and 1,460 observations. The mean monthly counts of reported ADEs per user were 0.57 ± 1.24 at pre-intervention compared to 0.94 ± 3.23 during the intervention period. The mean monthly counts of reported ADEs per user were steady during the pre-intervention period and fluctuated during the intervention period across all sites; the counts for LGH, RH and VGH reached the peak during the intervention period (Figure 2).

FIGURE 2

Figure 2. Mean monthly counts of reported ADEs per user at pre-intervention and intervention period by site.

Results from modelling showed that the rate of mean monthly counts of reported ADEs per user were significantly higher during the intervention period at LGH and RH, but null results were observed at SPH and VGH. The rate of mean monthly counts of reported ADEs per user during the intervention period was 3.74 times (RR: 3.74, 95% CI 2.78 to 5.01), 1.43 times (RR: 1.43, 95% CI 1.23 to 1.94) the rate for the pre-intervention period at LGH and RH, respectively. There was no difference in the rate between the pre-intervention and intervention periods at SPH (RR: 0.68, 95% CI: 0.43 to 1.09) and at VGH (RR: 1.17, 95% CI 0.92 to 1.49) (Table 3). Model validation plots showed that the observed and predicted values aligned closely, indicating that models fit the data well (data not shown).

TABLE 3

Table 3. Rate ratios of mean monthly counts of reported ADEs during the pre-intervention vs. intervention periods for the total sample and by site.

Research question 4: what were the implementation determinants that influenced uptake of ActionADE during the external facilitation?

During and after the external facilitation, the EF discussed implementation determinants that may have influenced the ADE reporting rates and corresponding solutions.

Available resources

As with the pre-intervention period, lack of dedicated staff time was the most frequently discussed barrier. Due to a higher rate of staff turnover and sick calls during the COVID-19 pandemic, clinical pharmacists faced higher workloads. This issue impacted not only ActionADE use but other patient care activities more broadly. The SPH site champion noted that staff turnover issues had impacted the reporting rates significantly during the intervention period because one clinical pharmacist champion went on parental leave unexpectedly early right after the external facilitation intervention began. The other site champion had limited time to move implementation activities forward.

Providers' knowledge and belief

Our qualitative data showed that discussions around providers' knowledge about ActionADE were less frequent after implementation of the external facilitation intervention. Site champions had noted that only a few pharmacists had questions about the eligibility of reporting for specific cases and duplicate reports. Regarding providers' beliefs, site champions reported that some pharmacists hesitated to use ActionADE because they feared that their reports would “scare prescribers” and take away necessary medications that should be re-dispensed.

Relative priority

Pharmacists were pulled into different initiatives during the intervention period, which may have influenced pharmacists' willingness and ability to use ActionADE. For example, the champion at LGH mentioned that pharmacists tended to prioritize treatment over preventive work. The champions at SPH also noted that the hospital prioritized admitted patients and hired a team of pharmacists to review their medications. However, many ADEs were identified in patients who were discharged from the ED who were not prioritized for medication review by some sites. Site champions also mentioned low levels of physician engagement may have prevented pharmacists from prioritizing ADE reporting in a team-based approach.

Other determinants included discontinuation of reporting competitions that had been designed to create a social milieu to stimulate ADE reporting. Site champions also suggested that regular reporters' work rotation schedules led to fluctuating monthly ADE report counts over time. While pharmacists were on a rotation with dispensary shifts, they rarely saw patients and would not encounter ADEs.

Discussions

This mixed-methods study examined whether, how, and in which contexts external facilitation increased the uptake of ActionADE. Consistent with previous research, we found that external facilitation was effective in increasing the uptake of ActionADE, but effects varied by sites (28, 29, 32). We observed significant increase in ADE reporting at LGH and RH but null effects at SPH and VGH. The significant intervention effects at LGH and RH suggested external facilitation can be effective in improving intervention uptake by assisting clinical teams in developing tailored strategies based on the implementation determinants. The EF and site champions co-created three functions during the external facilitation process. They included educating pharmacists about what and how to report in ActionADE, educating pharmacists about the impact of ActionADE on patient outcomes and providing social support for pharmacists to integrate ADE reporting in the clinical workflow. The identified functions were similar to the recommended practices for implementing new digital services into the routine work of healthcare professionals by Nadva et al. (45). We added value to the existing literature by providing a menu of forms for each function, which future studies can adopt, test and adapt. Developing functions and the corresponding menu of forms is important for others to replicate an intervention or implementation strategy. Very few studies have provided explicit guidance on adapting an evidence-based practice to fit local contexts (46). Specifying the functions and forms of an intervention or implementation strategy provides other researchers or practitioners with explicit guidance and options about which adaptations to the intervention's form are allowable while preserving fidelity (46, 47).

The positive intervention effect was more profound in LGH than in RH. We did not observe differences in implementation strategies or determinants between the two sites. Thus, we attributed the variable effects to other factors not measured in this study. A potential factor could be the characteristics of individual users. Compared to RH, users in LGH appeared to be more responsive to the strategies, particularly during the months with patient partner presentations and reporting competitions. This speculation is consistent with previous research. Rycroft-Malone et al. (48) found that individual characteristics are prominent in the interaction between context and strategies. Staff members' learning skills and motivation significantly influenced the effectiveness of facilitation on research uptake. We attempted to survey users' perceptions of the implementation strategies, but the response rate was very low amid the pandemic. Future studies are needed to explore how user characteristics interact with determinants at different levels (e.g., organizations level) in influencing the process and effectiveness of external facilitation.

The null effects in the other two sites provided insights into the contexts in which external facilitation was less effective. We attributed the null intervention effect at VGH to the failure to address all the identified functions. VGH was the only site that did not address function 2, which was to educate pharmacists about the potential impact of ActionADE on patient outcomes. The null intervention effects at VGH suggested that all three functions identified through the external facilitation must be addressed to achieve the expected outcome. Gustavson and colleagues (29) examined the effects of external facilitation on increasing use of medication treatment for opioid use disorder in nine veteran health administration facilities. They observed a significant increase in program uptake in facilities who achieved almost all the implementation goals. Previous evidence also supported that perceived benefits of the intervention were an important determinant for changing clinical practices among healthcare professionals (36, 49, 50). Future studies with a larger sample size and experimental design are needed to verify this finding.

SPH used a similar set of forms as LGH and RH but did not result in a significant improvement in ADE reporting rate. Our qualitative data suggested that the leave of absence of a key site champion during the external facilitation intervention may have attributed to the null intervention effects. As mentioned by the SPH champion, the absence of the clinical pharmacist champion substantially limited the execution of implementation activities and engagement with other pharmacists. This finding was not surprising because previous research consistently indicated that use of program champions was a critical implementation determinant for healthcare interventions (51–53). Two randomized trials (54, 55) tested the impact of program champions on changing clinical practices in healthcare professionals. McCabe et al. (54) found that the presence of a formally identified, designated champions was associated with an increase in residential aged care staff sensitivity to depression among residents. Bentz et al. (55) reported that the presence of clinical champions associated with an increased rate of referral to a state-level smoking quit line.

One interesting finding was that the external facilitation could not address several essential implementation determinants (i.e., staffing shortages, competing demands) that were out of our team's control, but it nonetheless achieved a significant improvement in ADE reporting at two sites. A plausible explanation was that being able to address other determinants, including providers' knowledge and belief, may have partially offset the negative impact from staffing shortages and competing demands. Previous studies suggested that implementation determinants interact synergistically to influence implementation success. A determinant that is perceived as less influential may be a preceding factor to improve another determinant (56, 57). For example, in ActionADE, improving pharmacists' belief might be a preceding factor to address staff shortage issue. Once pharmacists recognized the impact of ActionADE in improving patient outcomes, they may have been more motivated to prioritize their time for ADE reporting. However, we need future studies to verify these speculations. In the current study, we were unable to fully integrate ActionADE into the electronic medical record workflow due to resource constraints. Future work should assess the effectiveness of improved workflow integration compared to other strategies.

Reflections and lessons learned

When implementing ActionADE at multiple sites, adaptation to the local context was necessary to meet the diverse individuals' needs in order to avoid diminished intervention benefits (58). We found that external facilitation was an effective strategy to help implementation teams to identify the needs and focus of the adaptation. The functions and forms concept provided a new way of thinking when designing implementation strategies for a complex intervention undertaken in a complex health system. The function and form concept helped the team emphasize the intended function or purpose of the strategy instead of the dose (e.g., one or three training sessions). It also offered a practical tool for distinguishing between standardized and adaptable elements of the intervention or implementation strategies. However, conducting external facilitation was not without challenges. The external facilitation process was intense. It involved frequent communications with program champions, detailed records, and a rapid and timely evaluation-feedback loop. Nonetheless, the frequent contacts and in-depth evaluation of the contexts were beneficial for both parties to build a trusting relationship and co-create strategies that fit. The intense process was also necessary to keep the project on our site champions' agenda against other competing priorities. With the positive experiences, we decided to extend the facilitation intervention and continue to adapt and monitor changes in implementation strategies.

Our findings should be interpreted with the following limitations in mind. We conducted this study in four hospitals in one geographic location, limiting our findings' generalisability. Second, our qualitative data primarily captured the perspectives of the site champions, which may not be representative of all ActionADE users. We measured most of the implementation outcomes based on archival data. While this approach may not be able to explore a comprehensive list of implementation determinants, it provided a conducive and practical approach to capture longitudinal changes in providers' perceptions, contextual factors and implementation strategies. In this exploratory study, we did not assess fidelity of the implementation strategies. We need further studies to verify the effectiveness of the identified forms in addressing the functions and corresponding determinants.

Conclusion

This study offers new insights on whether, how, and in what contexts external facilitation promoted the uptake of clinical intervention. Our findings showed that external facilitation can be effective in promoting the uptake of ActionADE across multiple hospitals by assisting clinical teams in developing tailored strategies (functions and forms) based on pre-assessed implementation determinants. However, its effectiveness varied depending on the site's ability to deliver the identified strategies and the emergence of new determinants. Future studies are needed to examine the long-term success of external facilitation and strengthen the evidence base regarding factors influencing effectiveness of external facilitation.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The University of British Columbia Research Ethics Board approved all study procedures (H21–02973). This study involved the secondary use of de-identified archival data. ActionADE users were informed of and had to accept the app's privacy policy describing how de-identified data may be used for research purposes and presented in aggregate.

Author contributions

All authors contributed to the study's conception and design. EL: conducted the external facilitation, collected and analyzed the data and wrote the first draft of the manuscript. AC, CH, SS and KB: contributed to the refinement of the facilitation process. SS and KB: assisted with the external facilitation process. GL and SS: assisted in data collected. AC, CH: provided consultation to the statistical analysis and result interpretation. All authors contributed to the interpretation of the findings and commented on previous manuscript versions. All authors contributed to the article and approved the submitted version.

Funding

This study is funded by the Canadian Institutes of Health Research (CIHR #417477), with support from the Ministry of Health British Columbia, Vancouver Coastal Health, the Lower Mainland Consolidated Pharmacies, and the BC SUPPORT Unit. CMH is supported by the Michael Smith Foundation for Health Research Health Professional-Investigator award. EL is supported by the Michael Smith Health Research BC Research Trainee Award (GR018656). The funder has no role in the design, conduct, analysis, interpretation of data, or the drafting of scientific manuscripts

Acknowledgments

We acknowledge the site champions and users of ActionADE for their participation in this study. We would like to extend our sincere appreciation to Dr. Ellen Balka for consulting on the qualitative methodology of this study.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

1. Hohl CM, Nosyk B, Kuramoto L, Zed PJ, Brubacher JR, Abu-Laban RB, et al. Outcomes of emergency department patients presenting with adverse drug events. Ann Emerg Med. (2011) 58(3):270–9.e4. doi: 10.1016/j.annemergmed.2011.01.003

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Mouton JP, Mehta U, Parrish AG, Wilson DPK, Stewart A, Njuguna CW, et al. Mortality from adverse drug reactions in adult medical inpatients at four hospitals in South Africa: a cross-sectional survey. Br J Clin Pharmacol. (2015) 80(4):818–26. doi: 10.1111/bcp.12567

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Zed PJ, Abu-Laban RB, Balen RM, Loewen PS, Hohl CM, Brubacher JR, et al. Incidence, severity and preventability of medication-related visits to the emergency department: a prospective study. CMAJ. (2008) 178(12):1563–9. doi: 10.1503/cmaj.071594

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Hohl CM, Badke K, Zhao A, Wickham ME, Woo SA, Sivilotti MLA, et al. Prospective validation of clinical criteria to identify emergency department patients at high risk for adverse drug events. Acad Emerg Med. (2018) 25(9):1015–26. doi: 10.1111/acem.13407

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Hohl CM, Woo SA, Cragg A, Wickham ME, Ackerley CA, Scheuermeyer F, et al. Repeat adverse drug events associated with outpatient medications: a descriptive analysis of 3 observational studies in British Columbia, Canada. CMAJ Open. (2019) 7(3):E446–E53. doi: 10.9778/cmajo.20180190

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Hohl C, Lexchin JR, Balka E. Can reporting of adverse drug reactions create safer systems while improving health data? CMAJ. (2015) 187(11):789. doi: 10.1503/cmaj.150057

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Peddie D, Small SS, Badke K, Bailey C, Balka E, Hohl CM. Adverse drug event reporting from clinical care: mixed-methods analysis for a Minimum required dataset. JMIR Med Inform. (2018) 6(2):e10248-e. doi: 10.2196/10248

CrossRef Full Text | Google Scholar

8. Peddie D, Small SS, Badke K, Wickham ME, Bailey C, Chruscicki A, et al. Designing an adverse drug event reporting system to prevent unintentional reexposures to harmful drugs: study protocol for a multiple methods design. JMIR Res Protoc. (2016) 5(3):e169. doi: 10.2196/resprot.5967

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Hau JP, Brasher PMA, Cragg A, Small S, Wickham M, Hohl CM. Using ActionADE to create information continuity to reduce re-exposures to harmful medications: study protocol for a randomized controlled trial. Trials. (2021) 22(1):119. doi: 10.1186/s13063-021-05061-7

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Cragg A, Small SS, Lau EY, Rowe A, Lau A, Butcher K, et al. Benefits of introducing a user-friendly adverse drug event reporting platform with new data sharing functionality: a cohort study Canadian CMAJ. Drug Safety. Submitted.

Google Scholar

11. Grimshaw J, Shirran L, Thomas R, Mowatt G, Fraser C, Bero L, et al. Changing provider behaviour: an overview of systematic reviews of interventions to promote implementation of research findings by healthcare professionals. In: Haines A, Donald A (eds). Med Care. (2001) 39(8 Suppl 2):II2–45.

Google Scholar

12. Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, et al. A refined compilation of implementation strategies: results from the expert recommendations for implementing change (ERIC) project. Implement Sci. (2015) 10:21. doi: 10.1186/s13012-015-0209-1

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Mazza D, Bairstow P, Buchan H, Chakraborty SP, Van Hecke O, Grech C, et al. Refining a taxonomy for guideline implementation: results of an exercise in abstract classification. Implement Sci. (2013) 8(1):32. doi: 10.1186/1748-5908-8-32

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Cook CR, Lyon AR, Locke J, Waltz T, Powell BJ. Adapting a compilation of implementation strategies to advance school-based implementation research and practice. Prev Sci. (2019) 20:914–35. doi: 10.1007/s11121-019-01017-1

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Bero LA, Grilli R, Grimshaw JM, Harvey E, Oxman AD, Thomson MA. Closing the gap between research and practice: an overview of systematic reviews of interventions to promote the implementation of research findings. The cochrane effective practice and organization of care review group. Br Med J. (1998) 317(7156):465–8. doi: 10.1136/bmj.317.7156.465

CrossRef Full Text | Google Scholar

16. Lau R, Stevenson F, Ong BN, Dziedzic K, Treweek S, Eldridge S, et al. Achieving change in primary care—effectiveness of strategies for improving implementation of complex interventions: systematic review of reviews. BMJ Open. (2015) 5(12):e009993. doi: 10.1136/bmjopen-2015-009993

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Baker R, Camosso-Stefinovic J, Gillies C, Shaw EJ, Cheater F, Flottorp S, et al. Tailored interventions to address determinants of practice. Cochrane Database Syst Rev. (2015) 4:CD005470. doi: 10.1002/14651858.CD005470.pub3

CrossRef Full Text | Google Scholar

18. Grol R, Bosch M, Wensing M. Development and selection of strategies for improving patient care. In: Grol R, Wensing M, Eccles M, Davis D (eds). Improving patient care: The implementation of change in health care (2013). p. 165–84.

19. Powell BJ, Beidas RS, Lewis CC, Aarons GA, McMillen JC, Proctor EK, et al. Methods to improve the selection and tailoring of implementation strategies. J Behav Health Serv Res. (2017) 44(2):177–94. doi: 10.1007/s11414-015-9475-6

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Powell BJ, Fernandez ME, Williams NJ, Aarons GA, Beidas RS, Lewis CC, et al. Enhancing the impact of implementation strategies in healthcare: a research agenda. Front Public Health. (2019) 7(3):1–9. doi: 10.3389/fpubh.2019.00003

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Cranley LA, Cummings GG, Profetto-McGrath J, Toth F, Estabrooks CA. Facilitation roles and characteristics associated with research use by healthcare professionals: a scoping review. BMJ Open. (2017) 7(8):e014384. doi: 10.1136/bmjopen-2016-014384

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Gozdzik A. Applying the PARiHS framework in a knowledge dissemination initiative. CANNT J. (2013) 23(2):48–50. PMID:2406969824069698

PubMed Abstract | Google Scholar

23. Stetler CB, Legro MW, Rycroft-Malone J, Bowman C, Curran G, Guihan M, et al. Role of “external facilitation” in implementation of research findings: a qualitative evaluation of facilitation experiences in the veterans health administration. Implement Sci. (2006) 1(23):1–15. doi: 10.1186/1748-5908-1-23

CrossRef Full Text | Google Scholar

24. Dogherty EJ, Harrison M, Graham I, Keeping-Burke L. Examining the use of facilitation within guideline dissemination and implementation studies in nursing. JBI Evid Implement. (2014) 12(2):105–27. doi: 10.1097/XEB.0000000000000008

CrossRef Full Text | Google Scholar

25. Bidassie B, Williams LS, Woodward-Hagg H, Matthias MS, Damush TM. Key components of external facilitation in an acute stroke quality improvement collaborative in the veterans health administration. Implement Sci. (2015) 10:69. doi: 10.1186/s13012-015-0252-y

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Dogherty EJ, Harrison MB, Graham ID. Facilitation as a role and process in achieving evidence-based practice in nursing: a focused review of concept and meaning. Worldviews Evid Based Nurs. (2010) 7(2):76–89. doi: 10.1111/j.1741-6787.2010.00186.x

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Persson L, Nga NT, Målqvist M, Thi Phuong Hoa D, Eriksson L, Wallin L, et al. Effect of facilitation of local maternal-and-newborn stakeholder groups on neonatal mortality: cluster-randomized controlled trial. PLoS Med. (2013) 10(5):e1001445. doi: 10.1371/journal.pmed.1001445

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Chinman M, Goldberg R, Daniels K, Muralidharan A, Smith J, McCarthy S, et al. Implementation of peer specialist services in VA primary care: a cluster randomized trial on the impact of external facilitation. Implement Sci. (2021) 16(1):60. doi: 10.1186/s13012-021-01130-2

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Gustavson AM, Wisdom JP, Kenny ME, Salameh HA, Ackland PE, Clothier B, et al. Early impacts of a multi-faceted implementation strategy to increase use of medication treatments for opioid use disorder in the veterans health administration. Implement Sc Commun. (2021) 2(1):20. doi: 10.1186/s43058-021-00119-8

CrossRef Full Text | Google Scholar

30. Smelson DA, Yakovchenko V, Byrne T, McCullough MB, Smith JL, Bruzios KE, et al. Testing implementation facilitation for uptake of an evidence-based psychosocial intervention in VA homeless programs: a hybrid type III trial. PLoS One. (2022) 17(3):e0265396. doi: 10.1371/journal.pone.0265396

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Berta W, Cranley L, Dearing JW, Dogherty EJ, Squires JE, Estabrooks CA. Why (we think) facilitation works: insights from organizational learning theory. Implement Sci. (2015) 10:141. doi: 10.1186/s13012-015-0323-0

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Kauth MR, Sullivan G, Blevins D, Cully JA, Landes RD, Said Q, et al. Employing external facilitation to implement cognitive behavioral therapy in VA clinics: a pilot study. Implement Sci. (2010) 5(1):75. doi: 10.1186/1748-5908-5-75

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Creswell JW. Research design: qualitative, quantitative and mixed methods approaches. 4 ed. California: Thousand Oaks (2014).

34. Dogherty EJ, Harrison MB, Baker C, Graham ID. Following a natural experiment of guideline adaptation and early implementation: a mixed-methods study of facilitation. Implement Sci. (2012) 7(1):9. doi: 10.1186/1748-5908-7-9

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Sarkies M, Long JC, Pomare C, Wu W, Clay-Williams R, Nguyen HM, et al. Avoiding unnecessary hospitalisation for patients with chronic conditions: a systematic review of implementation determinants for hospital avoidance programmes. Implement Sci. (2020) 15(1):91. doi: 10.1186/s13012-020-01049-0

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. (2009) 4(1):50. doi: 10.1186/1748-5908-4-50

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Kirk MA, Haines ER, Rokoske FS, Powell BJ, Weinberger M, Hanson LC, et al. A case study of a theory-based method for identifying and reporting core functions and forms of evidence-based interventions. Transl Behav Med. (2019) 11(1):21–33. doi: 10.1093/tbm/ibz178

CrossRef Full Text | Google Scholar

38. Research CFfIRT-CfCM. Qualitative data–data analysis 2022. Available at: https://cfirguide.org/evaluation-design/qualitative-data/

39. Hawe P. Lessons from Complex interventions to improve health. Annu Rev Public Health. (2015) 36(1):307–23. doi: 10.1146/annurev-publhealth-031912-114421

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Perez Jolles M, Lengnick-Hall R, Mittman BS. Core functions and forms of Complex health interventions: a patient-centered medical home illustration. J Gen Intern Med. (2019) 34(6):1032–8. doi: 10.1007/s11606-018-4818-7

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Proctor EK, Powell BJ, McMillen JC. Implementation strategies: recommendations for specifying and reporting. Implement Sci. (2013) 8(1):139. doi: 10.1186/1748-5908-8-139

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Hall DB. Zero-inflated poisson and binomial regression with random effects: a case study. Biometrics. (2000) 56(4):1030–9. doi: 10.1111/j.0006-341X.2000.01030.x

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Liu W, Cela J, editors. Count data model in SAS. In: SAS Global forum SAS Institute Inc. 2018. Proceedings of the SAS® Global Forum 2018 Conference. Cary, NC: SAS Institute Inc 2008. (2008):1–12.

44. Lin N, Ensel WM, Simeone RS, Kuo W. Social support, stressful life events, and illness: a model and an empirical test. J Health Soc Behav. (1979) 20(2):108–19. doi: 10.2307/2136433

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Nadav J, Kaihlanen AM, Kujala S, Laukka E, Hilama P, Koivisto J, et al. How to implement digital services in a way that they integrate into routine work: qualitative interview study among health and social care professionals. J Med Internet Res. (2021) 23(12):e31668. doi: 10.2196/31668

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Gesell SB, Prvu Bettger J, Lawrence RH, Li J, Hoffman J, Lutz BJ, et al. Implementation of complex interventions: lessons learned from the patient-centered outcomes research institute transitional care portfolio. Med Care. (2021) 59(Suppl 4):S344–s54. doi: 10.1097/MLR.0000000000001591

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Esmail LC, Barasky R, Mittman BS, Hickam DH. Improving comparative effectiveness research of Complex health interventions: standards from the patient-centered outcomes research institute (PCORI). J Gen Intern Med. (2020) 35(2):875–81. doi: 10.1007/s11606-020-06093-6

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Rycroft-Malone J, Seers K, Chandler J, Hawkes CA, Crichton N, Allen C, et al. The role of evidence, context, and facilitation in an implementation trial: implications for the development of the PARIHS framework. Implement Sci. (2013) 8:28. doi: 10.1186/1748-5908-8-28

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Sanson-Fisher RW. Diffusion of innovation theory for clinical change. Med J Aust. (2004) 180(S6):S55–S6. doi: 10.5694/j.1326-5377.2004.tb05947.x

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Feijt MA, de Kort YAW, Bongers IMB, Ijsselsteijn WA. Perceived drivers and barriers to the adoption of eMental health by psychologists: the construction of the levels of adoption of eMental health model. J Med Internet Res. (2018) 20(4):e153. doi: 10.2196/jmir.9485

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Shea CM, Belden CM. What is the extent of research on the characteristics, behaviors, and impacts of health information technology champions? A scoping review. BMC Med Inform Decis Mak. (2016) 16(1):2. doi: 10.1186/s12911-016-0240-4

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Miech EJ, Rattray NA, Flanagan ME, Damschroder L, Schmid AA, Damush TM. Inside help: an integrative review of champions in healthcare-related implementation. SAGE Open Med. (2018) 6:2050312118773261. doi: 10.1177/2050312118773261

PubMed Abstract | CrossRef Full Text | Google Scholar

53. Santos WJ, Graham ID, Lalonde M, Demery Varin M, Squires JE. The effectiveness of champions in implementing innovations in health care: a systematic review. Implement Sci Commun. (2022) 3(1):80. doi: 10.1186/s43058-022-00315-0

PubMed Abstract | CrossRef Full Text | Google Scholar

54. McCabe MP, Karantzas GC, Mrkic D, Mellor D, Davison TE. A randomized control trial to evaluate the beyondblue depression training program: does it lead to better recognition of depression? Int J Geriatr Psychiatry. (2013) 28(3):221–6. doi: 10.1002/gps.3809

PubMed Abstract | CrossRef Full Text | Google Scholar

55. Bentz CJ, Bayley KB, Bonin KE, Fleming L, Hollis JF, Hunt JS, et al. Provider feedback to improve 5A's Tobacco cessation in primary care: a cluster randomized clinical trial. Nicotine Tob Res. (2007) 9(3):341–9. doi: 10.1080/14622200701188828

PubMed Abstract | CrossRef Full Text | Google Scholar

56. Popp J, Grüne E, Carl J, Semrau J, Pfeifer K. Co-creating physical activity interventions: findings from a multiple case study using mixed methods. Front Public Health. (2022) 10:1–15. doi: 10.3389/fpubh.2022.975638

CrossRef Full Text | Google Scholar

57. Lau EY, Wandersman AH, Pate RR. Factors influencing implementation of youth physical activity interventions: an expert perspective. TJACSM. (2016) 1(7):60–70. doi: 10.1249/TJX.0000000000000006

CrossRef Full Text | Google Scholar

58. Siu AL, Zimbroff RM, Federman AD, DeCherrie LV, Garrido M, Morano B, et al. The effect of adapting hospital at home to facilitate implementation and sustainment on program drift or voltage drop. BMC Health Serv Res. (2019) 19(1):264. doi: 10.1186/s12913-019-4063-8

PubMed Abstract | CrossRef Full Text | Google Scholar