Using the exploration, preparation, implementation, sustainment (EPIS) framework to assess the cooperative re-engagement controlled trial (CoRECT)

Background: “Data to Care” (D2C) is a strategy which relies on a combination of public health surveillance data supplemented by clinic data to support continuity of HIV care. The Cooperative Re-Engagement Controlled Trial (CoRECT) was a CDC-sponsored randomized controlled trial of a D2C model, which provided an opportunity to examine the process of implementing an intervention for people with HIV (PWH) who are out-of-care across three public health department jurisdictions. Using the EPIS (Exploration, Preparation, Implementation, Sustainment) framework, we aimed to retrospectively describe the implementation process for each site to provide insights and guidance to inform future D2C activities implemented by public health agencies and their clinical and community partners.

Methods: After completion of CoRECT, the three (Connecticut, Massachusetts, Philadelphia) trial sites reviewed study protocols and held iterative discussions to describe and compare their processes regarding case identification, interactions with partnering clinics and patients, and sustainability. The EPIS framework provided a structure for comparing key organizational and operational practices and was applied to the entire implementation process.

Results: The trial sites varied in their implementation processes and the specific elements of the intervention. Factors including prior D2C experience, data management and analytic infrastructure, staff capacity, and relationships with clinic partners informed intervention development and implementation. Additionally, this review identified key lessons learned including to: (1) explore new supplemental sources for public health surveillance data; (2) work with stakeholders representing core functions/components in the early stages of the intervention design process; (3) build flexibility into all components of the follow-up activities; and (4) integrate data sharing, project management, and follow-up activities within existing DPH organizational structure.

Conclusion: The CoRECT study provides a general blueprint and lessons learned for implementing a D2C intervention for re-engagement in HIV care. Interventions should be tailored to local operational and structural factors, and responsive to evolving clinical and public health practices.

Introduction

The National HIV/AIDS Strategy, the United States plan for addressing and ending the HIV epidemic, prioritizes increasing rapid linkage to care and improving retention in care to achieve viral suppression (1, 2). The HIV care continuum is a conceptual public health tool used by the Centers for Disease Control and Prevention (CDC) to monitor progress toward reaching national goals and is comprised of the diagnosis, linkage, and retention steps necessary to achieve viral suppression. Identified implementation gaps guide potential opportunities to improve engagement in care (3).

Various models of care have been developed to improve retention in care and viral suppression among people with HIV infection (PWH). An initial step in these care models is to define PWH who are out-of-care (OOC), which can vary (4). The CDC, the Health Resources and Services Administration (HRSA), and the Institute of Medicine (IOM, now National Academy of Medicine) offer differing definitions of retention in HIV care (1, 5–8). For example, the CDC defines retention in care as having at least two CD4 cell counts or viral load tests performed at least 3 months apart within 1 year, while the IOM and HRSA define retention as at least two medical visits every 12 months, with a minimum of 90 days between visits (5, 8).

Different clinic-based and public health strategies have been used to improve retention in HIV care. “Data to Care” (D2C), is one strategy that uses a combination of public health surveillance and other, often clinical, data to increase the number of PWH receiving sustained HIV care and viral suppression (9, 10). The CDC developed a toolkit for designing D2C interventions in 2017. Implementation across diverse health department jurisdictions, however, varies based on available data, methods for data collection, data management systems and processes, data sharing policies, and health department capacity, funding, and infrastructure (4).

The Cooperative Re-Engagement Controlled Trial (CoRECT) was a CDC-sponsored randomized control trial (RCT) of a D2C model with a research goal to establish data-sharing partnerships between health departments and HIV care clinical providers designed to identify and re-engage PWH that are OOC. This goal was operationalized by: (1) identifying newly OOC PWH through a collaborative data-sharing approach using public health surveillance and clinical data; and (2) implementing a public health outreach intervention to improve HIV care re-engagement, retention in care, and viral suppression for OOC PWH. Each of these elements differed by site. All three sites in this trial (Connecticut, Massachusetts, and Philadelphia health departments), found that a collaborative D2C strategy was successful at re-engaging PWH into HIV care at 90 days compared to the standard of care (11). Only the Philadelphia site resulted in downstream benefits of significantly improved retention in care at 12 months and viral suppression (10).

CoRECT’s D2C approach varied among the three participating health department jurisdictions and therefore provides fertile ground for compiling experiences and insights across multiple contexts. The objective of this study was to use a standardized implementation framework, EPIS (Exploration, Preparation, Implementation, Sustainment), to describe the implementation process for each of the three CoRECT sites as part of a retrospective assessment in order to better understand and appraise the components of the D2C intervention. Defining, clarifying, and analyzing D2C implementation within the EPIS framework enabled us to articulate lessons learned that can inform delivery, sustainability and replicability of future D2C activities.

Methods

Overview of the CDC’s cooperative re-engagement controlled trial

CoRECT was a multisite, randomized controlled trial to implement and evaluate a public health intervention compared to clinic standard of care (SOC) for re-engaging OOC PWH, using D2C. Participants were recruited from 8/2016–7/2018. The a priori definition for OOC included receiving HIV care at a participating clinic for at least 12 months and then subsequently neither a visit with a prescribing provider nor CD4 or viral load test result reported to health department surveillance for at least 6 months. Eligible OOC patients were randomized within each clinic to the clinic SOC for patient follow-up, or a D2C health department intervention in which patient follow-up was performed by public health field epidemiologists/Disease Intervention Specialists (DIS). There was a basic structure to the trial across all sites and participating clinics, but specific intervention characteristics varied across the three sites, with adjustments at each clinic consistent with variations in workflow, staffing, and capacity. Each site used a “case conferencing” methodology where health department staff interacted with clinic staff to reach consensus on persons who were OOC and eligible for randomization. After case randomization, public health field staff undertook patient outreach activities culminating in a warm handoff to clinic staff. The health department project management teams monitored surveillance data to determine re-engagement, viral suppression, and retention outcomes.

The Connecticut Department of Public Health/Yale School of Medicine (CTDPH/YSM), the Massachusetts Department of Public Health (MDPH), and the Philadelphia Department of Public Health (PDPH) participated as the three CoRECT sites. In 2016, Connecticut, Massachusetts, and Philadelphia reported approximately 10,286 (335 per 100,000), 20,060 (342 per 100,000), and 19,113 (1,252 per 100,000) PWH residing in their jurisdictions, respectively (12, 13). These sites estimated about 24 to 54% of their PWH met the OOC definition.

The CoRECT intervention allowed for site-specific flexibility regarding the case identification process and the re-engagement follow-up activities. Figure 1 illustrates how each site adapted the standard definition for case identification based on variations in workflow, data infrastructure, and communication processes at the sites and participating clinics. Table 1 provides further details regarding differences by site in the areas of case identification, case conference characteristics, randomization, and re-engagement follow-up activities, including field epidemiologist/DIS characteristics and contact methods, and intervention characteristics.

FIGURE 1

Figure 1. Description of case identification procedures used by each CoRECT site.

TABLE 1

Table 1. Organizational and operational aspects of CoRECT in the participating jurisdictions.

The exploration, preparation, implementation, and sustainment framework

While the EPIS framework was originally developed to aid in the development, execution, and sustainment of research interventions in a prospective manner, we chose to use this framework retrospectively to review implementation outcomes, which has been done previously elsewhere. Our rational for including implementation outcomes is based on a systematic review that suggests that most RCTs fail to report implementation outcomes, including retrospectively (14, 15). The four distinct phases – exploration, preparation, implementation, and sustainment – represent stages in the development and execution of a research intervention (16, 17). Each phase can be parsed into the outer context (e.g., the service and policy environment, target population characteristics, and inter-organizational relationships), inner context (e.g., leadership, organizational structures and resources, and staffing, of the implementing organization which can vary), innovation (e.g., how it fits into the organization, provider, and patient levels), and bridging factors (e.g., community-academic partnerships). Multiple factors comprise each of the contextual levels. The factors that comprise the outer context include sociopolitical, funding, interorganizational networks, and leadership, while organizational characteristics, individual adopter characteristics, leadership, participant recruitment, fidelity monitoring/support, and staffing make up the inner context. Both innovation and bridging factors are unifactorial; innovation characteristics and community and academic partnerships, respectively (14, 16). Reviewing the factors during each phase allows for the identification of barriers and facilitators to implementation.

Retrospective assessment of CoRECT implementation

To understand the different outcomes at each site, the three jurisdictions decided to evaluate implementation and associated processes to inform continued and future interventions, and identify key lessons that could assist further implementation of D2C programs. The process consisted of reviewing study protocols and holding discussions among primary investigators and project team members from each site. Using the EPIS framework, sites described and documented their processes for case identification, collaboration, and communication with partnering clinics, and interactions with patients (see Table 1 for details). The EPIS framework was selected because of its utility in understanding implementation within real-world settings and allowance for the systematic comparison of the trial sites according to key organizational and operational practices, enabling analysis of future delivery and sustainability of each CoRECT intervention component.

Results

Organizational and operational characteristics of DPH sites

The three trial sites differed in their implementation of CoRECT study activities. CTDPH/YSM recruited 23 clinics across four counties (out of eight counties statewide) and included community health centers, hospital-based, and other clinics funded by federal Ryan White HIV/AIDS Program. MDPH recruited nine clinics across four counties (out of 14 counties statewide) and included community health centers, a hospital-based practice, a public health hospital, and non-profit, academically affiliated teaching hospitals. Participating clinics represented a mix of funding sources (including Ryan White) and included facilities that received no dedicated HIV services funding. PDPH recruited eight clinical sites all within the City of Philadelphia. These participating clinics included Ryan White-funded community health centers and academic medical centers, as well as a Veterans Health Administration medical center and a private clinic (Table 1).

Table 1 provides a detailed description of each of the 3 sites organized by study population, case conferencing methodology, randomization strategy, field epidemiologist/DIS characteristics, contact methods and intervention characteristics.

Analysis of the CoRECT intervention elements using EPIS framework

Figure 2 illustrates how implementation of CoRECT fits within the four phases of the framework. The following highlights key selected findings within each of the 4 phases: Exploration, Preparation, Implementation and Sustainment.

FIGURE 2

Figure 2. CoRECTs implementation within the Exploration, Preparation, Implementation, Sustainment (EPIS) framework.

Exploration phase

Interorganizational networks

Strengthening collaboration between health departments and healthcare clinics was an explicit, secondary goal of CoRECT. Through strengthened collaboration, CoRECT provided sites with new tools to assist patients and healthcare providers with re-establishing engagement in HIV care (Supplementary Table 1).

Leadership

The leadership across the three participating DPHs prioritized applying data to public health action, developing/adopting methods for promoting and supporting quality improvement, promoting equity in access to healthcare, and achieving health outcomes. Additionally, primary consideration was given to addressing known gaps in HIV care including using novel D2C methods for improving care engagement.

Organizational characteristics

CTDPH, PDPH, and MDPH had varying levels of experience with D2C activities and participation in other HIV care research studies. Although CTDPH had no prior D2C experience, they relied on experienced HIV surveillance staff and prior experience working with Yale researchers on other related projects. PDPH staff had some prior D2C experience, in addition to the pilot, which preceded their CoRECT enrollment to test patient identification procedures. Prior to CoRECT, PDPH had received Gilead funding for START Care, which assessed data exchanges with providers to determine OOC patients. Unlike CoRECT, providers were responsible for directly administering the intervention. MDPH had prior D2C experience through SPECTRuM and Partnerships for Care (18–20).

Staffing

The three sites used their existing field investigation/patient follow-up activities as a base for implementing their public health field intervention. MDPH used existing field staff and integrated OOC engagement and navigation activities into existing functions associated with public health follow-up of syphilis and acute HIV, while PDPH and CTDPH/YSM hired designated CoRECT field staff focused primarily on re-engagement of OOC clients and assisted with other outreach activities as needed. The field staff (DIS) in Connecticut were hired and housed by local health departments (Bridgeport, New Haven, Hartford). PDPH hired and managed designated CoRECT field staff through STD Control Program within the Division of Disease Control. CTDPH/YSM coordinated their CoRECT outreach activities based on a combination of local and state HD practices. All three sites provided training on study procedures to field staff; training of clinic staff regarding study goals and procedures varied at participating clinics.

Data management staff varied among the sites. For MDPH, CoRECT data management activities were integrated with routine HIV/STD surveillance activities and coordinated by dedicated D2C staff. PDPH relied on a D2C data manager to work closely with a D2C project coordinator, both within the HIV Surveillance Unit, which would coordinate intervention follow-up activities with STD Control Program. Unlike the previous sites, CTDPH relied on their collaborators at YSM to perform data management including randomization and communication with local HDs.

Innovation characteristics

Sites had the flexibility to tailor both of the main study components (i.e., the patient identification process and patient follow-up). For example, sites used the initial general patient identification protocol (Figure 1) as a basis for discussion with partnering clinics about how to adjust the protocol to fit their individual operational and structural factors and abilities.

For follow-up of OOC patients by DIS, sites designed feasible public health outreach and patient navigation activities which encompassed short-term re-engagement assistance (similar to traditional public health disease intervention and follow-up) and longer-term case management (similar to traditional HIV medical case management or HIV prevention case management). CTDPH/YSM (Yale School of Medicine) implemented short-term patient re-engagement assistance through the use of a modified Anti-Retroviral Treatment and Access to Services (ARTAS) (21, 22) intervention. MDPH implemented a middle range approach by having a focus on short-term assistance with the possibility of longer-term assistance and motivational counseling for patients who needed or requested it, an adaptation of their standard procedures for persons with newly diagnosed HIV. MDPH further tailored patient follow-up activities to minimize collisions with ongoing HIV care initiatives. PDPH approach prioritized longer-term patient re-engagement assistance based on identified barriers to care and they modified their ARTAS approach by not including incentives (Table 1).

Community and academic partnerships

CTDPH supplemented their in-house knowledge and skills by partnering with YSM, leveraging their history of collaborating on grant-funded projects. YSM provided CTDPH with additional capacity to manage and execute the CoRECT trial. MDPH and PDPH relied on and benefitted substantially from prior experience and partnership with the participating clinics to manage and execute the intervention. Many of their clinic partners had experience in participating in HIV-related research, both clinical and interventional. Additionally, PDPH partnered with the University of Pennsylvania CFAR to serve as the community advisory board for the CoRECT project.

Preparation phase

The trial sites collaborated closely with participating clinical partners to tailor the patient identification protocols (Figure 1) and provide relevant staff trainings (Supplementary Table 2).

Interorganizational networks

All the trial sites highlighted the importance of establishing a strong buy-in internally among DPH staff and administration as well as the staff and administration of participating clinics with which the CoRECT teams would be working closely. Cultivating relationships with the clinics consisted of explaining the project’s importance, the benefits to patients and clinical practice, and clear expectations (submitting patient lists, discussion of individual cases at conferences, and expedited appointments for randomized patients). Seeking buy-in of DPH staff differed based on their roles within the CoRECT team and other duties. Trial sites and their collaborating partners also established data sharing agreements.

Individual adopter characteristics

Both MDPH and PDPH relied on internal staff to coordinate project activities. This approach increased the sense of DPH “project ownership” which improved staff buy-in. For CTDPH/YSM, YSM staff served as project coordinator. Use of an external (third-party) coordinator in which clinic partners and CoRECT-specific DIS staff did not interact directly with CTDPH staff impacted staff “buy-in.” All three sites identified having a dedicated project contact within each collaborating clinic as critical for streamlining communication and improving implementation of patient follow-up activities.

PDPH differed from the other sites in that they conducted a pilot study to assess procedures and collect feedback from clinic leadership and staff prior to the implementation of the full trial. Based on the findings from the pilot, PDPH CoRECT team modified the initial patient identification protocol to the one shown in Figure 1 and determined the best format for case conferences.

Participant recruitment

The first stage of patient recruitment was case identification which had two components: data sharing/reconciliation and case conferences. Each trial site had a general protocol which could be adapted during the implementation phase. Figure 1 illustrates how the trial sites adapted a general protocol to meet their needs and those of their clinical partners. Case conferences were the principal forum for health department and clinic staff to discuss and review patient eligibility for the trial. They could be conducted in-person or by phone and occurred monthly or quarterly. Table 1 provides a detailed comparison of conference characteristics across the three trial sites.

Through their pilot study, PDPH determined that it was most productive to have their clinic partners submit “in-care” or “all patients” lists and to have structured in-person case conferences led by the PDPH project coordinator.

Staffing

The trial sites used this phase to onboard and train staff. Table 1 provides details regarding the qualifications of field staff. Non-field project staff were trained to perform case identification through surveillance and clinic data exchange and reconciliation, participate or lead case conferences, and randomization of cases. At CTDPH/YSM, the CT DPH HIV epidemiologist and the data manager located at YSM received these trainings. At PDPH, they trained the D2C project coordinator, the D2C managers, and D2C epidemiologists. In contrast, MDPH trained their D2C epidemiologist to manage all these tasks with some assistance provided by other HIV/STD surveillance epidemiologists.

Innovation characteristics

The ability of the CoRECT sites to adapt/modify general case identification protocols and patient follow-up activities were key innovative characteristics. DPH and YSM CoRECT teams worked closely with clinic partners to develop novel data sharing and case review protocols that were feasible for all parties and allowed for the timely identification of OOC cases.

Implementation phase

The implementation phase of active recruitment highlighted the need to address unforeseen challenges during the RCT or for later consideration in the sustainment phase (Table 2).

TABLE 2

Table 2. Implementation phase CoRECT project.

Interorganizational networks

Throughout the RCT, CoRECT site teams continued to work closely with partnering clinics to maintain buy-in and address any challenges or concerns such as staffing changes within the clinic or CoRECT team.

Participant recruitment

The sites and the participating clinics used case conferences to review and discuss the eligibility of each potential OOC case. These conferences generally entailed DPH CoRECT team members (e.g., data managers, project coordinators, or epidemiologists) meeting with individual clinic points of contact, typically case managers, to discuss every patient identified by the reconciliation process to be OOC with the goal of finalizing a list of eligible patients for randomization. Clinicians, social workers, and nursing staff were also invited to participate to provide additional case-specific information. CTDPH/YSM had multiple participating clinics which necessitated the adoption of quarterly case conferences. MDPH allowed clinics to submit either “all patients” or OOC lists via three methods based on their capacity and individual preference. PDPH continued with the procedures that they developed during their pilot study (see Table 1; Figure 1).

As for the case conferences, PDPH developed a structured procedure led by the D2C project coordinator, primarily in-person. However, when the list of cases to review were less than five, meetings could be held over the phone. MDPH developed a semi-structured procedure led by the MDPH D2C epidemiologist, while CTDPH/YSM took a more unstructured approach led by the YSM CoRECT data manager (Table 1). All three sites experienced instances that required a second case conference to complete the review of potential OOC cases due to high OOC caseloads and the time-intensive conferencing process.

Fidelity monitoring/support

During the RCT, each site held monthly internal meetings and periodic project status meetings with clinical partners to provide support and discuss concerns and challenges. In retrospect, CTDPH/YSM did not hold regular monitoring meetings with local health departments that employed the DIS, which eventually posed a limitation. MDPH conducted routine reviews of field staff activities associated with HIV care re-engagement activities and held monthly meetings with field staff to discuss cases and new barriers/challenges.

Staffing

A common challenge faced was staff turnover throughout the course of the project among CoRECT teams, field staff, and partnering clinics which affected the productivity and efficiency of case conferences and patient follow-up. New team members or field staff required updated training to familiarize themselves with the project protocols and processes as well as to establish and build strong working relationships with the multiple different collaborators.

Innovation characteristics

CoRECT’s use of both surveillance and clinic data to identify truly OOC patients required ongoing flexibility to adjust procedures consistent with clinic capacity and operational features related to case identification and DIS field activities.

Sustainment phase

Sites encountered challenges such as changes in leadership at health departments and participating clinics, health department reorganization, and funding after the trial ended, which impacted determining what components of the CoRECT study activities that could be sustained (Table 3).

TABLE 3

Table 3. Sustainment phase CoRECT project.

Leadership

While all sites endorsed general principles of D2C, namely, continuing to actively advocate for HIV re-engagement activities using combined public health and clinical data, leadership challenges were notable. CTDPH faced the dual challenges of external study leadership and organizational restructuring with the hiring of new leaders. PDPH expanded its leadership by transitioning their D2C project coordinator to the head of the HIV specific Field Services Program. In contrast, there were no major changes in MDPH’s leadership or structure.

Organizational characteristics

PDPH maintained similar case identification and patient follow-up activities and retained their dedicated field staff through the reallocation of resources and the development of the HIV-specific Field Services Program, which took the lead in managing the field staff responsible for OOC follow-up activities. Additionally, personnel responsible for data analysis and management were grouped together in the HIV Surveillance Unit. The creation of this program has allowed for the expansion of D2C activities based on CoRECT to all Ryan-White funded facilities. In MDPH, the integration of CoRECT activities into the routine workflow of HIV/STI field epidemiologists and surveillance epidemiologists allowed for the continuation of patient follow-up activities, although in a more limited capacity. In contrast, without YSM to provide project management support and resources to support dedicated field staff, CTDPH had to halt their CoRECT follow-up activities. CTDPH has since hired a D2C coordinator and continues to re-visit their approach.

Participant recruitment

All three trial sites highlighted the need to include procedures to update OOC definitions based on application of evolving epidemiology and updated clinical guidelines and practices. Since CoRECT concluded, there were also the changes in health care delivery and access related to the COVID-19 pandemic that will certainly have impacts on how we ascertain whether someone is truly OOC and might need assistance.

Staffing

Staffing significantly impacts program sustainability, through turnover, maintenance of institutional knowledge and relationships, leadership, or existing organizational structure. PDPH and MDPH retained/sustained the field, surveillance, and D2C data management positions involved in CoRECT. Furthermore, PDPH was able to expand the number of dedicated D2C staff through the Field Services Program with new funding from the state of Pennsylvania. CTDPH were unable to retain their CoRECT-specific field staff and YSM project management staff; however, as part of an ongoing reorganization plan, they have included new internal positions such as a Data to Care Coordinator.

Community and academic partnerships

Even with the expertise and other resources offered by an external academic partner for project management, such as with CTDPH, long-term sustainability due to sub-optimal infrastructure development is challenging. However, one advantage of having an academic partner is the ability to create post-hoc analyses and publish in peer-reviewed journals (23). Outside of academic partners, other community relationships were also important in the sustainment of this work. MDPH and PDPH recognized that consulting with collaborating clinics, as well as advisory committees, including consumers, could provide beneficial input and support for future D2C interventions. PDPH continued their partnership with the University of Pennsylvania CFAR until the start of the COVID-19 pandemic.

Discussion

To our knowledge, CoRECT was the first prospective RCT to implement and assess the use of a D2C strategy accompanied by an intervention component for re-engaging newly OOC PWH (10, 11, 23). While the study demonstrated successful initial re-engagement in care for OOC persons compared to the SOC and the Philadelphia site also demonstrated successful retention in care at 12 months, the specific implementation components have not been fully analyzed (10). We used the EPIS framework to operationalize a systematic approach to detail the implementation processes of the three CoRECT sites to highlight the differences in jurisdictional approaches to data sharing, case reconciliation, patient follow-up activities, and sustainability. The consolidation of this information identified useful lessons learned for future D2C activities as well as dissemination to other jurisdictions.

Utilization of the EPIS framework retrospectively was well-suited for this assessment given its emphasis on implementation, as well as initial planning and long-term sustainability. EPIS allowed us to explore pre-defined contextual factors across both jurisdiction and stage and enabled a more nuanced understanding of the complex interplay between the intervention, the stakeholders involved, and larger socio-ecological considerations. This provided valuable insights into challenges and facilitators as well as future D2C delivery and sustainability.

Each trial site approached CoRECT grounded in their own context and informed by a multitude of factors including prior D2C experience, data management infrastructure, staffing capacity and roles relative to HIV patient recruitment navigation, and relationships with healthcare clinic partners. While CoRECT was a research study funded by the CDC, retrospective analysis using the EPIS framework, including the preparation and implementation components, highlighted the critical role of “ownership” of the D2C initiative by the health departments from the standpoint of leadership and organizational infrastructure. The approaches taken by PDPH and MDPH emphasized internal investment by using DPH HIV surveillance staff, building new processes to support OOC re-engagement, and incorporating OOC re-engagement activities of field staff into the existing health department organizational structure and processes. Although PDPH had prior D2C experience with a data exchange process, the pilot study proved to be extremely helpful in soliciting feedback from their CoRECT team and clinical collaborators to craft the full-scale field intervention and fine-tune their D2C process. This initial prioritization in the exploratory and preparation phases facilitated the development of a more sustainable intervention, which was further supported by the creation of an intraorganizational HIV-specific Field Services Program and leveraging other funding sources. MDPH incorporated lessons learned from prior D2C experience and through experience with routine HIV patient navigation to leverage existing infrastructure for data sharing and incorporated case assignments into existing field staff’s workflow. This sustainability focused approach allowed MDPH to continue some aspects of the CoRECT intervention after conclusion of the CoRECT study.

In contrast, CTDPH lacked prior D2C experience but prior research efforts and a commitment to principles inherent in D2C led them to partner with an academic institution, YSM. Newly created processes and infrastructure including hiring of dedicated CoRECT field staff through local health departments aided in the successful implementation of CoRECT, however, they negatively impacted sustainability because the YSM and field staff were outside of the CTDPH structure. Fortunately, the CTDPH was able to identify ways to address future D2C activities including the use of pilot testing, centralizing DIS activities within the CTDPH, designating a D2C coordinator, and including field and healthcare clinic staff in discussion regarding long-term feasibility.

A fundamental principle that emerged was adaptability, specifically the importance of being able to modify the procedures because there is no one-size-fits-all approach to designing/implementing a D2C project given the variability in target populations, health department characteristics/structure, and participating clinic characteristics. This was particularly critical given the changing clinical standards of care, evolving public health approaches such as integrated data systems, and updated goals of the National HIV/AIDS Strategy and Ending the HIV Epidemic initiatives (1). Adaptability was also indispensable given the need for flexibility and need for local tailoring among key participants at national and local levels. The successful strengthening of collaboration between health departments and healthcare clinics, as an explicit goal of CoRECT, required negotiation and compromise in building interorganizational networks.

Challenges identified during this assessment, including those related to leadership, staffing and turnover; evolving clinical practices regarding OOC definitions; sustainability of community and academic partnerships, fidelity monitoring and data management discrepancies, and inconsistent buy-in, underscore the importance of evaluation and integration steps throughout the process. While many of these challenges were addressed through prioritization of establishing standardized protocols, streamlined communication, consistent training practices, and clear expectations, flexible strategies that can accommodate changes within public health and clinical settings remain paramount for continued D2C evolution as it is refined in new contexts.

This retrospective assessment not only provides an overview of each trial site’s approach to the CoRECT intervention and the implementation process, but also provides some key take-aways, which may prove useful for those interested in developing and implementing their own D2C activities. We would like to highlight the following key principles as a guidance for health departments considering implementing or enhancing similar activities.

• Tailor D2C structures and activities to the jurisdictional needs based on their unique characteristics in order to be effective across varying contexts.

• Develop appropriate and sufficient technical and staffing infrastructure to enable the use of multiple different sources of data to supplement surveillance data (e.g., patient clinical records, Medicaid claims, medical case management data, or Ryan White program data) even as integrated data platforms continue to be explored.

• Appoint a health department “champion” for D2C who will serve as the internal point person for health department staff and clinic personnel. Ideally, the “champion” will have extensive HIV experience and existing relationships with the clinics, including knowledge of their patient population and data.

• Create follow-up and outreach activities by health department personnel that are feasible and sustainable within the jurisdiction’s capacity, organization, and infrastructure.

• Include stakeholders representing core functions/components (e.g., Health Department Medical Directors, Health Department HIV Directors, disease surveillance staff, field staff, program managers or administrators, clinic administrators, continuous quality improvement staff, patient navigators, and clinic data management) in the early stages of the intervention design process. This could entail the implementation of a pilot study to allow for feedback to address challenges prior to initiating the full intervention.

• Create ongoing and regular opportunities, such as project status meetings, for health department and clinic staff to provide feedback regarding barriers/challenges that have been identified. These meetings can also serve to maintain good intra- and interorganizational communications and serve as learning collaboratives.

• Build flexibility into all components of the activities, including, but not limited to the case reconciliation process (e.g., case conference location and methodologies for providing patient lists), OOC definitions, and have the flexibility to align/respond to organizational capacity, infrastructure, and operational efficiencies.

• Develop and incorporate the administration of D2C data sharing/management and follow-up activities within the existing health department organizational structure. This may include establishing and supporting cross department/division engagement (e.g., HIV surveillance staff working closely with STI field staff or IT department).

• Use a structured format for case conferences to review cases with clinic staff in a timely and efficient manner.

• Develop and document written procedures and fidelity processes early in the implementation process to ensure consistent implementation and accommodate for changes in health department and clinic staff and organization. Apply standardized procedures that can be periodically revisited to ensure they remain maximally responsive to environmental and contextual factors, so interruption in activities can be kept to a minimum.

• Monitor changes in guideline implementation, clinical practices, and standards of care to make timely updates to D2C OOC case identification algorithms and follow-up activities as necessary.

• Organize regular training and educational opportunities to sustain the D2C activities for health department and clinic staff.

Limitations

A limitation of this evaluation was the inability to obtain comprehensive feedback on the implementation process from field and clinic staff due to staff turnover. Future evaluation projects could address this by collecting feedback prospectively during the four EPIS framework stages. There were several strengths including a comprehensive appraisal of comparing the implementation processes across three CoRECT sites in a structured format and generate lessons learned for future D2C activities. By combining all three sites into a single evaluation, lessons learned are more robust, nuanced, and useful for future implementation.

Conclusion

In conclusion, we found that while the CoRECT study framework successfully provided a general blueprint for health departments to implement a D2C intervention for improving re-engagement in HIV care, it is important for jurisdictions to customize it based on their own needs and evolving practices. The lessons learned in the exploratory, preparation, implementation, and sustainment phases of implementation by CTDPH/YSM, PDPH, and MDPH provide initial starting points and critical aspects that are critical for leadership to consider when implementing D2C.

Data availability statement

The original contributions presented in the study are included in the article/Supplementary material, further inquiries can be directed to the corresponding author.

Author contributions

HE led the review and the writing processes. SGL assisted with major draft revisions and provided site specific information. BJ, KR, BG, and MB provided site specific information and assisted in the editing process. MV, KAB, AD, and LR contributed to the concept, site specific information. FLA provided guidance in the use of implementation frameworks. All authors contributed to the article and approved the submitted version.

Funding

This study was funded by the US Centers for Disease Control and Prevention (ClinicalTrials.gov, NCT02693145). The three grantees were the Philadelphia Department of Public Health, Massachusetts Department of Public Health, and the Connecticut Department of Public Health.

Acknowledgments

We acknowledge the contributions of the staff of public health disease intervention specialists, field epidemiologists, and their supervisors, and data personnel at the MDPH, CTDPH, and PDPH including Monica Morrison, Sophie Lewis, Lisa Nichols, and Heidi Jenkins. We also wish to acknowledge the participating clinics: Cornell Scott Hill Health Center, Fair Haven Community Health Center, Veterans Administration Medical Center in West Haven, Community Health and Wellness Center of Greater Torrington, Inc., Staywell Health Center, Waterbury Hospital, Danbury Hospital, Circle Care Center, Optimus Community Health Center, Southwest Community Health Center, Stamford Hospital, Bridgeport Hospital Primary Care Clinic, Internal Medicine and ID Associates, Norwalk Community Health Center, Burgdorf-Gengras-St. Francis, Community Health Center, Inc., Community Health Services, Hospital of Central Connecticut, CT Health Care Group, LLC, Hartford Hospital/Brownstone, University of Connecticut, Yale-New Haven Hospital Haelen Center, Yale-New Haven Hospital Nathan Smith Clinic, Baystate Medical Center, Beth Israel Deaconess Medical Center, Boston Medical Center, Brigham and Womens Hospital, Cape Cod Healthcare, Fenway Health, Greater Lawrence Community Health Center, Lemuel Shattuck Hospital, Massachusetts General Hospital, Drexel Partnership, Penn Community Infectious Disease, Temple Comprehensive Care Practice, Public Health Management Corporation Care Clinic, Philadelphia FIGHT, Philadelphia VA Medical Center, Mazzoni Center, and Infectious Disease Associates.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpubh.2023.1223149/full#supplementary-material

References

1. White House Office of National AIDS Policy. National HIV/AIDS strategy for the United States: Updated to 2020. Washington, DC. (2015). Available from:https://files.hiv.gov/s3fs-public/nhas-update.pdf

Google Scholar

2. The White House. National HIV/AIDS strategy for the United States 2022–2025. (2021). Washington, DC. Available from: https://www.whitehouse.gov/wp-content/uploads/2021/11/National-HIV-AIDS-Strategy.pdf

Google Scholar

3. Centers for Disease Control and Prevention. Understanding the HIV care continuum. (2019). Available from:https://www.cdc.gov/hiv/pdf/library/factsheets/cdc-hiv-care-continuum.pdf

Google Scholar

4. Centers for Disease Control and Prevention. Data to care program guidance: using HIV surveillance data to support the HIV care continuum. (2017). Available from:https://www.cdc.gov/hiv/pdf/funding/announcements/ps18-1802/CDC-HIV-PS18-1802-AttachmentJ-Data-to-Care-Program-Guidance.pdf

Google Scholar

5. Centers for Disease Control and Prevention. Monitoring selected national HIV prevention and care objectives by using HIV surveillance data—United States and 6 dependent areas, 2018. (2020). Available from:https://www.cdc.gov/hiv/pdf/library/reports/surveillance/cdc-hiv-surveillance-supplemental-report-vol-25-2.pdf

Google Scholar

6. Doshi, RK, Milberg, J, Isenberg, D, Matthews, T, Malitz, F, Matosky, M, et al. High rates of retention and viral suppression in the US HIV safety net system: HIV care continuum in the Ryan white HIV/AIDS program, 2011. Clin Infect Dis. (2015) 60:117–25. doi: 10.1093/cid/ciu722

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Health Resources and Services Administration. Ryan white HIV/AIDS program annual client-level data report 2018. (2019). Available from:https://ryanwhite.hrsa.gov/sites/default/files/ryanwhite/data/rwhap-annual-client-level-data-report-2018.pdf

Google Scholar

8. Rebeiro, PF, Horberg, MA, Gange, SJ, Gebo, KA, Yehia, BR, Brooks, JT, et al. Strong agreement of nationally recommended retention measures from the Institute of Medicine and Department of health and human services. PloS One. (2014) 9:e111772. doi: 10.1371/journal.pone.0111772

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Sweeney, P, DiNenno, EA, Flores, SA, Dooley, S, Shouse, RL, Muckleroy, S, et al. HIV data to care—using public health data to improve HIV care and prevention. JAIDS J Acquired Immune Deficiency Syndromes. (2019) 82:S1–5. doi: 10.1097/QAI.0000000000002059

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Neblett Fanfair, R, Khalil, G, Williams, T, Brady, K, DeMaria, A, Villanueva, M, et al. The cooperative re-engagement controlled trial (CoRECT): A randomised trial to assess a collaborative data to care model to improve HIV care continuum outcomes. Lancet Regional Health - Americas. (2021) 3:100057. doi: 10.1016/j.lana.2021.100057

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Neblett Fanfair, R, Shrestha, RK, Randall, L, Lucas, C, Nichols, L, Camp, NM, et al. Implementing data to care—what are the costs for the health department? JAIDS J Acquired Immune Deficiency Syndromes. (2019) 82:S57–61. doi: 10.1097/QAI.0000000000001968

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Centers for Disease Control and Prevention. HIV Surveillance Report, 2017. (2018). p. 129. Available from: http://www.cdc.gov/hiv/library/reports/hiv-surveillance.html.

Google Scholar

13. Philadelphia Department of Public Health. AIDS activities coordinating office surveillance report, 2017. Philadelphia, PA: City of Philadelphia (2018).

Google Scholar

14. Moullin, JC, Dickson, KS, Stadnick, NA, Rabin, B, and Aarons, GA. Systematic review of the exploration, preparation, implementation, sustainment (EPIS) framework. Implementation Sci. (2019) 14:1. doi: 10.1186/s13012-018-0842-6

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Powell, BJ, Proctor, EK, and Glass, JE. A Systematic Review of Strategies for Implementing Empirically Supported Mental Health Interventions. Research on Social Work Practice. (2014) 22:192–212. doi: 10.1177/1049731513505778

CrossRef Full Text | Google Scholar

16. Aarons, GA, Hurlburt, M, and Horwitz, SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm Policy Ment Health. (2011) 38:4–23. doi: 10.1007/s10488-010-0327-7

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Aarons, GA, Ehrhart, MG, Farahnak, LR, and Sklar, M. Aligning leadership across systems and organizations to develop a strategic climate for evidence-based practice implementation. Annu Rev Public Health. (2014) 35:255–74. doi: 10.1146/annurev-publhealth-032013-182447

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Hague, JC, John, B, Goldman, L, Nagavedu, K, Lewis, S, Hawrusik, R, et al. Using HIV surveillance laboratory data to identify out-of-care patients. AIDS Behav. (2019) 23:78–82. doi: 10.1007/s10461-017-1742-5

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Massachusetts Department of Public Health Bureau of infectious disease and laboratory sciences office of HIV/AIDS. The SPECTRuM Project Implementation Manual (2016). Available from:https://www.mass.gov/doc/spectrum-implementation-manual-strategies-1-and-2-1/download

Google Scholar

20. Kunzweiler, C, Kishore, N, John, B, Roosevelt, K, Lewis, S, Klevens, RM, et al. Using HIV surveillance and clinic data to optimize data to care efforts in community health centers in Massachusetts: the Massachusetts partnerships for care project. JAIDS J Acquired Immune Deficiency Syndromes. (2019) 82:S33–41. doi: 10.1097/QAI.0000000000002019

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Centers for Disease Control and Prevention. ARTAS. (2022). Available at:https://www.cdc.gov/hiv/effective-interventions/treat/artas/index.html [Accessed April 26, 2022].

Google Scholar

22. Lee, AI, Engle, NM, Corcoran, CL, Little, S, Rapp, R, Lane, T, et al. (2016) Anti-retroviral treatment and access to services (ARTAS) an individual-level, multi-session intervention for people who are recently diagnosed with HIV. Academy for Educational Development Center on AIDS & Community Health (COACH) Available from: https://aidsetc.org/sites/default/files/resources_files/artas_implementation_manual.pdf

Google Scholar

23. Villanueva, M, Miceli, J, Speers, S, Nichols, L, Carroll, C, Jenkins, H, et al. Advancing data to care strategies for persons with HIV using an innovative reconciliation process. PloS One. (2022) 17:e0267903. doi: 10.1371/journal.pone.0267903

CrossRef Full Text | Google Scholar