External relationships as implementation determinants in community-engaged, equity-focused COVID-19 vaccination events

Background: While relationships and connectedness among organizations have been included in implementation theories, models, and frameworks, the increased attention to health equity in implementation science raises the urgency of understanding the role of relationships external to the implementing organization. This paper addresses this gap through an exploration of the role of external relationships in community-based, equity-focused interventions.

Methods: This study focuses on an equity-focused, community-based COVID-19 vaccination intervention in Arkansas, drawing upon long-term community-engaged relationships among University of Arkansas for Medical Sciences and the Hispanic and Marshallese Islander communities. We used an exploratory qualitative descriptive design to examine barriers and facilitators to implementation of COVID-19 vaccination events analyzing in-depth qualitative interviews with implementation team members (n = 17).

Results: All participants described pre-existing relationships among the implementing organization, partner organizations, and communities as a key implementation determinant for this equity-focused program. At the inter-organizational level, external relationships included formal connections and informal relationships among staff (e.g., communication channels from prior partnerships). At the individual level, strong external relationships with the community were facilitators leveraging long-term engagement, community familiarity, and staff from the communities of focus. Strong external relationships facilitated program reach in underserved communities through three mechanisms: (1) reduced time required to establish functional working relationships among partners; (2) accessibility and cultural congruence of health services; and (3) increased trust among community members. Barriers to implementation also existed in external relationships, but had less influence than facilitators.

Conclusions: Achieving health equity in implementation science requires greater understanding of external relationships as implementation determinants. This exploratory study makes a significant contribution to the literature by describing the types of external relationships that facilitate equitable implementation and identifying the mechanisms through which they may work. We argue that approaches to community engagement drawn from community-engaged research approaches may be useful, as these processes require investment in building/maintaining formal and informal organizational and interpersonal relationships. Further research is needed to understand connections among external relationships and other implementation determinants.

Introduction

Relationships and connectedness among organizations within and across the implementation environment has been included in implementation-related theories, models, and frameworks prior to the inception of implementation science (IS) as a field of study in the United States (US) at the turn of the 21st century (1). Communication channels and social systems are two of the four main elements in Roger's Diffusion of Innovation theory (2), a pillar upon which IS rests. Organizational relationships are also reflected in more recent implementation determinant frameworks (3) as “external networks” (4), “interconnections/linkages” (5), and “inter-organizational networks & relationships” (6). Despite the recognition of external relationships as a critical determinant of implementation (7–12), this construct has not been studied with the same depth as many other constructs within IS frameworks (13).

In the original version of the Consolidated Framework for Implementation Research (CFIR), one of the most widely-cited implementation determinants frameworks, “the degree to which an organization is networked with other external organizations” is identified as a critical implementation factor (14). In the revised CFIR “2.0” (4), the “partnerships and connections” construct broadly captures relationships with external organizations. This construct is situated within CFIR's Outer Setting domain, which captures “macro-level” implementation factors emanating from outside the Inner Setting, or the site where implementation is occurring. Perhaps as a result of the highly interventional nature of implementation research in healthcare, more attention has been paid to the Inner Setting and specifically to constructs and determinants within this category (e.g., available resources, infrastructure, incentive systems) which may be modifiable (or leveraged) within projects designed to improve implementation of a specific intervention or practice within specific healthcare organizations/locations (3). Other implementation frameworks have provided alternate constructs to focus on macro-level implementation factors, such as “inter-organizational networks” and “community-academic partnerships,” in the Exploration, Preparation, Implementation, Sustainment (EPIS) framework (15) and “inter-organizational networks & relationships” in the integrated-Promoting Action on Research Implementation in Health Services (iPARIHS) framework (6).

The limited attention on relationships among implementing organizations and external partners may be due to a perception that they are a more remote implementation determinant and that they are less amenable to rapid intervention. As such, relationships among the implementing organization and organizations and communities external to the implementing organization are a determinant that remains relatively under-conceptualized, and to-date few scholars have explored this construct in depth (16). Underdevelopment is a noted challenge for all outer setting constructs, which are “notoriously difficult to evaluate and influence” (13). Extant implementation research frameworks have also predominantly focused on relationships among similar or peer organizations (e.g., healthcare organizations, social services providers) (5, 14). Most published studies assessing external relationships primarily focus on links among peer organizations, such as formal implementation networks (17), quality improvement collaboratives (18), or organizations providing similar client services (19–21). The updated definition of the “partnerships and connections” construct in CFIR 2.0 helps expand the scope of external relationships to include collaboratives, professional societies, referral networks, community-academic partnerships, advocacy groups, and technical assistance organizations (4). While it has been noted that relationships with community organizations in different sectors (e.g., churches, non-profits) can benefit implementation (22, 23), and intersectoral relationships are a common approach in public health programs (24–26), their role is understudied in IS.

Community engagement and equitable implementation

The increased attention to health equity in IS raises the urgency of understanding external relationships (organizational and/or among individuals) as determinants of implementation (27, 28). It is well-recognized that communities with the highest burden of health disparities are often unreached, or the last to be reached by evidence-based interventions (29–31). The determinants of healthcare organizations' ability to reach disproportionately-impacted communities is understudied in IS (32). A key recommendation for advancing health equity in IS is to engage equity partners in sectors outside of health systems [e.g., employers, housing, school, and faith-based organizations (FBO)] (27). Yet, little is currently understood about the extent to which healthcare organizations are able to engage external equity partners in the implementation of interventions, how best to engage partner organizations, and how these external relationships might improve equity of implementation and outcomes. Thus, when and how healthcare organizations engage underserved communities and the degree of the connectedness among these organizations and communities may emerge as a critical determinant of equitable outcomes.

Relationships between implementing organizations and community organizations are not an explicit component of new, equity-oriented implementation research frameworks (33, 34). While activities to engage communities can be considered as an equity-focused implementation strategy, the nature and strength of external relationships with community groups will likely determine the success of this strategy (35, 36), although recent work by Wallerstein and colleagues highlights that the science has lagged behind practice (37). IS research can draw upon the rich literature on Community-Engaged Research (CEnR) to facilitate our understanding of the role for relationships among community-based equity partners and implementing organizations (4–8).

Trust is another underdeveloped concept in the IS literature but is recognized as critical to building implementation partnerships and to recipients' participation (38). Conceptualizations of trust in CEnR have highlighted how relational dynamics contribute to trust-building (39), and establishing partnerships with underserved communities builds trust in healthcare organizations (40). This is also critical for equity in vaccine uptake, as community-engagement has frequently been identified as a critical factor in effectively promoting vaccine uptake and building trust in public health authorities and interventions, especially among marginalized and underserved communities (41–45). Therefore, it is necessary to understand whether, and how, external relationships improve the equity of implementation, relationship strength, and trust.

This paper addresses the gap in research on external relationships in implementation exploring the role of these relationships in implementing an equity-focused COVID-19 vaccination program. We utilized an exploratory qualitative descriptive study design to understand the barriers and facilitators to implementation of COVID-19 vaccination events within FBOs as a way to reach Hispanic and Marshallese Islander (hereafter Marshallese) community members.

Methods

Setting and intervention

The focus of this paper is the implementation of a community-based COVID-19 vaccination program in Arkansas. This program drew upon long-term community-engaged relationships between University of Arkansas for Medical Sciences (UAMS) and the Hispanic and Marshallese communities in the region (46, 47). These relationships originated in 2013 to address social determinants of health and associated chronic disease disparities among the Hispanic and Marshallese communities in Arkansas (41, 46). Since its inception, these relationships have utilized a community-engaged approach, which seeks to build trust among academic researchers, healthcare providers, and communities through direct engagement, honoring those communities' unique contributions at all stages of health interventions. Further details are published elsewhere (46, 48).

As an extension of these formal and informal pre-existing, community-engaged relationships, a COVID-19 response taskforce was developed and led by community-based organizations, and the taskforce met weekly between March 2020 (within one week of the first identified case of COVID-19 in Arkansas) and continued to meet through August of 2022, with daily communication among organizations to address the COVID-19 health disparities among the Hispanic and Marshallese populations in Northwest Arkansas (the details of which have been previously published) (41). The taskforce developed a comprehensive COVID-19 response involving education, outreach, testing, contact tracing, and support for quarantining (46). Vaccination outreach was included as COVID-19 vaccines became available in December 2020. Leveraging these relationships, academic researchers and healthcare organizations implemented COVID-19 vaccination events in community settings, primarily in partnership with FBOs, with a goal of improving reach, increasing attendee comfort, and providing native-language facilitation and education.

To support vaccination outreach programs, the academic medical center, UAMS, received funding from the National Institutes of Health-funded Community Engagement Alliance Against COVID-19 Disparities (CEAL); Racial and Ethnic Approaches to Community Health (REACH), administered by the Centers for Disease Control and Prevention; and the Health Resources and Services Administration (HRSA) of the United States Department of Health and Human Services. To maximize accessibility of the events, most were held at local FBOs with Hispanic and/or Marshallese congregations on days and times chosen to facilitate attendance and reduce barriers (described in previous publications) (49). Community health workers affiliated with FBOs and/or UAMS promoted attendance by scheduling appointments and providing resources such as transportation to attendees. Events were staffed by members of the implementing and partner organizations. The implementation team included healthcare providers, program staff [many of whom were community health workers (CHWs)], and staff of FBOs. All vaccines were administered by clinical staff, and all events included bilingual (English/Spanish or English/Marshallese) team members who provided medical translation.

Data collection

Our exploratory qualitative descriptive study examined the barriers and facilitators to implementation of COVID-19 vaccination events within FBOs as a way to reach Hispanic and Marshallese community members. Data was collected at vaccination events held between July 2021 and September 2021. For transparency, our diverse research team and co-authors' self-identified positionalities include five men, six women, six identifying their race/ethnicity as non-Hispanic White, three identifying as Marshallese and Pacific Islanders, one identifying as Hispanic, and one identifying as mixed-race and ethnicity. Three qualitative researchers (GC, JV, and SP) conducted five observations of vaccination events held in Hispanic FBOs (n = 2), Marshallese FBOs (n = 2), and one (n = 1) church-affiliated community space. The three qualitative researchers also conducted informal interviews during events (n = 55) and invited team members at vaccination events to participate in a semi-structured interview at a later date. Informal interviews consisted of short, unstructured conversations with team members concerning their experiences with vaccination events.

Following a purposive sampling approach (50), the study team recruited 17 participants reflecting diverse roles in the implementation of vaccination events, which follows standard qualitative approaches to determining sample sizes based on the scope and nature of the study (51, 52). Inclusion criteria for participation consisted of adults (≥18 years of age) who were members of the implementation team. Formal, semi-structured qualitative interviews were conducted with participants via secure video conferencing in the fall of 2021. All interviews were conducted in English, transcribed verbatim, and de-identified before analysis. Verbal consent was obtained prior to interviewing and recorded in REDCap, along with demographic information (53, 54). Most interviews lasted between 30 and 60 min, and participants were provided a $50 incentive.

We used a semi-structured interview guide combining grand tour, open-ended questions, probe questions based on a priori CFIR categories, and topics emerging from informal interviews and observations at vaccination events to maintain consistency across formal interviews. The CFIR framework was chosen due to its comprehensive focus on implementation determinants and its frequent utilization within the IS literature. Examples of grand tour questions include, “What do you think worked well at the event(s)?”, “What were some barriers or challenges to delivering the COVID-19 vaccine in a non-clinical setting?”, and “What do you think could have made the event(s) more successful?” Based on the responses to the grand tour questions, additional probe questions were used based on CFIR categories and specific determinants. In addition, each participant was asked to discuss the extent to which the event(s) achieved the goal of reaching the communities of interest. All study materials and procedures were approved by the UAMS Institutional Review Board (IRB#262917).

Data analysis

The co-authors conducted rapid thematic analysis following a modified framework approach (55, 56), utilizing CFIR as the a priori coding framework. Themes from each interview transcript were independently summarized by co-authors (GC, SP, JCK, and RM) using a structured coding template. The research team met regularly to consolidate the templates into one final coded template per interview, resolve discrepancies in interpretation, and assign identified barriers and facilitators to CFIR constructs. Barriers and facilitators were added to the operational definitions of constructs in the study-specific CFIR codebook. Summaries of coded data were transferred to charts with a column for each CFIR construct and a row for each participant to facilitate identification of patterns and outliers. The research team reached thematic saturation, e.g., the point at which patterns in the data were clearly identified through analysis and no new themes were identified, after analysis of 10 transcripts. Illustrative quotes were identified for each theme using a consensus approach.

Results

Seventeen participants completed qualitative interviews (Table 1). Participants were healthcare providers (n = 4), program staff (many of whom were CHWs) (n = 10), and FBO staff (n = 3). The median age of participants was 41 years, and 65% of participants were women. Participants were racially/ethnically diverse; eight participants identified their race/ethnicity as White (47%), four identified as Hispanic (24%), four identified as Marshallese (24%), and one identified as Asian (6%).

Table 1

Table 1. Demographics of participants (n = 17).

During analysis, we identified emergent themes within the a priori CFIR 1.0 category of “cosmopolitanism.” Participants described both formal and informal organizational-level relationships among UAMS and team members, government agencies {e.g., Arkansas Department of Health, the Consulate of the Republic of the Marshall Islands, healthcare organizations [e.g., Federally Qualified Health Centers (FQHCs), hospital systems]}, and community organizations and FBOs. We identified themes at the organizational and individual levels for external relationships as implementation determinants (see Table 2). At the organizational level, formal inter-organizational structures within the implementation environment and informal relationships among organizations emerged as implementation determinants. At the individual level, we identified themes of cultural congruence arising from the overlapping staff roles of the implementing organization with communities of focus and community members' familiarity with organization as determinants of community-based COVID-19 vaccination events. Formal inter-organizational structures included contractual agreements between organizations. Informal relationships among the implementing organization and partner organizations were developed over time through prior community-engaged collaboration (41, 47, 48). In addition, participants identified individual-level relationships, highlighting cultural congruence among community team members with dual roles in the implementing organization and as trusted members of the Hispanic or Marshallese communities. Cultural congruence, e.g., inclusion of community members as implementation team members, also facilitated community members' familiarity with the organization. The nature of external relationships, and their role as barriers and facilitators of community-based COVID-19 vaccination events, are described below highlighting the salient formal and informal relationships. Figure 1 presents an overview of the salient relationships within the implementation context.

Table 2

Table 2. Emergent themes by level of relationship.

Figure 1

Figure 1. External relationships in the implementation context.

Organizational level: formal inter-organizational structures

The central inter-organizational structure for the intervention was a COVID-19 taskforce, described above. The community-led taskforce was organized as an extension of the community-engaged relationships among UAMS and team members, government agencies (e.g., Arkansas Department of Health), healthcare organizations (e.g., FQHC, hospital systems), community organizations and FBOs, and the Hispanic and Marshallese communities. The taskforce met monthly with an agenda focused on addressing COVID-19 disparities in the Hispanic and Marshallese communities. The taskforce was predicated on long-standing community-based relationships. Most taskforce members had established memoranda of understanding (MOU) or other formal agreements, such as data use agreements.

These formal relationships were identified by participants as facilitating implementation of the intervention in three key ways. First, the established weekly meetings of the formal partner organizations facilitated communication, serving as a forum for coordinating activities and providing input on intervention design. Second, pre-existing formal relationships (e.g., pre-existing MOU and data use agreements) among partner organizations enabled the quick start-up times for developing and deploying the vaccine events. One participant explained, “We keep getting in these projects, but it's not like we’re having to shift gears or do any 180s to do that work… these projects just allow us to continue, I guess, the work that we’ve already been doing, and enhance on it” (program staff #1, White). Third, the long-standing formal collaborations contributed to the trust among the partners in external networks. Explained by one participant, “We’ve worked with [community-based partner organizations] a lot in the past. They’ve always been a good partner, so it was kind of natural for us to go with them” (program staff #2, White).

Formal relationships were not uniformly discussed as facilitators, with some participants identifying barriers, such as differences in organizational policies which created operational challenges during events. For example, the categories of healthcare providers qualified to provide vaccinations varied by organization. One participant stated, “The most challenging thing about administering vaccines off-site is the regulation around it, [for UAMS supported events, the policy is] a credentialed provider [an MD, PA, or APRN] has to be on-site. Making sure that we're in compliance with all of the policies […] was probably the most challenging thing” (healthcare provider #2, Asian).

Additionally, the collaborative nature of the events meant variability in sponsors at some events which required changes in paperwork and electronic health records systems from one event to another, along with corresponding staffing and workflow changes. A team member stated, “When we first started partnering with [the FQHC], […] there's two consent forms they have to do, vs. the one when it's UAMS. […] At first, we didn’t have a good flow for when people came in and did the consent forms, and then got their shot and waited” (program staff #2, White). External relationships were also identified as requiring increased time and effort in communication and coordination by one participant: “I think definitely more communication between the partners [is important] so that everybody understands everybody else's roles and so when we start the event and we're there—again, to be flexible and culturally sensitive to who's there, understanding that not everything is gonna work like clockwork” (healthcare provider #1, Marshallese).

Organizational level: informal relationships among organizations

Informal relationships across organizational boundaries were described as a primary facilitator at the organizational level and consisted of personal and professional connections among team members at the implementing organization and partner organizations (e.g., FBOs, government agencies, healthcare organizations). These informal relationships, described as developing over time, fostered collaboration across organizational boundaries during implementation of vaccination events. A leader at an FBO described how a connection between that organization's Executive Director and the Director of Health Outreach at a local hospital facilitated a vaccine clinic event. The participant stated, “We were talking to [Hospital] about that, and what all services they offered already […] and would they be willing to be our community provider” (FBO staff #2, White). That participant concluded, “So, we just facilitated it, and made it happen.”.

Other participants explained how long-standing informal relationships between organizations engendered trust and willingness to engage in implementing vaccination events. One participant noted, “It really helps already having the relationships there because we know that, ‘Oh, hey, I can work with this person from [a partner organization],’ and they’ve been a good partner in the past; we know they’re gonna show up if they say they’re gonna show up” (program staff #2, White). Conversely, there was greater difficulty in engaging organizations in vaccine events when informal relationships did not already exist. Participants noted that when trying to recruit new FBOs to participate in vaccination events, “there have been several churches that said, ‘We don't need ‘em’” (program staff #3, Marshallese).

Individual level: staff roles in organization and community

As a result of long-term engagement with the Hispanic and Marshallese communities in Northwest Arkansas, several team members employed by UAMS are members of the Hispanic and Marshallese communities facilitating cultural congruence between the implementing organization, FBO, and the Hispanic and Marshallese communities. Participants overwhelmingly stated that cultural congruence among the implementing organization and communities of focus made it possible for team members at vaccine events to provide culturally and linguistically appropriate services. For example, one participant described culturally-appropriate messaging delivered by a trustworthy community ambassador as a facilitator for reaching the Marshallese community: “We're speaking with the Marshallese community, in Marshallese, in a way that's culturally appropriate by somebody that they trust” (healthcare provider #1, Marshallese). Team members' work in their communities was also considered a source of intrinsic motivation. One participant stated, “[Implementation team members are] part of the community that we’re protecting; the same people who come and work our events. […] So, they really take ownership […] sometimes they’re like, ‘Hey, this is my church. I’m gonna be there. I’m gonna work it. I’m gonna make sure it's successful’” (program staff #4, White).

Several Hispanic and Marshallese participants noted burnout resulting from their dual roles as UAMS team members and as members of the communities of focus. Participants described how this overlap in social roles became an implementation barrier as events required the involvement of bilingual team members at every stage of the process (check in, registration, consent, vaccination, waiting period, and evaluation activities). Team members with the requisite language skills worked additional and non-standard hours to accommodate vaccination events which occurred on nights and weekends and in varying locations in the region. Furthermore, some Hispanic and Marshallese staff reported serving as an unofficial point-of-contact or source of information for members of their own community outside of their work hours. A participant described, “'Cause I work with UAMS, they assume that I have the answer when they ask me things [about COVID-19 or the vaccine]” (program staff #6, Marshallese). Institutional policies related to flexible hours and paid time off were described by participants as critical to helping mitigate this barrier. One participant explained, “You get—we call it flex time. Say you worked five hours on an event on Sunday. You can then take those five hours off somewhere else in the next two weeks during your regular work time, without having to submit time through the system. That works well, except everybody's very busy, and needs their office time too” (program staff #5, Hispanic).

Individual level: community members' familiarity with organization

Participants frequently described how Hispanic and Marshallese community members in Northwest Arkansas were familiar with UAMS due to the organization's prior community-based and community-engaged research, programs, and outreach. Participants described community members' familiarity with UAMS, and the employment of staff from their communities, as facilitating reach: “Many of the Marshallese events are being organized by Marshallese folks [staff]. They know of us, so when we approach them—I don't think anybody has turned us down yet” (healthcare provider #1, Marshallese). Long-term community engagement and outreach was described as building trust, supporting the sustainability of external relationships, and improving community buy-in with the vaccine events. As one participant narrated, “I moved here about five years ago, and I saw some of my colleagues work out in the community. […] I was impressed with the work they were doing out in the community. So, I said to myself, well, I'm gonna apply there ‘cause I wanna do work there. I wanna be among those people that are doing impact work with the community” (program staff #6, Marshallese).

In spite of the facilitating role of familiarity with UAMS, participants identified limits to reach stemming from unmet communication needs, especially for some sub-populations who were not as easily reached, even by bilingual Hispanic and Marshallese team members. One participant specifically mentioned reaching older members of the Hispanic community as a barrier: “I have noticed that most of the elderlies [from the Hispanic community] that we get vaccinated, it's somebody else who's bringin’ them. It is not them who got the information firsthand.” This participant also stated that low literacy among older members of the Hispanic community was not addressed through normal outreach techniques: “We still give them the [printed flyers] but there is also the fact that usually Hispanic populations […] our elderlies, […] most of them don't know how to read or write” (program staff #7, Hispanic).

Discussion

This paper explored external relationships (organizational and individual) as barriers and facilitators for equitable community-based implementation of a COVID-19 vaccination intervention leveraging FBOs to reach underserved and hard-to-reach Hispanic and Marshallese communities. Engagement with FBOs to promote health equity and mitigate health disparities among Hispanic and/or Marshallese communities is described in detail in prior publications (41, 49, 57–60). Importantly, vaccination events held in partnership with FBOs reached a higher proportion of Hispanic and Marshallese persons compared to vaccination events in secular, community contexts, and individuals vaccinated at these events were more likely to report completely trusting the COVID-19 vaccine (49). We identified themes at the organizational and individual levels for external relationships as implementation determinants. At the organizational level, formal inter-organizational structures and informal relationships among organizations, which predated the development and implementation of the community-based COVID-19 vaccine events, emerged as implementation determinants. At the individual level, implementation determinants included staff roles in organization and community and community members' familiarity with organization. Participants also identified team members who belonged to the communities of focus as a facilitator for vaccination events, especially for providing culturally appropriate services in language and for leveraging cultural congruence and community members' familiarity with the implementing organization to improve reach. Participants described external relationships as critical factors in creating and maintaining trust among partner organizations and within the Hispanic and Marshallese communities.

Across organizational and individual levels, our analysis highlights three critical factors among organizational and interpersonal relationships as implementation determinants. First, cultural congruence of the implementation team with target communities can be critical to implementing equitable community-engaged interventions. Our findings highlight how implementing organizations can leverage cultural congruence among implementation team members and communities of focus to facilitate outreach, build and maintain trust, and improve the reach of interventions into underserved and hard-to-reach communities. These team members served as an important intermediary between healthcare organizations, community partner organizations, and community members (14, 17, 61–64). Community representation among healthcare workers has also been reported to build trust among communities and healthcare actors (65, 66). However, these team members face unique pressures from overlapping social roles which intertwine their personal and professional lives and may result in greater levels of stress, emotional exhaustion, and burnout (67, 68), particularly during health emergencies that disproportionately impact their communities, such as the COVID-19 pandemic. Future studies of the role of organizational relationships in implementation should consider the interaction between these relationships and “characteristics of individuals” involved in implementation (4), particularly the needs, capability, and motivation of individuals with dual roles in the implementing organization and the community of focus.

Second, long-term, pre-existing relationships among organizations and individuals facilitated implementation and outcomes. Specifically, the preexisting investment in community engagement facilitated the rapid development and implementation of the vaccine events. While it is well-established that building strong relationships is often an essential step in successful community-based interventions, building these relationships with communities requires time and focused effort, even when sufficient organizational resources are available (69, 70). Time, as a contextual variable in implementation, is not explicitly addressed by most IS frameworks, and it is often reduced to a static resource (3). Our findings suggest that the positive effects of community engagement and trust-building strengthen over time, with long-standing formal and informal relationships described most frequently as a critically-important facilitator. The role of time in strengthening community and organizational relationships contrasts with the dynamics of time in the Stages of Implementation Completion framework, where longer durations for tasks predict poor implementation. This gap highlights the lack of focus on community engagement and organizational relationship-building as ongoing processes and implementation determinants (71, 72). However, CEnR scholars, including Wallerstein and colleagues, have cited the importance of long timeframes noting that “despite enhanced focus on research and health outcomes” in community-engaged participatory projects, “the science lags behind the practice,” with little evidence on the mechanisms through which engagement results in outcomes (37). The CEnR literature, especially for community-based participatory approaches, focuses more explicitly on time as a critical factor, and this body of work could be drawn upon to inform future research on community-engaged implementation (35, 73).

Finally, our findings also have implications for trust and trustworthiness as important constructs in equity-focused, community-engaged implementation. Participants explained how long-standing informal relationships among target communities and organizations involved in the intervention engendered trust in new activities or programs facilitating the involvement of hard-to-reach populations, which is broadly consistent with the community engagement literature (37, 69, 74–76). Long-term community engagement also contributed to organizational capacity for culturally-appropriate interventions, as well as the recruitment and retention of community staff whose contributions to COVID-19 vaccine events were described as essential factors in the success of the events through cultural congruence which created and reinforced trust among Hispanic and Marshallese individuals who were more comfortable with members of their own community (74, 77–79).

Recent calls for IS to focus on health equity and adopt justice-focused approaches requires an increased focus on organizational relationships and community engagement, and implementation researchers could draw on the robust CEnR literature to improve health equity (73, 77, 80, 81). Addressing structural health inequalities at the intersections of race, gender, sexual orientation, and immigration status will require a greater focus on incorporating interested communities and individuals into the development, implementation, and evaluation of interventions (27, 28, 32, 82, 27). Partnership strategies have been found to help mitigate and reduce inequities in care, promote individual empowerment, and reduce social stigma of health conditions, which further supports the results of the present study (77, 80). Our findings suggest that if implementation researchers are to engage with communities to mitigate health disparities and promote health equity, attention needs to be paid to both formal and informal relationships among academic institutions, healthcare systems, healthcare providers and clinics, community-based organizations, and communities.

Strengths and limitations

This exploratory study was limited to Northwest Arkansas and focused on COVID-19 vaccination promotion to the Hispanic and Marshallese communities, and hence, generalizability may be limited. Further, our methods were solely qualitative in nature, which also limits generalizability of the findings. These limitations are offset by the diversity and rigor of the qualitative methods used and by the experience and expertise of the research team. As well, the relationships among organizations and the Hispanic and Marshallese communities under study were large and well-established, which provided an opportunity to investigate the potential longer-term implementation-related impacts of these relationships.

Conclusion

Achieving health equity in IS requires a greater understanding of external relationships, both at the organizational and individual levels, as implementation determinants; however, there are significant gaps in the current understanding of how these relationships affect implementation. This article makes a significant contribution to the literature through our exploration of formal and informal relationships among organizations and individuals as critical implementation determinants for community-based COVID-19 vaccination events within FBOs as a way to reach underserved and hard-to-reach Hispanic and Marshallese community members. Across all levels of our analysis, we identified three critical factors for external relationships as implementation determinants. First, cultural congruence of the implementation team with communities of focus can be leveraged to facilitate outreach, build and maintain trust, and improve the reach of interventions into underserved and hard-to-reach communities. Second, long-term, pre-existing relationships allowed for rapid implementation. This study expands the current literature on time as a contextual implementation determinant, which is underdeveloped in IS, with our findings suggesting that the positive effects of community engagement and trust-building strengthen over time and facilitate subsequent outreach and interventions. Finally, this study has implications for understanding trust and trustworthiness in equity-focused, community-engaged implementation, with long-term community engagement contributing to organizational capacity for culturally- and linguistically-appropriate interventions. While these concepts are understudied in IS, the literature of CEnR may be useful in informing IS. Further research is needed to clarify and understand the precise effects of external, organizational relationships on other implementation determinants.

Data availability statement

The deidentified data underlying the results presented in this study may be made available upon reasonable request from the corresponding author, PM, at pamcelfish@uams.edu.

Ethics statement

The studies involving humans were approved by the UAMS Institutional Review Board (IRB#262917). The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.

Author contributions

RM: Formal Analysis, Visualization, Writing – original draft, Writing – review & editing, Methodology. JCK: Conceptualization, Formal Analysis, Writing – original draft, Writing – review & editing, Methodology. JV: Data curation, Formal Analysis, Writing – original draft, Writing – review & editing. SP: Formal Analysis, Writing – original draft, Writing – review & editing. MS: Data curation, Formal Analysis, Writing – original draft, Writing – review & editing. SR: Formal Analysis, Writing – original draft, Writing – review & editing. EA: Writing – original draft, Writing – review & editing. APR: Writing – original draft, Writing – review & editing. ST: Writing – original draft, Writing – review & editing. PM: Conceptualization, Writing – original draft, Writing – review & editing. GC: Conceptualization, Data curation, Formal Analysis, Writing – original draft, Writing – review & editing, Methodology.

Funding

The author(s) declare that financial support was received for the research, authorship, and/or publication of this article.

Support was provided by Community Engagement Alliance (CEAL) Against COVID-19 Disparities (NIH 10T2HL156812-01); a Racial and Ethnic Approaches to Community Health (REACH) award (5 NU58DP006595) from the Centers for Disease Control and Prevention; and University of Arkansas for Medical Sciences Translational Research Institute funding awarded through the National Center for Advancing Translational Sciences of the National Institutes of Health (UL1 TR003107). The content is solely the responsibility of the authors and does not necessarily represent the official views of the funders. This project was also supported by the Health Resources and Services Administration (HRSA) of the United States (US) Department of Health and Human Services (HHS) (6 U3UHS45467-01-01). The contents are those of the authors and do not necessarily represent the official views of, nor an endorsement by, HRSA, HHS, or the US Government. Support was also provided in part by the Arkansas Biosciences Institute, the major research component of the Arkansas tobacco Settlement Proceeds Act of 2000.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/frhs.2024.1338622/full#supplementary-material

References

1. Lenfant C. Clinical research to clinical practice — lost in translation? N Engl J Med. (2003) 349(9):868–74. doi: 10.1056/NEJMsa035507

PubMed Abstract | Crossref Full Text | Google Scholar

2. Rogers EM. Diffusion of Innovations. 5th Ed New York: Free Press (2003).

3. Nilsen P, Bernhardsson S. Context matters in implementation science: a scoping review of determinant frameworks that describe contextual determinants for implementation outcomes. BMC Health Serv Res. (2019) 19(1):1–21. doi: 10.1186/s12913-019-4015-3

PubMed Abstract | Crossref Full Text | Google Scholar

4. Damschroder LJ, Reardon CM, Widerquist MAO, Lowery J. The updated consolidated framework for implementation research based on user feedback. Implement Sci. (2022) 17(1):75. doi: 10.1186/s13012-022-01245-0

PubMed Abstract | Crossref Full Text | Google Scholar

5. Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm Policy Ment Health. (2011) 38:4–23. doi: 10.1007/s10488-010-0327-7

PubMed Abstract | Crossref Full Text | Google Scholar

6. Harvey G, Kitson A. PARIHS revisited: from heuristic to integrated framework for the successful implementation of knowledge into practice. Implement Sci. (2016) 11(1):33. doi: 10.1186/s13012-016-0398-2

PubMed Abstract | Crossref Full Text | Google Scholar

7. DeSisto CL, Kroelinger CD, Estrich C, Velonis A, Uesugi K, Goodman DA, et al. Application of an implementation science framework to policies on immediate postpartum long-acting reversible contraception. Public Health Rep. (2019) 134(2):189–96. doi: 10.1177/0033354918824329

PubMed Abstract | Crossref Full Text | Google Scholar

8. Cannon JS, Gilbert M, Ebener P, Malone PS, Reardon CM, Acosta J, et al. Influence of an implementation support intervention on barriers and facilitators to delivery of a substance use prevention program. Prev Sci. (2019) 20(8):1200–10. doi: 10.1007/s11121-019-01037-x

PubMed Abstract | Crossref Full Text | Google Scholar

9. Daniels SI, Cheng H, Gray C, Kim B, Stave CD, Midboe AM. A scoping review of implementation of health-focused interventions in vulnerable populations. Transl Behav Med. (2022) 12(9):935–44. doi: 10.1093/tbm/ibac025

PubMed Abstract | Crossref Full Text | Google Scholar

10. Rogers LQ, Goncalves L, Martin MY, Pisu M, Smith TL, Hessong D, et al. Beyond efficacy: a qualitative organizational perspective on key implementation science constructs important to physical activity intervention translation to rural community cancer care sites. J Cancer Surviv. (2019) 13(4):537–46. doi: 10.1007/s11764-019-00773-x

PubMed Abstract | Crossref Full Text | Google Scholar

11. Hosey GM, Rengiil A, Maddison R, Agapito AU, Lippwe K, Wally OD, et al. U.S. associated pacific Islands health care teams chart a course for improved health systems: implementation and evaluation of a non-communicable disease collaborative model. J Health Care Poor Underserved. (2016) 27(4A):19–38. doi: 10.1353/hpu.2016.0187

PubMed Abstract | Crossref Full Text | Google Scholar

12. Van Deinse TB, Bunger A, Burgin S, Wilson AB, Cuddeback GS. Using the consolidated framework for implementation research to examine implementation determinants of specialty mental health probation. Health Justice. (2019) 7(1):17. doi: 10.1186/s40352-019-0098-5

PubMed Abstract | Crossref Full Text | Google Scholar

13. McHugh S, Dorsey CN, Mettert K, Purtle J, Bruns E, Lewis CC. Measures of outer setting constructs for implementation research: a systematic review and analysis of psychometric quality. Implement Res Pract. (2020) 1:2633489520940022. doi: 10.1177/2633489520940022

PubMed Abstract | Crossref Full Text | Google Scholar

14. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. (2009) 4(1):50. doi: 10.1186/1748-5908-4-50

PubMed Abstract | Crossref Full Text | Google Scholar

15. Moullin JC, Dickson KS, Stadnick NA, Rabin B, Aarons GA. Systematic review of the exploration, preparation, implementation, sustainment (EPIS) framework. Implement Sci. (2019) 14(1):1. doi: 10.1186/s13012-018-0842-6

PubMed Abstract | Crossref Full Text | Google Scholar

16. Leong J, Jang SH, Bishop SK, Brown EVR, Lee EJ, Ko LK. “We understand our community”: implementation of the healthy eating healthy aging program among community-based organizations. Transl Behav Med. (2021) 11(2):462–9. doi: 10.1093/tbm/ibaa049

PubMed Abstract | Crossref Full Text | Google Scholar

17. Ellis J, Vassilev I, James E, Rogers A. Implementing a social network intervention: can the context for its workability be created? A quasi-ethnographic study. Implement Sci Commun. (2020) 1:93. doi: 10.1186/s43058-020-00087-5

PubMed Abstract | Crossref Full Text | Google Scholar

18. Carter P, Ozieranski P, McNicol S, Power M, Dixon-Woods M. How collaborative are quality improvement collaboratives: a qualitative study in stroke care. Implement Sci. (2014) 9(1):32. doi: 10.1186/1748-5908-9-32

PubMed Abstract | Crossref Full Text | Google Scholar

19. Palinkas LA, Holloway IW, Rice E, Fuentes D, Wu Q, Chamberlain P. Social networks and implementation of evidence-based practices in public youth-serving systems: a mixed-methods study. Implement Sci. (2011) 6(1):113. doi: 10.1186/1748-5908-6-113

PubMed Abstract | Crossref Full Text | Google Scholar

20. Valente TW, Fujimoto K, Palmer P, Tanjasiri SP. A network assessment of community-based participatory research: linking communities and universities to reduce cancer disparities. Am J Public Health. (2010) 100(7):1319–25. doi: 10.2105/AJPH.2009.171116

PubMed Abstract | Crossref Full Text | Google Scholar

21. Wolbring L, Schmidt SCE, Niessner C, Woll A, Wäsche H. Community networks of sport and physical activity promotion: an analysis of structural properties and conditions of cooperation. BMC Public Health. (2022) 22(1):1966. doi: 10.1186/s12889-022-14383-3

PubMed Abstract | Crossref Full Text | Google Scholar

22. Kellam SG. Developing and maintaining partnerships as the foundation of implementation and implementation science: reflections over a half century. Adm Policy Ment Health. (2012) 39(4):317–20. doi: 10.1007/s10488-011-0402-8

PubMed Abstract | Crossref Full Text | Google Scholar

23. Valente TW, Palinkas LA, Czaja S, Chu KH, Brown CH. Social network analysis for program implementation. PloS One. (2015) 10(6):e0131712. doi: 10.1371/journal.pone.0131712

PubMed Abstract | Crossref Full Text | Google Scholar

24. Kothari A, McPherson C, Gore D, Cohen B, MacDonald M, Sibbald SL. A multiple case study of intersectoral public health networks: experiences and benefits of using research. Health Res Policy Syst. (2016) 14:11. doi: 10.1186/s12961-016-0082-7

PubMed Abstract | Crossref Full Text | Google Scholar

25. Provan KG, Milward HB. Do networks really work? A framework for evaluating public-sector organizational networks. Public Adm Rev. (2001) 61(4):414–23. doi: 10.1111/0033-3352.00045

Crossref Full Text | Google Scholar

26. Schwab M, Syme SL. On paradigms, community participation, and the future of public health. Am J Public Health. (1997) 87(12):2049–52. doi: 10.2105/AJPH.87.12.2049

PubMed Abstract | Crossref Full Text | Google Scholar

27. Brownson RC, Kumanyika SK, Kreuter MW, Haire-Joshu D. Implementation science should give higher priority to health equity. Implement Sci. (2021) 16(1):28. doi: 10.1186/s13012-021-01097-0

PubMed Abstract | Crossref Full Text | Google Scholar

28. Kerkhoff AD, Farrand E, Marquez C, Cattamanchi A, Handley MA. Addressing health disparities through implementation science—a need to integrate an equity lens from the outset. Implement Sci. (2022) 17(1):13. doi: 10.1186/s13012-022-01189-5

PubMed Abstract | Crossref Full Text | Google Scholar

29. Bailey ZD, Krieger N, Agénor M, Graves J, Linos N, Bassett MT. Structural racism and health inequities in the USA: evidence and interventions. Lancet. (2017) 389(10077):1453–63. doi: 10.1016/S0140-6736(17)30569-X

PubMed Abstract | Crossref Full Text | Google Scholar

30. Dickman SL, Himmelstein DU, Woolhandler S. Inequality and the health-care system in the USA. Lancet. (2017) 389(10077):1431–41. doi: 10.1016/S0140-6736(17)30398-7

PubMed Abstract | Crossref Full Text | Google Scholar

31. Victora CG, Joseph G, Silva ICM, Maia FS, Vaughan JP, Barros FC, et al. The inverse equity hypothesis: analyses of institutional deliveries in 286 national surveys. Am J Public Health. (2018) 108(4):464–71. doi: 10.2105/AJPH.2017.304277

PubMed Abstract | Crossref Full Text | Google Scholar

32. Baumann AA, Long PD. Equity in implementation science is long overdue (SSIR). Stanf Soc Innov Rev. (2021) 19(3):A15–7. doi: 10.48558/GG1H-A223

Crossref Full Text | Google Scholar

33. Shelton RC, Chambers DA, Glasgow RE. An extension of RE-AIM to enhance sustainability: addressing dynamic context and promoting health equity over time. Front Public Health. (2020) 8:1–8. doi: 10.3389/fpubh.2020.00134

PubMed Abstract | Crossref Full Text | Google Scholar

34. Woodward EN, Matthieu MM, Uchendu US, Rogal S, Kirchner JE. The health equity implementation framework: proposal and preliminary study of hepatitis C virus treatment. Implement Sci. (2019) 14(1):26. doi: 10.1186/s13012-019-0861-y

PubMed Abstract | Crossref Full Text | Google Scholar

35. Brunton G, Thomas J, O’Mara-Eves A, Jamal F, Oliver S, Kavanagh J. Narratives of community engagement: a systematic review-derived conceptual framework for public health interventions. BMC Public Health. (2017) 17(1):1–15. doi: 10.1186/s12889-017-4958-4

PubMed Abstract | Crossref Full Text | Google Scholar

36. McIlduff CD, Forster M, Carter E, Davies J, Thomas S, Turner KMT, et al. Model of engaging communities collaboratively: working towards an integration of implementation science, cultural adaptation and engagement. Int J Crit Indig Stud. (2020) 13(1):45–69. doi: 10.5204/ijcis.v13i1.1346

Crossref Full Text | Google Scholar

37. Wallerstein N. Engage for equity: advancing the fields of community-based participatory research and community-engaged research in community psychology and the social sciences. Am J Community Psychol. (2021) 67(3–4):251–5. doi: 10.1002/ajcp.12530

PubMed Abstract | Crossref Full Text | Google Scholar

38. Metz A, Jensen T, Farley A, Boaz A. Is implementation research out of step with implementation practice? Pathways to effective implementation support over the last decade. Implement Res Pract. (2022) 3:26334895221105585. doi: 10.1177/26334895221105585

PubMed Abstract | Crossref Full Text | Google Scholar

39. Brookman-Frazee L, Stahmer A, Stadnick N, Chlebowski C, Herschell A, Garland AF. Characterizing the use of research-community partnerships in studies of evidence-based interventions in children’s community services. Adm Policy Ment Health. (2016) 43(1):93–104. doi: 10.1007/s10488-014-0622-9

PubMed Abstract | Crossref Full Text | Google Scholar

40. Wesson DE, Lucey CR, Cooper LA. Building trust in health systems to eliminate health disparities. JAMA. (2019) 322(2):111–2. doi: 10.1001/jama.2019.1924

PubMed Abstract | Crossref Full Text | Google Scholar

41. McElfish PA, Cleek AB, Willis DE, Purvis RS, James LP. Leveraging community engagement capacity to address COVID-19 disparities among Pacific Islander and Latinx communities in Arkansas. J Clin Transl Sci. (2021) 5(1):1–4. doi: 10.1017/cts.2020.562

Crossref Full Text | Google Scholar

42. Martin HR, Brown DR, Fluney E, Trepka MJ, Marty AM, Roldan EO, et al. Community-engaged research: COVID-19 testing, infection, and vaccination among underserved minority communities in Miami, Florida. Vaccines (Basel). (2024) 12(2):117. doi: 10.3390/vaccines12020117

PubMed Abstract | Crossref Full Text | Google Scholar

43. Cataldi JR, Suresh K, Brewer SE, Perreira C, Nederveld A, Skenadore A, et al. Boot camp translation using community-engaged messaging for adolescent vaccination: a cluster-randomized trial. Vaccine. (2024) 21:S0264-410X(24)00054-9. doi: 10.1016/j.vaccine.2024.01.042

PubMed Abstract | Crossref Full Text | Google Scholar

44. Haboush-Deloye A, Marquez E, Dunne R, Pharr JR. Nevada Minority Health and Equity Coalition. The importance of community voice: using community-based participatory research to understand the experiences of African American, native American, and Latinx people during a pandemic. Prev Chronic Dis. (2023) 20:E12. doi: 10.5888/pcd20.220152

PubMed Abstract | Crossref Full Text | Google Scholar

45. Willis E, Sabnis S, Hamilton C, Xiong F, Coleman K, Dellinger M, et al. Improving immunization rates through community-based participatory research: community health improvement for Milwaukee’s children program. Prog Community Health Partnersh. (2016) 10(1):19–30. doi: 10.1353/cpr.2016.0009

PubMed Abstract | Crossref Full Text | Google Scholar

46. McElfish PA, Rowland B, Porter A, Felix HC, Selig JP, Semingson J, et al. Use of community-based participatory research partnerships to reduce COVID-19 disparities among Marshallese Pacific Islander and Latino communities—Benton and Washington counties, Arkansas, april–december 2020. Prev Chronic Dis. (2021) 18:210124. doi: 10.5888/pcd18.210124

Crossref Full Text | Google Scholar

47. McElfish PA, Moore R, Laelan M, Ayers BL. Using CBPR to address health disparities with the Marshallese community in Arkansas. Ann Hum Biol. (2018) 45(3):264–71. doi: 10.1080/03014460.2018.1461927

PubMed Abstract | Crossref Full Text | Google Scholar

48. McElfish PA, Kohler P, Smith C, Warmack S, Buron B, Hudson J, et al. Community-driven research agenda to reduce health disparities. Clin Transl Sci. (2015) 8(6):690–5. doi: 10.1111/cts.12350

PubMed Abstract | Crossref Full Text | Google Scholar

49. McElfish PA, Rowland B, Hall S, CarlLee S, Reece S, Macechko MD, et al. Comparing community-driven COVID-19 vaccine distribution methods: faith-based organizations vs. outpatient clinics. J Family Med Prim Care. (2022) 11(10):6081. doi: 10.4103/jfmpc.jfmpc_327_22

PubMed Abstract | Crossref Full Text | Google Scholar

50. Marshall C, Rossman GB. Designing Qualitative Research. Sixth edn Los Angeles, California: SAGE (2016). p. 323.

51. Morse JM. Determining sample size. Qual Health Res. (2000) 10(1):3. doi: 10.1177/104973200129118183

Crossref Full Text | Google Scholar

52. Boddy CR. Sample size for qualitative research. QMR. (2016) 19(4):426–32. doi: 10.1108/QMR-06-2016-0053

Crossref Full Text | Google Scholar

53. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. (2009) 42(2):377–81. doi: 10.1016/j.jbi.2008.08.010

PubMed Abstract | Crossref Full Text | Google Scholar

54. Harris PA, Taylor R, Minor BL, Elliott V, Fernandez M, O’Neal L, et al. The REDCap consortium: building an international community of software platform partners. J Biomed Inform. (2019) 95:103208. doi: 10.1016/j.jbi.2019.103208

PubMed Abstract | Crossref Full Text | Google Scholar

55. Gale NK, Heath G, Cameron E, Rashid S, Redwood S. Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Med Res Methodol. (2013) 13(1):117. doi: 10.1186/1471-2288-13-117

PubMed Abstract | Crossref Full Text | Google Scholar

56. Ritchie J, Lewis J, Nicholls CM, Ormston R. Qualitative Research Practice: A Guide for Social Science Students and Researchers. Thousand Oaks: SAGE Publications Inc. (2013). p. 457.

57. McElfish PA, Boyers J, Purvis RS, O’Connor B, Carleton A, Bing W, et al. Family model diabetes self-management education and support in faith-based organizations in the republic of the Marshall Islands study protocol. Contemp Clin Trials Commun. (2021) 21:100710. doi: 10.1016/j.conctc.2021.100710

PubMed Abstract | Crossref Full Text | Google Scholar

58. McElfish P, Bing W, Ayers BL, Smith L, Stephens M, Wilmoth R, et al. Lessons learned through a partnership with Marshallese faith-based organizations to screen for hypertension and diabetes. J Reg Med Campuses. (2018) 1(3):1–6. doi: 10.24926/jrmc.v1i3.1044

Crossref Full Text | Google Scholar

59. McElfish PA, Rowland B, Long CR, Hudson J, Piel M, Buron B, et al. Diabetes and hypertension in Marshallese adults: results from faith-based health screenings. J Racial Ethn Health Disparities. (2017) 4(6):1042–50. doi: 10.1007/s40615-016-0308-y

PubMed Abstract | Crossref Full Text | Google Scholar

60. McElfish PA, Riklon S, Purvis RS, Long CR, Felix HC, Hudson JS, et al. Study protocol for family model diabetes self-management education with Marshallese participants in faith-based organizations. Contemp Clin Trials Commun. (2022) 30:101007. doi: 10.1016/j.conctc.2022.101007

PubMed Abstract | Crossref Full Text | Google Scholar

61. Chamie G, Prado P, Oviedo Y, Vizcaíno T, Arechiga C, Marson K, et al. Reproducibility and implementation of a rapid, community-based COVID-19 “test and respond” model in low-income, majority-latino communities in Northern California. PloS One. (2022) 17(10):e0276257. doi: 10.1371/journal.pone.0276257

PubMed Abstract | Crossref Full Text | Google Scholar

62. Demeke J, McFadden SM, Dada D, Djiometio JN, Vlahov D, Wilton L, et al. Strategies that promote equity in COVID-19 vaccine uptake for undocumented immigrants: a review. J Community Health. (2022) 47(3):554–62. doi: 10.1007/s10900-022-01063-x

PubMed Abstract | Crossref Full Text | Google Scholar

63. Peretz PJ, Islam N, Matiz LA. Community health workers and COVID-19 — addressing social determinants of health in times of crisis and beyond. N Engl J Med. (2020) 383(19):e108. doi: 10.1056/NEJMp2022641

PubMed Abstract | Crossref Full Text | Google Scholar

64. Wells KJ, Dwyer AJ, Calhoun E, Valverde PA. Community health workers and non-clinical patient navigators: a critical COVID-19 pandemic workforce. Prev Med. (2021) 146:106464. doi: 10.1016/j.ypmed.2021.106464

PubMed Abstract | Crossref Full Text | Google Scholar

65. Purvis RS, Bing WI, Jacob CJ, Lang S, Mamis S, Ritok M, et al. Community health warriors: Marshallese community health workers’ perceptions and experiences with CBPR and community engagement. Prog Community Health Partnersh. (2017) 11(3):315–20. doi: 10.1353/cpr.2017.0037

PubMed Abstract | Crossref Full Text | Google Scholar

66. Kok MC, Ormel H, Broerse JEW, Kane S, Namakhoma I, Otiso L, et al. Optimising the benefits of community health workers’ unique position between communities and the health sector: a comparative analysis of factors shaping relationships in four countries. Glob Public Health. (2017) 12(11):1404–32. doi: 10.1080/17441692.2016.1174722

PubMed Abstract | Crossref Full Text | Google Scholar

67. Catalani CEC, Findley SE, Matos S, Rodriguez R. Community health worker insights on their training and certification. Prog Community Health Partnersh. (2009) 3(3):227–35. doi: 10.1353/cpr.0.0082

PubMed Abstract | Crossref Full Text | Google Scholar

68. Katzen LS, Skeen S, Dippenaar E, Laurenzi C, Notholi V, le Roux K, et al. Are we listening to community health workers? Experiences of the community health worker journey in rural South Africa. Res Nurs Health. (2022) 45(3):380–9. doi: 10.1002/nur.22220

PubMed Abstract | Crossref Full Text | Google Scholar

69. Tembo D, Hickey G, Montenegro C, Chandler D, Nelson E, Porter K, et al. Effective engagement and involvement with community stakeholders in the co-production of global health research. Br Med J. (2021) 372:n178. doi: 10.1136/bmj.n178

Crossref Full Text | Google Scholar

70. Shalowitz MU, Isacco A, Barquin N, Clark-Kauffman E, Delger P, Nelson D, et al. Community-Based participatory research: a review of the literature with strategies for community engagement. J Dev Behav Pediatr. (2009) 30(4):350. doi: 10.1097/DBP.0b013e3181b0ef14

PubMed Abstract | Crossref Full Text | Google Scholar

71. Chamberlain P, Brown CH, Saldana L. Observational measure of implementation progress in community based settings: the stages of implementation completion (SIC). Implement Sci. (2011) 6(1):116. doi: 10.1186/1748-5908-6-116

PubMed Abstract | Crossref Full Text | Google Scholar

72. Saldana L, Chamberlain P, Wang W, Hendricks Brown C. Predicting program start-up using the stages of implementation measure. Adm Policy Ment Health. (2012) 39(6):419–25. doi: 10.1007/s10488-011-0363-y

PubMed Abstract | Crossref Full Text | Google Scholar

73. Boden-Albala B, Rebello V, Drum E, Gutierrez D, Smith WR, Whitmer RA, et al. Use of community-engaged research approaches in clinical interventions for neurologic disorders in the United States: a scoping review and future directions for improving health equity research. Neurology. (2023) 101(7 Supplement 1):S27–46. doi: 10.1212/WNL.0000000000207563

PubMed Abstract | Crossref Full Text | Google Scholar

74. Lucero JE, Boursaw B, Mickey EM, Greene-Moton E, Wallerstein N, Oetzel JG. Engage for equity: the role of trust and synergy in community-based participatory research. Health Educ Behav. (2020) 47(3):372–9. doi: 10.1177/1090198120918838

PubMed Abstract | Crossref Full Text | Google Scholar

75. Karris MY, Dube K, Moore AA. What lessons it might teach US! community engagement in HIV research. Curr Opin HIV AIDS. (2020) 15(2):142–9. doi: 10.1097/COH.0000000000000605

PubMed Abstract | Crossref Full Text | Google Scholar

76. Agley J, Xiao Y, Thompson EE, Chen X, Golzarri-Arroyo L. Intervening on trust in science to reduce belief in COVID-19 misinformation and increase COVID-19 preventive behavioral intentions: randomized controlled trial. J Med Internet Res. (2021) 23(10):e32425. doi: 10.2196/32425

PubMed Abstract | Crossref Full Text | Google Scholar

77. Khurshid K, Dutta T, Khan KS, Haider SA, Ahmed UA, Jatoi H, et al. Addressing stigma among persons living with HIV through the sujaag project in Pakistan: a socioecological analysis. J Soc Issues. (2023) 79(1):446–64. doi: 10.1111/josi.12547

Crossref Full Text | Google Scholar

78. Dutta T, Agley J, Xiao Y, Golzarri-Arroyo L, Ali S. Students’ COVID-19 vaccine behaviors, intentions, and beliefs at a US native American-serving nontribal institution (NASNTI). BMC Res Notes. (2023) 16(1):175. doi: 10.1186/s13104-023-06439-3

PubMed Abstract | Crossref Full Text | Google Scholar

79. Dutta T, Agley J, Meyerson BE, Barnes PA, Sherwood-Laughlin C, Nicholson-Crotty J. Perceived enablers and barriers of community engagement for vaccination in India: using socioecological analysis. PloS One. (2021) 16(6):e0253318. doi: 10.1371/journal.pone.0253318

PubMed Abstract | Crossref Full Text | Google Scholar

80. Stollak I, Valdez M, Story WT, Perry HB. Reducing inequities in maternal and child health in rural Guatemala through the CBIO+ approach of curamericas: 8. Impact on women’s empowerment. Int J Equity Health. (2023) 21(2):200. doi: 10.1186/s12939-022-01760-y

PubMed Abstract | Crossref Full Text | Google Scholar

81. Comfort M, Raymond-Flesch M, Auerswald C, McGlone L, Chavez M, Minnis A. Community-engaged research with rural latino adolescents: design and implementation strategies to study the social determinants of health. Gateways: Int J Comm Res Engagement. (2018) 11(1):90. doi: 10.5130/ijcre.v11i1.5721

Crossref Full Text | Google Scholar

82. Adsul P, Chambers D, Brandt HM, Fernandez ME, Ramanadhan S, Torres E, et al. Grounding implementation science in health equity for cancer prevention and control. Implement Sci Commun. (2022) 3(1):1–13. doi: 10.1186/s43058-022-00311-4

PubMed Abstract | Crossref Full Text | Google Scholar