Unpacking Partnership, Engagement, and Collaboration Research to Inform Implementation Strategies Development: Theoretical Frameworks and Emerging Methodologies

Background: Partnership, engagement, and collaboration (PEC) are critical factors in dissemination and implementation (D&I) research. Despite a growing recognition that incorporating PEC strategies in D&I research is likely to increase the relevance, feasibility, impacts, and of evidence-based interventions or practices (EBIs, EBPs), conceptual frameworks and methodologies to guide the development and testing of PEC strategies in D&I research are lacking. To address this methodological gap, a review was conducted to summarize what we know, what we think we know, and what we need to know about PEC to inform D&I research.

Methods: A cross-field scoping review, drawing upon a broad range of PEC related literature in health, was conducted. Publications reviewed focused on factors influencing PEC, and processes, mechanisms and strategies for promoting effective PEC. The review was conducted separately for three forms of partnerships that are commonly used in D&I research: (1) consumer-provider or patient-implementer partnership; (2) delivery system or implementation team partnership; and (3) sustainment/support or interagency/community partnership. A total of 39 studies, of which 21 were review articles, were selected for an in-depth review.

Results: Across three forms of partnerships, four domains (cognitive, interpersonal/affective, behavioral, and contextual domains) were consistently identified as factors and strategies for promoting PEC. Depending on the stage (preparation or execution) and purpose of the partnership (regulating performance or managing maintenance), certain PEC strategies are more or less relevant. Recent developments of PEC frameworks, such as Partnership Stage of Change and multiple dynamic processes, provide more comprehensive conceptual explanations for PEC mechanisms, which can better guide PEC strategies selection and integration in D&I research.

Conclusions: This review contributes to D&I knowledge by identifying critical domain factors, processes, or mechanisms, and key strategies for PEC, and offers a multi-level PEC framework for future research to build the evidence base. However, more research is needed to test PEC mechanisms.

Background

Introduction

Dissemination and Implementation (D&I) research, which involves the use of diverse strategies to facilitate adoption, integration, and sustainability of evidence-based interventions and practices (EBIs/EBPs) in diverse settings, is a rapidly growing field in health research (1). The successful development, implementation, dissemination, and sustainability of EBIs/EBPs requires communication, collaboration, and consensus among all involved, including consumers (end users), implementers, and related partners who contribute to sustainability of EBIs/EBPs. To accomplish this, Partnership, partner Engagement, and Collaboration (PEC) have been identified as critical strategies in D&I research (2, 3). In a recent compilation of recommended strategies by D&I experts, more than one-third were PEC-related strategies (e.g., coalition building, creating a learning collaborative, developing academic partnerships, involving patient/consumers and family members, organizing clinician implementation meetings, and promoting network weaving) (3). Despite the importance of PEC strategies and their potential contribution for improving service implementation and health outcomes at the individual, community, and population levels, the conceptualization and methodologies for studying PEC are not well defined and not well integrated into D&I research. Although PEC research has been applied in multiple fields over the years, including military, business, sports, academia, and health, lessons learned, and findings from these fields have not been systematically applied to inform PEC strategies for enhancing EBIs/EBPs in implementation research.

Multilevel Partnership, Engagement, and Collaboration in D&I Research

In D&I research, PEC can be applied across different programs and interventions (4) and with diverse partners form multiple levels (5). Partners involved in D&I usually include consumers, a team of providers/implementers (i.e., those who provide EBIs/EBPs), and a team of multi-disciplinary partners (i.e., those who set up structures and policies, and provide support for implementation and sustainment of EBIs/EBPs). The purpose of developing strong PEC in D&I research is to build support across the individual, team, and organizational levels to work toward common goals for EBIs/EBPs, use or share skills and resources to implement EBIs/EBPs, and seek input and support of experts from different disciplines. Therefore, D&I research requires consideration of PEC strategies for multiple forms or multiple levels of partnerships.

At the consumer-provider level (consumers also defined as patients or targets of EBIs/EBPs, and providers also defined as implementers who provide EBIs/EBPs), PEC between consumers and providers is critical because substantial research has documented that an effective patient-provider relationship can optimize the patient's use of intervention strategies and engagement in treatment (6, 7). Greater patient-centered care or patient-provider partnerships are associated with better patient outcomes, including increased health knowledge, management skills, competency, self-efficacy, and sense of control and wellbeing over personal health, and well-being (8). Additionally, better patient-provider partnership also benefits providers because of increased patient satisfaction with their care (8). Therefore, application of PEC strategies to promote patient-provider partnerships has implications to improve EBIs/EBPs acceptability and patient-centered care outcomes, and to enhance patients' use of EBI health promotion and management strategies.

At the EBI/EBP delivery system level (or implementation team level), quality of interaction, relationship and behavioral processes of implementation team members can influence teams' performance and effectiveness in EBIs/EBPs implementation (9, 10). Partnership and implementation barriers that are commonly identified at this level include: lack of effective communication and coordination among team members, lack of sufficient buy-in from team members, high turnover, failure of partnership leaders to engage team members, lack of sufficient funds to support partnerships, and team member burnout (11). Therefore, PEC strategies that engage teams' long term collaborative efforts, and empower and motivate members to proactively problem-solve partnership barriers may enhance team efficiency and the quality of EBIs/EBPs implementation (9, 10).

At the sustainment/support system level, D&I research requires consideration of the sustainment and sustainability of EBIs/EBPs. Sustainment is the continued use of EBIs/EBPs within practice settings (12); sustainability is the extent to which the EBIs/EBPs can be delivered with their “intended benefits” over an extended period of time after external support from the donor agency terminates (13). Thus to support sustainment and sustainability, D&I research requires PEC between implementation team members and external partners (e.g., patient advocates, EBI/EBP providers, funders, researchers, institutions, community-based organizations, relevant policymakers, and healthcare system partners) (4, 14). Such cross-disciplinary and cross-organizational partnerships address potential structural and system-level barriers and to the expansion and sustainability of EBIs/EBPs (5). Partnership barriers that commonly occur at this level include conflicts between: (1) priorities and competing demands across organizations or communities; (2) leaders' and partners' roles; and (3) models of partner/ community relationships (15). Therefore, utilizing PEC strategies to address conflicts and promote partner engagement and cross-disciplinary partnerships will have important implications for gaining greater support in implementing and sustaining EBIs/EBPs.

The Study Aims

While partnerships in D&I research commonly occur at multiple levels, there has been no multi-level conceptual model to guide PEC strategy development or testing. PEC research is often carried out separately for different partnership levels, and commonly focuses on one level at a time. To inform the development of an integrated D&I framework for PEC strategies, it is important to understand and summarize current research on each partnership level, especially related to the core components and theoretical processes that contribute to effective PEC strategies. Thus, the overall goal of this paper is to address D&I knowledge gaps by reviewing PEC literature and synthesizing knowledge to guide the development of a multi-level PEC theoretical framework. The review focuses specifically on PEC factors influencing PEC processes and outcomes, theoretical frameworks, and evidence from testing of PEC strategies. Given that the central component of the partnership is interpersonal relationship building, we expected that the literature would identify core components that work across different levels of partnerships. The review was therefore synthesized separately for the three levels of partnership. This paper was not intended to be an exhaustive review of the literature, but rather to provide a high-level view of the approaches in which multi-level PEC strategies are studied in D&I contexts.

Method

Definitions

Terms related to PEC have been widely used interchangeably and inconsistently. For the purpose of this study, definitions from an array of review papers, as detailed below, were applied to guide our review (5, 14, 16). Review papers were selected based on the inclusion criteria described in the Method section.

Partnership

In D'amour et al.'s review paper, partnership is defined as “two or more actors join[ed] in a collaborative undertaking (or a set of common goals and specific outcomes) that is characterized by a collegial like relationship that is authentic and constructive.” A partnership can be a relationship between as few as two partners or it can involve a larger number of individuals from groups and organizations (e.g., a network, coalition, or consortium) (5). Under this definition, partnership research has focused on approaches to developing partnerships (e.g., formalizing, sustaining, and ending partnerships) and strategies to build strong working relationships (e.g., cognitive, emotional, and behavioral strategies) (17).

Engagement

Based on Concannon et al.'s review, engagement is defined as “A bi-directional relationship between the patient (or consumer, family) and provider (or implementer) or between the partner and researcher that results in informed decision-making about the selection, conduct, and use of research or interventions” (14). Under this definition, engagement research has focused on strategies to build strong bi-directional relationships between the partners that enhance trust, commitment to collaborate, shared decision-making, problem-solving, and behavioral changes. The terms engagement and alliance are often used inter-changeably (14).

Collaboration

Based on Mattessich et al.'s review (16), collaboration is defined as a mutually beneficial and well-defined relationship entered into by two or more organizations to achieve common goals. Collaborations include a commitment to mutual objectives, a jointly developed structure, shared responsibility, mutual authority, and accountability for success, and sharing of resources and rewards (16). Under this definition, collaboration in research has focused on strategies to enhance partners' ability to work together to achieve mutual benefits (17). Collaboration is conceptualized as distinctive from cooperation and coordination, which represent earlier stages of organizational partnership (16). Specifically, cooperation is characterized by informal relationships (that exists without any commonly defined mission or planning effort), informal information sharing, preserved authority in each organization, and separated resources by organizations. Coordination is characterized by a more formal relationship, an understanding of compatible missions, with some planning and division of roles, and some established communication channels (16).

Taken together, based on the listed definitions, partnership can be conceptualized as a broader umbrella term that includes engagement and collaboration. Partnerships can occur in multiple forms and at different levels. Therefore, in our review, we included inter-related PEC literatures and diverse types and forms of partnerships.

Literature Review Methods

A cross-field scoping review was conducted, and the review was carried out separately for three forms of partnerships that are commonly applied in D&I research (described above). The scoping review method was used because it provides a useful initial approach to generate foundational knowledge (for each level of PEC research), and to inform approaches for future systematic review (18). In the scoping review, the 5-step method outlined by Arksey and O'Malley(18) was applied. The 5 steps include: (1) identifying the research question (i.e., factors and processes for three levels of PEC); (2) identifying relevant studies/literature; (3) study selection; (4) charting the data; and (5) collating, summarizing, and reporting results. The overall inclusion criteria of articles for this review included studies that: (1) examined partnership, engagement, and/or collaboration factors, processes, mechanisms, or effectiveness of strategies; (2) examined diverse types and forms of partnerships; (3) had health implications; and (4) were published in English language, peer reviewed literature, from 2000 to 2017, and in PubMed, Ovid MEDLINE, or by credible federal research institutions (e.g., NIH, AHRQ, CDC). Studies that only characterized or described partnership development approaches, and did not examine factors associated with PEC, discuss theoretical frameworks, or assess partnership outcomes were excluded.

To understand consumer-provider level PEC, the literature about patient-provider partnerships, patient/family-centered care, patient/family engagement research that focuses on factors that influence PEC processes and outcomes, and intervention strategies for promoting consumer-provider relationships and engaging consumers to actively use EBI strategies were reviewed. For delivery-system-level PEC, relevant literature about team collaboration, teamwork, inter-professional collaboration, and teamwork interventions that focused on factors that influence PEC processes and outcomes, and intervention strategies for effective team partnership and teamwork were reviewed (9). To understand sustainment/support system level PEC, the literature about multidisciplinary collaboration, quality improvement collaboration, patient-centered outcome research (PCOR), patient/community participation in research, community-based participatory research (CBPR), and collaborative/team science research that focused on factors that influence PEC processes and outcomes, and intervention strategies for effective collaboration across diverse organizations and disciplines were reviewed. These themes were considered because they included diverse partners from multiple organizations, emphasized equitable partnership building, studied factors related to development and sustainment of collaboration, and considered complexity in collaboration process (14, 19–24).

Figure 1 shows the multilevel PEC conceptual framework that guided this literature review [adapted from Proctor et al. (25)]. The gray boxes represent a summary of the findings from the content synthesis.

FIGURE 1

Figure 1. An integrated multilevel partnership, engagement, and collaboration framework for D&I research. Model adapted from Proctor et al.'s (25). Model of implementation research. Each level of PEC represents various types of partnerships. The consumer-provider level includes partnerships between consumers or targets of EBIs/EBPs and implementers who provide EBIs/EBPs. The EBI/EBP delivery system level includes implementation team members from EBIs/EBPs. The sustainment/support system level includes partners for sustainment and sustainability of EBIs/EBPs. Description in boxes “Features for effective PEC” and “PEC implementation outcomes” resulted from the PEC literature review.

Results

Tables 1S–3S in the Supplemental file document the charting of review data in detail for studies included in the three levels of PEC literature. Below, findings for each level of partnership are synthesized. Each section is divided into a description of factors that influence PEC processes and outcomes, frameworks used to study PEC strategies, and studies that tested PEC strategies and impact evidence for PEC strategies. A very large body of PEC related studies met our inclusion and exclusion criteria (see number below for each level of PEC review); therefore, priority was placed on review papers, when available. Reviews that examined features for effective PEC and provided approaches for assessing PEC outcomes were included first. Additional selected articles were added when predictors and processes for effective PEC were not covered in the reviewing articles. Figure 1 and Table 1 show a summary of key results from the cross-level PEC literature review, based on the included articles.

TABLE 1

Table 1. Summary of literature review on PEC related factors, frameworks, and intervention strategies.

Consumer-Provider (or Patient-Implementer) Level Partnership

More than 5,000 articles related to patient-provider partnership and patient/family engagement were identified, along with several review papers. To avoid redundancy, this review focused on synthesizing findings from eight relevant consumer-provider level PEC review papers (four focused on PEC interventions, four focused on PEC related factors) and two framework papers. In total, the selected 10 studies represented findings from 425 research articles.

Factors that Influence PEC Processes and Outcomes

The literature outlined four domains that influence effective consumer-provider partnerships. These include: (a) cognitive domain (e.g., providing knowledge; listening and recognizing patients' perspectives and experiences; assessing patients' strengths and needs); (b) affective and interpersonal relationship domain (e.g., developing trust, caring, empathetic, respectful, supportive relationship; partnership alliance; identifying and handling emotional problems); (c) behavioral domain (e.g., shared decision-making, providing support, actions to increase EBI/EBP accessibility, actions for finding and trying out solutions to address problems or increase participation, and reinforcement management or homework assignment to increase positive behaviors) (6, 8, 26–32); and (d) contextual factor domain (e.g., health service environment, system, and resources; social determinants; individual partner characteristics). The contextual factors influence not only patient-provider partnership behaviors and processes, but also subsequent PEC outcomes (e.g., cognitive benefit, satisfaction, intervention engagement) (28, 30).

PEC Frameworks

The Mutual Participation Model of Care and Transtheoretical Model of Behavior Change (TTM) model have been applied in studying patient-provider PEC processes. The Mutual Participation Model proposes that patient-provider mutual participation and approximately equal power in the treatment process will increase patients' sense of self-efficacy, improve self-management of health, and increase active participation in treatment (6). The TTM, also known as the Stages of Change Model, proposes that patients move through five stages of change that represent different levels of readiness to engage in behavior change: pre-contemplation/not ready to change, contemplation/getting ready, preparation/ready, action/making change, and maintenance stages. Based on this model, providers are more likely to successfully engage patients in behavior change if they use communication strategies that match recommendations with patients' level of readiness to change (33). For patients in the early stages of TTM, providers may focus on cognitive and affective strategies to build buy-in, awareness, and trust to prepare for change. For patients in the later stages of TTM, providers may focus on behavioral management, support, and motivation strategies to build strong relationship that support and maintain patient's change (33).

PEC Strategies Testing

Based on findings derived from four review articles, which synthesize results from 100 intervention studies, most intervention research has targeted providers and patients separately. In the interventions that targeted providers, most strategies tested were communication and consultation strategies, particularly focused on psychological and relational aspects of communication. Strategies included helping providers gain skills in identifying and managing patients' emotional states, sharing decision-making, demonstrating empathy, and seeing each patient as a whole and unique individual (34, 35). In the interventions that targeted the patients/consumers, most strategies have focused on promoting patients' cognitive preparation (e.g., psychoeducation that promotes knowledge, realistic expectations, and participation in EBIs), increasing use of assessment (i.e., assessing patients' barriers to participate and then discussing solutions with patients), and increasing participation (e.g., promoting access to services, increasing attendance and/or adherence) (26, 36).

Impact Evidence for Individual Level PEC Strategy Testing

Evidence showed that interventions focused on provider communication/consultation style training (with 52 randomized controlled trial [RCT] studies out of 60 studies included in the review papers) resulted in significant impact on improving consultation processes, providers' communication skills, and patient satisfaction (34, 35). However, the effects of such interventions on patient healthcare behaviors and health outcomes were limited. Only complex interventions directed at both providers and patients that included condition-specific educational materials demonstrated greater health benefits for patients compared to single component targeted interventions (34, 35). For interventions focused on patient engagement (with 40 RCT studies included in the selected reviews), researchers found that assessment, strategies that promoted access to services, and psychoeducation were more likely to improve patients' engagement in EBIs (measured by attendance, adherence, and cognitive preparation) compared to interventions that did not use these strategies (26).

Delivery System Level (Implementation Team Partnership)

To understand implementation team member partnerships, literature related to teamwork was reviewed. More than 4,000 articles were identified on team collaboration, teamwork, inter-professional collaboration, and teamwork intervention research. For this review, seven relevant papers were reviewed, four of which were review papers (three focused on factors influencing team PEC or/and processes, and one focused on interventions for promoting teamwork and team performance). The seven reviewed papers represented findings from 434 research articles.

Factors that Influence PEC Process and Outcomes and PEC Frameworks

Factors that influence PEC processes and outcomes were based on two representative teamwork frameworks (10, 37, 38). Specifically, the Integrated Framework for effective teamwork, developed by Rousseau et al. based on a review of 29 studies that examined teamwork behaviors and processes, proposes that teamwork behaviors are constructed in a nested hierarchical structure (10). Effective team PEC needs to consider two domains: (1) behaviors that function to regulate a team's performance; and (2) management of team maintenance. With regard to regulating team performance, PEC strategies need to include those that occur (a) before team task performance (e.g., creating action plans, team mission analysis, goal setting, cognitive preparation for team task performance); (b) during the execution of team performance (e.g., coordination, cooperation, and information exchange, task-related collaborative behaviors/strategies, monitoring team performance, reflection); and (c) after task team adjustment period (e.g., intra-team coaching, collaborative problem-solving, and team practice innovation). With regard to management of team maintenance, PEC strategies may include psychological support and integrative conflict management (10).

Different from Rousseau's framework (10). which focuses more on PEC strategies based on the stage of team development, Kozlowski et al. (37, 38) proposed a Team Process Framework that posits that the context in which a team works influences team processes, which in turn influence team effectiveness and performance. In Kozlowski's model, team PEC needs to be conceptualized in a multilevel context (considering individual, organizational system, and environmental influences). Moreover, in order to have effective team-level PEC, partnership factors in three distinct but inter-related team processes need to be considered, including: (a) cognitive team processes (e.g., collective team climate and safety climate, team mental models, team learning factors); (b) team interpersonal, motivation, and affective processes (e.g., team cohesion, team efficacy, team affect/emotion/conflict); and (c) team action and behavioral processes (e.g., team coordination/cooperation/communication, team competencies/functions, team regulation, performance dynamics, adaptation) (37, 38).

Authors of other review and theoretical perspective papers (39, 40) also suggest that promoting teamwork requires similar processes to the frameworks described above [e.g., in a review paper of teamwork monitoring instruments, most have focused on team contexts and behavioral processes based on the two conceptual frameworks described above (40)]. In team contexts, team composition and structure, organizational climate, individual attitudes, beliefs, value, and commitment about teamwork were commonly assessed. In assessing team behaviors, collaborative behaviors, such as communication, goal settings, task analysis, monitoring, adjustment collaboration, problem-solving, decision-making, workload sharing, conflict, and team leadership were commonly assessed (40). Team climate, including climate related to psychological safety, team objectives, team commitment, and support for innovation, has also been proposed for fostering effective team PEC and recommended for carefully monitoring (39). These commonly assessed constructs represent the importance of these factors in team PEC processes.

PEC Strategies Testing

In a meta-analysis based on 72 interventions from diverse fields, researchers reported that most intervention strategies have targeted team member training and most content designs were based on the Integrated Framework (10). Strategies commonly included were related to team regulation strategies (e.g., strategies to keep teams engaged during teamwork preparation, execution, and reflection) and team maintenance strategies (e.g., conflict management and psychological support) (9). Other training models applied holistic/humanities and team co-learning approach. This training approach focused on the patient holistic care concept and provided tools and opportunities to facilitate team members' co-learning and inter-professional team collaboration to provide holistic patient care (41).

Impact Evidence for Team-Level PEC Strategy Testing

Authors of the meta-analysis reported that overall, team training had significant, medium-sized effects in enhancing both teamwork and team performance across a variety of team contexts and training methods (9). In addition, regardless of the targeted domains (e.g., preparation, execution, reflection, interpersonal dynamics) and number of teamwork domains targeted, teamwork training significantly improved team performance. However, in terms of improving teamwork behaviors, significant effects only emerged when two or more domains of teamwork were targeted (9). Trainings using the holistic/humanities and team co-learning approach resulted in significant improvements in team efficiency, team value, shared roles, knowledge, satisfaction, and reactions to working in team across all levels of learners (including non-English-speaking and diverse provider staff) (41).

Sustainment/Support System Level PEC

To understand sustainment/support system-level partnerships, literature on interdisciplinary collaboration, quality improvement collaboration, patient/community research partnership research (e.g., PCOR, CBPR, patient/community participation in research), and team/collaborative science was reviewed. More than 7,000 articles were identified. For this review, 22 relevant studies were included, 9 of which were review papers and 13 of which were frameworks or empirical studies that examined PEC factors or/and processes. The reviewed 22 studies represented findings from 597 research articles.

Factors that Influence PEC Process and Outcomes

Factors and processes for two key topic areas of literature—interdisciplinary collaboration and patient/community-academic partnership research—were examined separately given the rich and diverse topics within each field of research.

Twelve studies that described interdisciplinary collaboration research were reviewed in detail. These included studies on interdisciplinary, quality improvement collaboration [QIC], and team/collaborative science (six were reviews). Overall, results revealed that factors influencing effective PEC mapped onto two broad domains: (a) Factors related to team foundation; and (b) factors related to processes. Factors frequently studied under team foundation were related to collaboration environment (e.g., history, political/social climate, interdependence, flexibility, reflection on process, collective ownership, mutual respect, ability to compromise, trust), team composition (e.g., team diversity, disciplinary dynamic, multiple layer of participation, representation of organization), and organization characteristics (e.g., resource, fund, staff, time, incentive, skilled leadership). Better understanding of team foundation factors and PEC contexts can guide the use of PEC strategies to prepare PEC set-up and increase partners' readiness for PEC. Factors frequently studied under the processes were related to cognitive processes (e.g., clear roles, shared visions/values, concrete attainable goals, cross-disciplinary learning), interpersonal/ motivational/affective processes (e.g., established informal relationships, communication mechanisms, value the contribution of collaborators), and behavioral processes (e.g., having open and honest communication, sharing decision-making, power sharing, acknowledging egalitarian nature of relationships, identifying barriers, and problem-solving) (5, 16, 42–47).

Separate from the interdisciplinary collaboration literature, eight studies on patient/community-academic partnership research, including literatures from PCOR, CBPR, and patient/community research partnership research, were also reviewed (four were reviewed studies). Overall, similar PEC factors were identified as in the interdisciplinary collaboration literature, as well as in the implementation team-level partnership literature (described above). However, there were some differences in two areas of research. Patient/community-academic partnership literature was more likely to discuss factors or strategies based on stages of partnership (rather than domains). Factors or strategies frequently studied during the preparation period were interpersonal and operational process related strategies. These might include sharing goals, establishing an engaged and supportive organizational culture, developing institutional structure to address and support potential system barriers, developing mutual respect, and building partners' capacity for partnering skills (22, 24, 48, 49). Factors or strategies frequently studied during the PEC execution period were partnership synergy, knowledge exchange, monitoring, and support related strategies (e.g., co-learning strategies, building reciprocal/equal relationships, assessment, and feedback) (22, 24, 50).

PEC Frameworks

There are several conceptual frameworks (49) for explaining the effect of PEC at the level of sustainability and support systems. Because of the complexity of partnership at this level, more PEC frameworks have been developed. Conceptual frameworks, such as the Team Efficiency Framework, Social Exchange Theory, PCOR, CBPR, Stage Process Framework, have been applied in interdisciplinary/interagency PEC research (5, 51–54). The Team Efficiency Framework, which is commonly applied in team member partnership (described above), proposes that multi-disciplinary collaboration is a process/configuration of input (contextual factors) process (cognitive, relationship/affective, and behavioral processes) outcomes (performance, innovation, viability) (55). The Social Exchange Theory proposes that an individual/organization joins a group for exchange purposes. The partnership provides specific benefits to individuals/organizations and that, in return, the individuals/organizations are expected to help the group attain its objectives. From this perspective, the challenges that commonly occur during partnerships are related to concerns about power-sharing in attaining equal benefits (56). Therefore, PEC strategies focused on power, decision-making, and interaction dynamics are commonly proposed. The PCOR Framework, developed by the Patient-Centered Outcomes Research Institute/PCORI, emphasizes trust, honesty, co-learning, transparency, reciprocal relationships, partnership, and respect in collaboration processes. PCOR proposes two broader domains of PEC factors to be considered: (a) contextual factors: including internal factors such as awareness of methods for PCOR, a patient centered culture, and external factors (e.g., ways for patients and researchers to partner, resources and infrastructure, policies and governance); and (b) engagement action of PEC factors: including initiating and maintaining partnership; facilitating cross-communication among partners; capturing and optimizing partners' perspective across phases of research; ensuring meaningful influence on research; providing training for partnering; and sharing and applying learnings (52).

The CBPR Framework is also based on trust, respect, mutual benefit, and equitable and shared decision-making principles similar to the PCOR framework (57, 58), and proposes two overarching domains that are relevant to PEC (53, 54). The contextual domain includes contextual factors that influence partnerships, including: social, economic, cultural, local/national governance, policies, and funding trends, role of institutions, historical context of trust/mistrust, both university and community partners' capacities, readiness, and experience in participatory research, and perceived severity of health issue. The group dynamic domain considers three areas of factors that influence PEC dynamics. These include: (a) structural dynamic (e.g., diversity, complexity, formal agreements, real power/resource sharing, alignment with CBPR principles); (b) individual dynamics (e.g., core values, motivations for participating); and (c) relational dynamics (e.g., safety, trust, flexibility in dialogue, listening and mutual learning, leadership influence, power dynamics, self, and collective reflection, participatory decision-making) (54). It is conceptualized that positive collaboration contexts and group dynamics will result in positive synergistic partnerships, appropriate interventions, and research, and improved systems and community capacity (53, 54).

The Stage Process Engagement Framework (proposed by NIH, CDC and other researchers) suggests that key factors for effective PEC depend on the stage of collaboration (51, 59–61). At the initial stage, PEC may focus on clarifying collaboration goals, promoting knowledge about the collaborators, and better understanding the strengths and weaknesses of the partnering contexts. For engagement to occur, it is necessary to visit communities to establish relationship and build trust, and subsequently work toward developing mutual beneficial goals. For engagement to succeed, sharing responsibility, recognizing and respecting the diversity of partners/communities, and creating transparency are necessary. For partnerships to be sustained, mobilizing community assets and strengths, developing the community capacity, resources, and social capital to facilitate creation of innovative strategies, releasing control of action to the community, and being flexible enough to meet changing needs are also needed (51, 59).

Other frameworks derived from team science literature can also be applied to study multi-disciplinary collaboration process. The Trust Framework proposes that successful PEC outcomes hinge largely on the most basic of human relationship “trust.” The nature of complex collaborative relationships is shaped and formed by three trust related factors: openness, transparency, and diversity. High levels of openness (in team social network) and transparency (related to information and knowledge sharing) will foster diversity in innovation. All three factors are required and need to be balanced for the eventual win-win-win success (62). The Team Science Concept Map proposes not only team related factors, but also support and meta factors that influence the performance of PEC need to be considered. Examples for the team factors may include disciplinary dynamic; structure and context for team; and characteristics and dynamics of teams. Examples for the support factors may include institutional support and professional development and management and organization for team. Examples for the meta factors may include definitions of team collaboration and models, measurement, monitoring, and evaluation (63).

PEC Strategies Testing

Some strategies focused on the sustainment/support systems level were identified from the literature on Quality Improvement Collaboration (QIC), and PCOR and CBPR patient/community research partnerships. In QIC research, several PEC strategies have been tested by researchers during QIC set-up and execution periods. Specifically, during QIC set-up, 7 key PEC strategies were commonly tested. These include pre-work-convened expert panel, pre-work-organizational commitment, in-person learning sessions, Plan-Do-Study-Act/ PDSAs cycles, multidisciplinary team, team calls, email, and/or web support). During the execution period, strategies such as monitoring data collection, reviewing data for feedback, and using external support for monitoring data synthesis and feedback have also been applied. At the organizational level, PEC strategies such as involving leadership and providing QIC training for staff members were applied (19). In PCOR research, PEC strategies developed from the PCOR engagement framework were also studied in PCORI funded projects (52). In CBPR research, training strategies based on CBPR framework have also been studied. The goals of these CBPR trainings are to build partners capacity, develop structured communication mechanisms to facilitate opportunities for discussion, develop partners' partnership skills, and capacity, and provide technical assistance on research related design (48).

PEC Evidence at the Sustainment/Support System Level

Research in this level of PEC strategy testing is limited and relies more on qualitative and short term data collection. A QIC review study (based on 24 RCTs or quasi-experimental studies) found some positive evidence for PEC strategies. In general, the impact of QIC tends to be greater for providers than for patients. At the provider level, about 47% QIC studies showed positive findings (42% mixed findings and 11% no findings) related to patient-centered cares, such as showing improvement on patient health screening/monitoring, use of data to inform interventions, and/or provider teamwork. At the patient-level, only 23% studies showed positive findings (46% mixed findings, and 31% no findings) related to an increase in patients' participation in care or reduction in health symptoms (19). Regardless, findings were not surprising because most QIC focused on provider related PEC strategies.

For PEC strategy testing based on the PCOR framework, findings from a recent study of 221 PCORI funded projects between 2012 and 2016 (based on self-report data from 235 investigators and 260 partners) provide some supporting evidence for the PCOR approach of collaboration. There were 11–52% investigators and partners endorsing improvements on patient-centeredness of study processes and outcomes (e.g., choices of research topics were driven by patients and related to their needs), and 20–81% investigators and partners endorsing improvement in study design, conduct, or/and efficiency (e.g., increasing the appropriateness of research question selection, design, and outcome measures) (52).

Other PEC strategy testing studies have been based on the CBPR framework and have used provider training and technical support to researchers and community partners to promote effective community-academia collaboration. Positive findings have been documented in several studies, especially related to achieving deliverables (e.g., written pilot study proposal, IRB approved study protocol, carried out pilot studies) (48, 64–66).

Discussion

The purpose of this paper is to describe a PEC framework and methodological gaps in D&I research by reviewing and summarizing findings from a broad range of PEC literature. A total of 39 articles (including 21 review articles) were selected for this review, representing findings from 1,456 research studies. Through this review, factors and theoretical processes that influence PEC, and strategies that promote effective PEC were identified. Findings guided the development of a multi-level PEC framework, which can be applied to strengthen the evidence-based for PEC research in the field of D&I.

In identifying factors for effective PEC, four domains were consistently identified across three levels of PEC. These included cognitive, interpersonal/affective, behavioral, and context preparation (see Table 1 and Figure 1 for summary) (28, 30, 37, 38). Furthermore, certain strategies were found to be more critical based on the partnership stage. In the earlier stages of PEC, cognitive, affective, and experiential behavioral change strategies were more important in order to build buy-in, awareness, mutual goal development, and trust to prepare for partnership (33, 37, 38, 51, 59). In regulating partnership performance, cognitive and behavioral strategies were more relevant, and in management of partnership maintenance (keeping the partnership going), relationship and affective strategies were more relevant (10). In inter-organizational types of partnerships, structural dynamics, and related strategies (e.g., alignment of collaboration goal with agency mission, resource sharing, leadership representation, and power) need to be carefully considered because of their potential influence on relational dynamics (e.g., integration of agency beliefs to team partnership process) (54).

Related to PEC mechanisms, several useful theoretical frameworks that include factors related to PEC processes were identified. For example, the relationship among context-mechanisms-outcome (or input → process → outcome) has been used to develop causative explanations about PEC processes. This approach allows process modeling wherein the outcome of one context-mechanism-outcome becomes the context for the next chain of implementation steps. Although this framework is useful to guide PEC process research, it may not be as useful for studying PEC mechanisms at the sustainment/support system level due to the complexity of behavioral dynamics within and across agencies during different partnership stages. Recently developed integrated frameworks of change, such as the stage of change (e.g., TTM), partnership stage of change (e.g., cooperation-coordination-collaboration; preparation-execution-adjustment) (33, 37, 38, 51, 59, 60), and multiple dynamic processes (e.g., cognitive, interpersonal relationship, behavioral dynamics) (37, 38, 53, 54) may generate more complex and explanatory theories to guide the design of PEC strategies in D&I research.

Leadership is a factor that is frequently studied in D&I contexts. In this review, we found that leadership's function varies based on stages or levels of partnership. At the delivery system level (or implementation team level), team leadership plays a role during the task execution period (to facilitate activity coordination) (40). At the sustainment-level of PEC, skilled/effective leadership is considered to be a collaboration foundation strategy, which plays a supporting role in the interdisciplinary collaboration process and facilitate initial institutional structure set-up to support PEC (15, 16, 51). In inter-organizational collaborations, leadership also plays an important role given the involvement of different partnering organizations (e.g., in negotiation of collaboration goals), and the importance of facilitating cross-communication among agency staff and external partners (5, 19). Therefore, leadership strategies should be considered when they are relevant to study design.

In PEC strategies testing, four approaches were found to be effective. Training (or psychoeducation) for consumers, providers/ implementers, and partnering members was consistently identified as an important PEC strategy across all level of partnerships. Training provides an opportunity for cognitive preparation and skill building to allow partners to communicate more effectively and to actively participate in partnership activities. Training can also target positive relationship building, partnership behavior engagement, and partnership sustainment (9, 26, 35, 41, 48). However, training alone does not change health behaviors. Training that incorporates multiple strategies and targets all partners is more likely to change behavior (9) and to improve patient health benefits (34, 35).

In addition, strategies that focus on assessment/ monitoring/ reflection (e.g., partner members' ability to assess barriers or collect monitoring data for feedback), participation (e.g., power sharing, involvement in decision-making), and relationship building (e.g., communication style, conflict management) are useful and could be included in training initiatives (9, 19, 26, 34, 35, 48).

Implications for D&I Research

Three main lessons for D&I research can be drawn from this review. First, researchers may want to consider gathering data about PEC contexts, associated factors, and processes at multiple levels as part of initial assessments of D&I contexts to enable examination of how PEC contexts and processes from each level of partnership contribute to service use, patient health, and sustainment outcomes [as defined by (25)].

Second, several research questions emerged. Most tests of PEC strategies have only evaluated short-term or intermediate outcomes. Limited evidence is available related to long-term effects. In addition, most sustainment/support PEC strategies have only been evaluated as case studies, qualitatively, or in non-experimental designs. Integrated conceptual frameworks have only recently been developed that could elucidate the complexities of analysis of sustainment strategies. Furthermore, the lack of measurement tools for construct assessment has been an impediment. As measurement tools become more refined and feasible, future studies can take advantage of these advances to study these types of strategies.

Third, PEC has not yet been integrated into D&I training. While partnerships are common, true power sharing is rare. Many D&I experts recognize the importance of including PEC strategies, but do not systematically incorporate them into training and evaluate impacts on PEC effectiveness. It would be useful to include training in specific skills related to PEC. These might include team science, community engagement, communication strategies, conflict management, and interpersonal and intrapersonal intelligence (67, 68).

Conclusion

As population level health issues continue to require complex healthcare policy solutions, it will become increasingly important to improve partnerships, engagement of different constituents, and new collaborations to craft cost-effective and creative solutions on a broad scale. This will entail at a minimum active involvement of patients, policy-makers, providers, community leaders, and researchers. This paper provides several new directions to address D&I knowledge and methodological gaps related to these partnerships. The review and framework not only provides guidance on how PEC related factors and outcomes can be conceptualized, but also how PEC processes can be integrated into more robust D&I designs. As has been reiterated, more research is needed to elucidate both cross and multi-level partnership mechanisms. In particular, systematic and long-term follow-up research will strengthen understanding of PEC strategies to advance EBI/EBPs implementation-effectiveness, sustainability, and system and population health outcomes.

Author Contributions

K-YH, SK, LB, DS, SAK, OO, and KH contributed conception and design of the study. K-YH, SK, and SC was involved in the acquisition, analysis, and interpretation of data. K-YH wrote the first draft of the manuscript. K-YH, SK, SC, DS, DK, and KH contributed to manuscript writing. All authors contributed to manuscript revision, read, and approved the submitted version.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

This research was supported by grant NIH/NCATS 1UL1TR001445 from the National Institutes of Health (NIH) and U19 MH110001-01 from the National Institute of Mental Health (NIMH). The reviews and opinions expressed in this article are solely those of the authors and do not necessarily represent the views of NIH. This review relied on publicly available documents and, therefore, was exempt from Institutional Review Board determination.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpubh.2018.00190/full#supplementary-material

References

1. Brown CH, Kellam SG, Kaupert S, Muthen BO, Wang W, Muthen LK, et al. Partnerships for the design, conduct, and analysis of effectiveness, and implementation research: experiences of the prevention science and methodology group. Adm Policy Ment Health (2012) 39:301–16. doi: 10.1007/s10488-011-0387-3

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Leeman J, Birken SA, Powell BJ, Rohweder C, Shea CM. Beyond “implementation strategies”: classifying the full range of strategies used in implementation science and practice. Implement. Sci. (2017) 12:125. doi: 10.1186/s13012-017-0657-x

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. (2015) 10:21. doi: 10.1186/s13012-015-0209-1

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Powell BJ, Proctor EK, Glass JE. A systematic review of strategies for implementing empirically supported mental health interventions. Res Soc Work Pract. (2014) 24:192–212. doi: 10.1177/1049731513505778

PubMed Abstract | CrossRef Full Text | Google Scholar

5. D'Amour D, Ferrada-Videla M, Rodriguez L, Beaulieu MD. The conceptual basis for interprofessional collaboration: core concepts and theoretical frameworks. J Interprof Care (2005) 19:116–31. doi: 10.1080/13561820500082529

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Apollo A, Golub SA, Wainberg ML, Indyk D. Patient-provider relationships, HIV, and adherence: requisites for a partnership. Soc Work Health Care (2006) 42:209-24. doi: 10.1300/J010v42n03_13

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Carcone AI, Ellis DA, Chen X, Naar S, Cunningham PB, Moltz K. Multisystemic therapy improves the patient-provider relationship in families of adolescents with poorly controlled insulin dependent diabetes. J Clin Psychol Med Set. (2015) 22:169–78. doi: 10.1007/s10880-015-9422-y

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Cunningham BJ, Rosenbaum PL. Measure of processes of care: a review of 20 years of research. Dev Med Child Neurol. (2014) 56:445–52. doi: 10.1111/dmcn.12347

PubMed Abstract | CrossRef Full Text | Google Scholar

9. McEwan D, Ruissen GR, Eys MA, Zumbo BD, Beauchamp MR. The effectiveness of teamwork training on teamwork behaviors and team performance: a systematic review and meta-analysis of controlled interventions. PLoS ONE (2017) 12:e0169604. doi: 10.1371/journal.pone.0169604

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Rousseau V, Aubé C, Savoie A. Teamwork behaviors: a review and an integration of frameworks. Small Group Res. (2006) 37:540–70. doi: 10.1177/1046496406293125

CrossRef Full Text | Google Scholar

11. Breslau ES, Weiss ES, Williams A, Burness A, Kepka D. The implementation road: engaging community partnerships in evidence-based cancer control interventions. Health Promot Pract. (2015) 16:46–54. doi: 10.1177/1524839914528705

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm Policy Ment Health (2011) 38:4–23. doi: 10.1007/s10488-010-0327-7

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Rabin BA, Brownson RC, Haire-Joshu D, Kreuter MW, Weaver NL. A glossary for dissemination and implementation research in health. J Public Health Manag Pract. (2008) 14:117–23. doi: 10.1097/01.PHH.0000311888.06252.bb

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Concannon TW, Meissner P, Grunbaum JA, McElwee N, Guise J-M, Santa J, et al. A new taxonomy for stakeholder engagement in patient-centered outcomes research. J Gen Int Med. (2012) 27:985–91. doi: 10.1007/s11606-012-2037-1

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Palinkas LA, Allred CA, Landsverk JA. Models of research-operational collaboration for behavioral health in space. Aviat Space Environ Med. (2005) 76(6 Suppl):B52–60.

PubMed Abstract | Google Scholar

16. Matteessich PW, Murray-Close M, Monsey BR, Wilder Research Center. Collaboration: What makes it work (Second addition). Saint Paul, MN: Fieldstone Alliance (2001).

17. CDC. Engaging, building, expanding: An NBCCEDP partnership development toolkit. Atlanta, GA: Centers for Disease Control and Prevention (2011).

18. Arksey H, O'Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. (2005) 8:19–32. doi: 10.1080/1364557032000119616

CrossRef Full Text | Google Scholar

19. Nadeem E, Olin SS, Hill LC, Hoagwood KE, Horwitz SM. Understanding the components of quality improvement collaboratives: a systematic literature review. Milbank Q (2013) 91:354–94. doi: 10.1111/milq.12016

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Hasnain-Wynia R, Beal AC. Role of the patient-centered outcomes research institute in addressing disparities and engaging patients in clinical research. Clin Ther. (2014) 36:619–23. doi: 10.1016/j.clinthera.2014.04.005

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Kidd SA, Kral MJ. Practicing participatory action research. J Counsel Psychol. (2005) 52:187–95. doi: 10.1037/0022-0167.52.2.187

CrossRef Full Text | Google Scholar

22. Drahota A, Meza RD, Brikho B, Naaf M, Estabillo JA, Gomez ED, et al. Community-academic partnerships: a systematic review of the state of the literature and recommendations for future research. Milbank Q (2016) 94:163–214. doi: 10.1111/1468-0009.12184

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Carman KL, Workman TA. Engaging patients and consumers in research evidence: Applying the conceptual model of patient and family engagement. Patient Educ Couns. (2017) 100:25–9. doi: 10.1016/j.pec.2016.07.009

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Duffett L. Patient engagement: what partnering with patient in research is all about. Thromb Res. (2017) 150:113–20. doi: 10.1016/j.thromres.2016.10.029

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Proctor EK, Landsverk J, Aarons G, Chambers D, Glisson C, Mittman B. Implementation research in mental health services: an emerging science with conceptual, methodological, and training challenges. Admin Policy Ment Health (2009) 36:24–34. doi: 10.1007/s10488-008-0197-4

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Becker KD, Lee BR, Daleiden EL, Lindsey M, Brandt NE, Chorpita BF. The common elements of engagement in children's mental health services: which elements for which outcomes? J Clin Child Adolesc Psychol. (2015) 44:30–43. doi: 10.1080/15374416.2013.814543

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Ogden K, Barr J, Greenfield D. Determining requirements for patient-centred care: a participatory concept mapping study. BMC Health Serv Res. (2017) 17:780. doi: 10.1186/s12913-017-2741-y

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Hill C, Knafl KA, Santacroce SJ. Family-centered care from the perspective of parents of children cared for in a pediatric intensive care unit: an integrative review. J Pediatr Nurs. (2017) doi: 10.1016/j.pedn.2017.11.007. [Epub ahead of print].

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Richards CA, Starks H, O'Connor MR, Doorenbos AZ. Elements of family-centered care in the pediatric intensive care unit: an integrative review. J Hospice Palliat Nurs. (2017) 19:238–46. doi: 10.1097/NJH.0000000000000335

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Higgins T, Larson E, Schnall R. Unraveling the meaning of patient engagement: a concept analysis. Patient Educ Couns. (2017) 100:30–6. doi: 10.1016/j.pec.2016.09.002

PubMed Abstract | CrossRef Full Text | Google Scholar

31. King SM, Rosenbaum PL, King GA. Parents' perceptions of caregiving: development and validation of a measure of processes. Dev Med Child Neurol. (1996) 38:757–72.

PubMed Abstract | Google Scholar

32. King S, King G, Rosenbaum P. Evaluating health service delivery to children with chronic conditions and their families: development of a refined Measure of Processes of Care (MPOC−20). Child Health Care (2004) 33:35–57. doi: 10.1207/s15326888chc3301_3

CrossRef Full Text | Google Scholar

33. Mauriello LM, Johnson SS, Prochaska JM. Meeting patients where they are at: using a stage approach to facilitate engagement. In: O'Donohue W, editor. Practical Strategies and Tools to Promote Treatment Engagement. (Cham: Springer International Publishing AG) (2017). p. 25–44.

Google Scholar

34. Lewin SA, Skea ZC, Entwistle V, Zwarenstein M, Dick J. Interventions for providers to promote a patient-centred approach in clinical consultations. Cochr Database Syst Rev. (2001):Cd003267. doi: 10.1002/14651858.Cd003267

CrossRef Full Text | Google Scholar

35. Dwamena F, Holmes-Rovner M, Gaulden CM, Jorgenson S, Sadigh G, Sikorskii A, et al. Interventions for providers to promote a patient-centred approach in clinical consultations. Cochr Database Syst Rev. (2012) 12:CD003267. doi: 10.1002/14651858.CD003267.pub2

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Lindsey MA, Brandt NE, Becker KD, Lee BR, Barth RP, Daleiden EL, et al. Identifying the common elements of treatment engagement interventions in children's mental health services. Clin Child Fam Psychol Rev. (2014) 17:283–98. doi: 10.1007/s10567-013-0163-x

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Kozlowski SWJ, Ilgen DR. Enhancing the effectiveness of work groups and teams. Psychol Sci Public Inter. (2006) 7:77–124. doi: 10.1111/j.1529-1006.2006.00030.x

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Kozlowski SWJ. Enhancing the effectiveness of work groups and teams: a reflection. Perspect Psychol Sci. (2017) 13:205–12. doi: 10.1177/1745691617697078

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Agreli HF, Peduzzi M, Bailey C. Contributions of team climate in the study of interprofessional collaboration: A conceptual analysis. J Interprof Care (2017) 31:679–84. doi: 10.1080/13561820.2017.1351425

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Brennan SE, Bosch M, Buchan H, Green SE. Measuring team factors thought to influence the success of quality improvement in primary care: a systematic review of instruments. Implement Sci. (2013) 8:20. doi: 10.1186/1748-5908-8-20

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Hall P, Brajtman S, Weaver L, Grassau PA, Varpio L. Learning collaborative teamwork: an argument for incorporating the humanities. J Interprof Care (2014) 28:519–25. doi: 10.3109/13561820.2014.915513

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Bronstein LR. A model for interdisciplinary collaboration. Soc Work (2003) 48:297–306. doi: 10.1093/sw/48.3.297

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Weaver TE. Enhancing multiple disciplinary teamwork. Nurs Outlook (2008) 56:108–14e2. doi: 10.1016/j.outlook.2008.03.013

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Mellin EA. Unpacking interdisciplinary collaboration in expanded school mental health: a conceptual model for developing the evidence base. Adv Sch Mental Health Promot. (2009) 2:4–14. doi: 10.1080/1754730x.2009.9715706

CrossRef Full Text | Google Scholar

45. Mellin EA, Bronstein L, Anderson-Butcher D, Amorose AJ, Ball A, Green J. Measuring interprofessional team collaboration in expanded school mental health: model refinement and scale development. J Interprof Care (2010) 24:514–23. doi: 10.3109/13561821003624622

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Cooke NJ, Hilton ML, Committee on the Science of Team Science, National Research Council. Enhancing the Effectiveness of Team Science. Washington, DC: National Academies Press (2015).

47. Matteessich PW, Murray-Close M, Monsey BR, Wilder Research Center. The Wilder Collaboration Factor Inventory. Saint Paul, MN: Fieldstone Alliance (2001).

48. Rodgers KC, Akintobi T, Thompson WW, Evans D, Escoffery C, Kegler MC. A model for strengthening collaborative research capacity: Illustrations from the Atlanta clinical translational science institute. Health Educ Behav. (2014) 41:267–74. doi: 10.1177/1090198113511815

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Brookman-Frazee L, Stahmer AC, Lewis K, Feder JD, Reed S. Building a research-community collaborative to improve community care for infants and toddlers at-risk for autism spectrum disorders. J Commun Psychol. (2012) 40:715–34. doi: 10.1002/jcop.21501

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Shippee ND, Domecq Garces JP, Prutsky, Lopez GJ, Wang Z, Elraiyah TA, Nabhan M, et al. Patient and service user engagement in research: a systematic review and synthesized framework. Health Expect. (2013) 18:1151–66. doi: 10.1111/hex.12090

PubMed Abstract | CrossRef Full Text | Google Scholar

51. NIH CDC ATSDR CTSA. Principles of Community Engagement 2nd Edn (2011). Available online at: https://www.atsdr.cdc.gov/communityengagement/pdf/PCE_Report_508_FINAL.pdf

52. Forsythe L, Heckert A, Margolis MK, Schrandt S, Frank L. Methods and impact of engagement in research, from theory to practice and back again: early findings from the Patient-Centered Outcomes Research Institute. Qual Life Res. (2017) 27:17–31. doi: 10.1007/s11136-017-1581-x

PubMed Abstract | CrossRef Full Text | Google Scholar

53. Wallerstein N, Oetzel J, Duran B, Belone L, Tafoya G, Rae R. CBPR: What predicts outcomes? In: Minkler M, Wallerstein N, editors. Community-Based Participary Research for Health: From Process to Outcomes. San Francisco, CA: Jossey-Bass (2008). p. 371–92.

Google Scholar

54. Belone L, Lucero JE, Duran B, Tafoya G, Baker EA, Chan D, et al. Community-based participatory research conceptual model: community partner consultation and face validity. Qual Health Res. (2016) 26:117–35. doi: 10.1177/1049732314557084

PubMed Abstract | CrossRef Full Text | Google Scholar

55. West M, Borrill CA, Unsworth KL. Team effectiveness in organizations. In: Cooper CL and Robertson I, editors. International Review of Industrial and Organizational Psychology. Vol 13. (Chichester: John Wiley) (1998). p. 1–48.

Google Scholar

56. Gitlin LN, Lyons KJ, Kolodner E. A model to build collaborative research or educational terms of health professionals in Gerontology. Educ Gerontol. (1994) 20:15–34.

Google Scholar

57. Israel BA, Eng E, Schulz AJ, Parker EA. Methods in Community-Based Participatory Research for Health. Satcher D, editor. San Francisco, CA: John Wiley & Sons, Inc. (2005).

58. Jagosh J, Bush PL, Salsberg J, Macaulay AC, Greenhalgh T, Wong G, et al. A realist evaluation of community-based participatory research: partnership synergy, trust building and related ripple effects. BMC Public Health (2015) 15:725. doi: 10.1186/s12889-015-1949-1

PubMed Abstract | CrossRef Full Text | Google Scholar

59. de-Graft Aikins A, Arhinful DK, Pitchforth E, Ogedegbe G, Allotey P, Agyemang C. Establishing and sustaining research partnerships in Africa: a case study of the UK-Africa Academic Partnership on Chronic Disease. Global Health (2012) 8:29. doi: 10.1186/1744-8603-8-29

CrossRef Full Text | Google Scholar

60. Goodman MS, Sanders Thompson VL. The science of stakeholder engagement in research: classification, implementation, and evaluation. Transl Behav Med. (2017) 7:486–91. doi: 10.1007/s13142-017-0495-z

PubMed Abstract | CrossRef Full Text | Google Scholar

61. Maselli, D, Lys J-A, Schmid J. Improving impacts of research partnerships. Swiss Commission for Research Partnerships with Developing Countries, KFPE. Berne: Geographica Bernensia (2004). Available online at: http://www.managingforimpact.org/sites/default/files/resource/improving_impacts_of_research_partnerships.pdf

62. Adams LG. Putting together a scientific team: collaborative science. Trends Microbiol. (2014) 22:483–5. doi: 10.1016/j.tim.2014.05.001

PubMed Abstract | CrossRef Full Text | Google Scholar

63. Falk-Krzesinski HJ, Borner K, Contractor N, Fiore SM, Hall KL, Keyton J, et al. Advancing the science of team science. Clin Transl Sci. (2010) 3:263–6. doi: 10.1111/j.1752-8062.2010.00223.x

PubMed Abstract | CrossRef Full Text | Google Scholar

64. Trinh-Shevrin C, Pollack HJ, Tsang T, Park J, Ramos MR, Islam N, et al. The Asian American hepatitis B program: building a coalition to address hepatitis B health disparities. Prog Community Health Partnersh. (2011) 5:261–71. doi: 10.1353/cpr.2011.0039

PubMed Abstract | CrossRef Full Text | Google Scholar

65. Kwon S, Rideout C, Tseng W, Islam N, Cook WK, Ro M, et al. Developing the community empowered research training program: building research capacity for community-initiated and community-driven research. Prog Community Health Partnersh. (2012) 6:43–52. doi: 10.1353/cpr.2012.0010

PubMed Abstract | CrossRef Full Text | Google Scholar

66. VanDevanter N, Kwon S, Sim SC, Chun K, Trinh-Shevrin C. Evaluation of community-academic partnership functioning: center for the elimination of hepatitis B health disparities. Prog Community Health Partnersh. (2011) 5:223–33. doi: 10.1353/cpr.2011.0032

PubMed Abstract | CrossRef Full Text | Google Scholar

67. Gonzales R, Handley MA, Ackerman S, O'Sullivan P S. A framework for training health professionals in implementation and dissemination science. Acad Med. (2012) 87:271–8. doi: 10.1097/ACM.0b013e3182449d33

PubMed Abstract | CrossRef Full Text | Google Scholar

68. Masuda JR, Creighton G, Nixon S, Frankish J. Building capacity for community-based participatory research for health disparities in Canada: the case of “Partnerships in Community Health Research”. Health Promot Pract. (2011) 12:280–92. doi: 10.1177/1524839909355520

CrossRef Full Text | Google Scholar