Toward a more comprehensive understanding of organizational influences on implementation: the organization theory for implementation science framework

Introduction: Implementation is influenced by factors beyond individual clinical settings. Nevertheless, implementation research often focuses on factors related to individual providers and practices, potentially due to limitations of available frameworks. Extant frameworks do not adequately capture the myriad organizational influences on implementation. Organization theories capture diverse organizational influences but remain underused in implementation science. To advance their use among implementation scientists, we distilled 70 constructs from nine organization theories identified in our previous work into theoretical domains in the Organization Theory for Implementation Science (OTIS) framework.

Methods: The process of distilling organization theory constructs into domains involved concept mapping and iterative consensus-building. First, we recruited organization and implementation scientists to participate in an online concept mapping exercise in which they sorted organization theory constructs into domains representing similar theoretical concepts. Multidimensional scaling and hierarchical cluster analyses were used to produce visual representations (clusters) of the relationships among constructs in concept maps. Second, to interpret concept maps, we engaged members of the Cancer Prevention and Control Research Network (CPCRN) OTIS workgroup in consensus-building discussions.

Results: Twenty-four experts participated in concept mapping. Based on resulting construct groupings' coherence, OTIS workgroup members selected the 10-cluster solution (from options of 7–13 clusters) and then reorganized clusters in consensus-building discussions to increase coherence. This process yielded six final OTIS domains: organizational characteristics (e.g., size; age); governance and operations (e.g., organizational and social subsystems); tasks and processes (e.g., technology cycles; excess capacity); knowledge and learning (e.g., tacit knowledge; sense making); characteristics of a population of organizations (e.g., isomorphism; selection pressure); and interorganizational relationships (e.g., dominance; interdependence).

Discussion: Organizational influences on implementation are poorly understood, in part due to the limitations of extant frameworks. To improve understanding of organizational influences on implementation, we distilled 70 constructs from nine organization theories into six domains. Applications of the OTIS framework will enhance understanding of organizational influences on implementation, promote theory-driven strategies for organizational change, improve understanding of mechanisms underlying relationships between OTIS constructs and implementation, and allow for framework refinement. Next steps include testing the OTIS framework in implementation research and adapting it for use among policymakers and practitioners.

Introduction

Individual healthcare providers' behaviors are often constrained by factors that are beyond their own control (1). The assumption that all behaviors are largely under conscious control has taken a “theoretical battering” due to research showing the importance of non-conscious processes that operate in organizations (2). Research suggests that many healthcare provider behaviors that are repeatedly performed become non-reflective and more or less automatic (3). Individual behavior is also constrained by factors at collective levels (1). Collective levels include interpersonal (e.g., relations between healthcare providers), group (e.g., healthcare professionals providing care in a breast medical oncology practice), intraorganizational (e.g., hospital culture), and interorganizational (e.g., accreditation standards). Collective-level influences may also be largely non-conscious, having become internalized and taken for granted (e.g., norms and values of a professional culture) (1).

Various implementation determinant frameworks include factors at the organizational level. For example, the Consolidated Framework for Implementation Research (CFIR) and the Exploration, Preparation, Implementation, Sustainment (EPIS) Framework include inner setting (i.e., intraorganizational) and outer setting (i.e., interorganizational) domain (4, 5). Domains are comprised of constructs (i.e., explanatory concepts that cannot be directly observed but can be inferred from observed data) (6). Organization-level domains include constructs such as structural characteristics (“the social architecture, age, maturity, and size of an organization”), cosmopolitanism (“the degree to which an organization is networked with other external organizations”), and funding (“fiscal support provided by the system in which implementation occurs”) (7–9). The Theoretical Domains Framework (TDF) similarly includes the environmental context and resources domain, which includes constructs such as material resources and barriers and facilitators (10).

Commonly used implementation determinant frameworks encourage implementation scientists to consider organizational influences on implementation; however, the scope of organization-level constructs described in extant frameworks is limited. Furthermore, determinant frameworks often lack explanations of the mechanisms underlying the relationships between organization-level constructs and implementation. Extant frameworks' limited scope impedes progress in implementation science by obscuring the influence of organization-level constructs that may drive implementation outcomes. A substantial body of work in industries other than healthcare provides evidence of the significant influence of organizational influences on implementation, pointing to high-leverage strategies to promote organizational change. Organization theory has been applied to educational and budgetary reform, elucidating the critical importance of addressing power dynamics among leadership and fostering positive change culture to facilitate implementation (11, 12). In the non-profit industry, organization theory can be used to build capacity, assist with decision-making, narrow target populations, and clarify organizational needs (13).

Failing to account for the critical influence of organization-level constructs on implementation introduces omitted variable bias–i.e., the faulty attribution of the influence of the omitted variable(s) to variables that were included (14). In the case of implementation research, this may amount to, for example, attributing the influence of organizational inertia (i.e., resistance to change) to a construct that is related but distinct (e.g., readiness for implementation) or an unrelated construct (e.g., individual provider motivation). The misattribution of omitted organization-level constructs to the constructs that extant implementation frameworks include has important implications for subsequent stages of implementation research, such as selecting and identifying strategies to target the constructs influencing implementation.

Many extant implementation determinant frameworks are conceptual frameworks, in that they offer a menu of constructs thought to influence implementation, but they do not address how change takes place or any causal mechanisms, which is critical for falsifying hypothesized relationships through empirical study (15). The ability to falsify hypothesized relationships between constructs depends on explanations of the mechanisms underlying relationships between constructs that are derived from theory (16). The constructs in conceptual frameworks such as the CFIR derive from a combination of theory and empirical studies. For example, the CFIR peer pressure construct derives from Institutional Theory, but patient needs and resources derives from a combination of empirical evidence and other conceptual frameworks rather than theory.

In contrast to conceptual frameworks, theoretical frameworks are based on theories, which propose mechanisms underlying the relationship between constructs and implementation. One commonly used theoretical framework in implementation science is the TDF. As a theoretical framework, the TDF can be used to identify mechanisms proposed in included theories; however, the TDF does not offer nuanced insight into organization-level influences on implementation. The TDF's environmental context and resources domains contains constructs that derive from several theories that are identified as organization theories; however, many of the included theories are not in fact organization theories (e.g., decision-making theory). As such, the TDF is limited in its contributions to understanding organization-level influences on implementation.

Organization theories provide explanations for the complex interactions within and between organizations and their context (environment, surrounding policies, cultural norms). These theories not only describe and explain these interactions, but can also be used to predict implementation outcomes based on contextual factors. Organization theories have the potential to explain how policies, institutions, funding, and workforce dynamics affect implementation outcomes (17). Organization theories have been historically used to an explanatory tool in fields of education, nonprofit organizations, management, and health services research, dating back to the 1950 s (11–13, 18, 19). These theories, while widely used and published, remain largely inaccessible outside of organization science. Organization theories provide their own inventory of constructs, which often require significant training to apply with fidelity.

To equip implementation scientists with understanding of a broader scope of organization-level constructs and their hypothesized influence on implementation, a comprehensive yet accessible framework of organizational influences on implementation is needed. In this paper, we describe the development of the Organization Theory for Implementation Science (OTIS) framework, which summarizes constructs from nine organization theories identified as relevant to implementation in preliminary studies (20). Our overarching goal is to increase implementation scientists' familiarity with and conceptualization of the myriad organizational factors that influence implementation through mechanisms clearly articulated by organization theories.

Materials and methods

We developed the OTIS framework using a combination of concept mapping and iterative consensus-building, with support for interpretation from members of the Cancer Prevention and Control Research Network (CPCRN) OTIS workgroup (21). CPCRN is a national network of academic, public health, and community partners whose work focuses on reducing the burden of cancer within specific workgroup and interest group projects. CPCRN OTIS workgroup members include investigators conducting research at the intersection of implementation science and cancer prevention and control. This study was approved by the Wake Forest University School of Medicine IRB (IRB00072134) on 6/2/21.

Concept mapping

Recruitment and sampling

We used a purposive sampling approach to recruit approximately 25 scholars with expertise at the intersection of implementation and organization science to participate in an online concept mapping exercise via the Concept Systems Global MAX™ web platform (22). The premise of our sampling approach for the survey on organization theories of relevance to implementation science that provided the foundation for this study was that scholars with primary training in implementation and organization science had the knowledge required to identify organization theories with relevance to implementation science. For this study, we purposively included a more diverse group of scholars with implementation and organization expertise with the objective of generating a framework that would reflect the perspective of targeted users of the OTIS framework. Between 20 and 30 sorters have been found to maximize concept mapping fit consistency, yielding results similar to concept mapping by several hundred participants (23). Members of the study team identified potential participants from their respective professional networks in Canada, the UK, and the USA, as well as professional organizations such as the VA QUERI Implementation Research Group. We sent up to three emails offering potential participants a $50 incentive to engage in the concept mapping exercise.

Procedure

To identify conceptually distinct categories (domains) of constructs, we asked participants to sort virtual cards for each of the 70 constructs from nine organization theories relevant to implementation identified in previous work (20), accompanied by their definitions, into piles as they deemed appropriate. We then asked participants to name each pile. Participants could engage in the activities in the order of their choosing and could do so over multiple online sessions, at their convenience, until their responses were complete.

Analysis

Data analysis involved the use of multidimensional scaling and hierarchical cluster analyses to produce visual representations of the relationships among the constructs (23). Specifically, multidimensional scaling was used to generate a point map depicting each of the constructs and the relationships between them based upon a summed square similarity matrix. Constructs frequently sorted together were placed closer together on the point map (23). Hierarchical cluster analysis was used to partition the point map into non-overlapping clusters (i.e., domains) (23). The Concept Systems Global Max™ suggested potential cluster labels based upon participant responses. Model fit was assessed using the stress value, an indicator of goodness of fit between the point map and the total similarity matrix. Cross-study syntheses of concept mapping studies have consistently found mean stress values of 0.28 (24). The stress value of the concept map represents goodness of fit of the configuration, demonstrating how close the solution is to the original groupings made by the participants. Lower stress values indicate a better fit than higher stress values (24).

Consensus-building

Recruitment and sampling

We invited members of the CPCRN OTIS workgroup to review concept mapping results and provide feedback. All CPCRN OTIS workgroup members were eligible to participate.

Procedure

To build upon the results of the concept mapping activity, CPCRN OTIS workgroup members provided their expertise in reviewing results of the concept mapping activity. Participation occurred over the course of three months, beginning with the CPCRN Annual Meeting and continuing through regular workgroup meetings.

Analysis

During a hybrid meeting held in May 2022, CPCRN OTIS workgroup members (6 in-person; 4 virtual) considered a range of potential cluster solutions, ranging from seven to 10 clusters, to determine which solution best suited the purposes of the current study. Each member identified the cluster map that they deemed most conceptually clear based on their knowledge of the field. The group then discussed their choices and worked to reach consensus on what the group believed to provide the most conceptually clear map and moved constructs to clusters that provided the best fit. The group also discussed and altered the automatically generated labels created by Global Max™. Following the initial analysis, two workgroup members reviewed notes, and a third member reconciled discrepancies, suggesting additional shifts of constructs among clusters. Finally, the lead investigator revised clusters based on extensive knowledge of organization theory. The resulting clusters were again reviewed, revised, and approved by CPCRN OTIS workgroup members during workgroup meetings until a consensus was reached.

Results

Concept mapping

Twenty-four scholars participated in the concept mapping exercise. Participant demographics are described in Table 1. Most participants (84%) held a PhD degree and worked in an academic institution (80%). The plurality of participants had multidisciplinary training (29%), and the majority had multidisciplinary expertise (57%).

TABLE 1

Table 1. Concept mapping participant demographics.

All 24 participants completed the sorting exercise. We confirmed that sorts were valid by checking 5 participants' responses to ensure that criteria were sorted into generally logical categories. The stress value was 0.32, demonstrating poor fit. Our consensus-building process was designed to address poor fit by developing a more coherent solution.

Consensus-building

Workgroup members (n = 18) participated throughout in-person and virtual discussion sessions. Participant demographics are described in Table 2. The concept mapping software produced multiple cluster options, ranging from 7 to 13 clusters. CPCRN OTIS workgroup members narrowed the clusters to 8–10 (Figures 1–3), ultimately selecting the 10-cluster solution to use as a starting point for the consensus-building process. Workgroup members then reorganized the clusters to increase coherence, yielding six final OTIS framework domains: organizational characteristics; governance and operations; characteristics of a population of organizations; tasks and processes; knowledge and learning; and interorganizational relationships. The final solution was informed by the 10-cluster solution. A total of 70 constructs are organized across the six domains. Supplementary File S1 organizes constructs by domain and includes brief descriptions/definitions for each, as well as the source theory.

TABLE 2

Table 2. Consensus gathering participant demographics.

FIGURE 1

Figure 1. Eight-cluster concept map solution.

FIGURE 2

Figure 2. Nine-cluster concept map solution.

FIGURE 3

Figure 3. Ten-cluster concept map solution and constructs.

Organizational Characteristics (number of constructs = 6) refers to the features of an organization that may predispose it to governance, operations, interorganizational relationships, etc. Included constructs relate to change dynamics (e.g., inertia; adaptability), orientation to operations (e.g., professionalization; specialization); and dominance within its population [e.g., age; size (i.e., indicators of viability; on average, older, larger organizations are more likely to survive than younger, smaller organizations)].

Governance and Operations (n = 7) refer to the rules and operating procedures that govern an organization. An organization's rules and operating procedures may be established explicitly (e.g., intentionally, by a governing body) or implicitly (e.g., passively, through repeated operations). Constructs include approaches to operating (e.g., governance structure; internal arrangements) and structures that characterize an organization's operations (e.g., internal arrangements; feedback loops).

Characteristics of a Population of Organizations (n = 16) refer to the features of a group of organizations of which the referent organization is a member (25). The institutions that comprise an organization's population may vary depending on the objective or problem in question. That is, a referent organization may be part of several populations. For example, a hospital's population may be defined as local healthcare organizations with respect to competition for physicians and patients, but with respect to adherence to government regulations, a hospital's population may be defined as all of the country's hospitals. Constructs included in the Characteristics of a Population of Organizations domain are features of the population as a whole rather than features of the organizations that comprise the population. Constructs relate to change within the population (e.g., dynamism; stability); competition (e.g., competition; selection pressure); variation within the population (e.g., isomorphism; spatial variation); and availability of resources (e.g., munificence; constraint).

Tasks and Processes (n = 16) characterize the work that an organization pursues and the conditions that influence its approach to accomplishing the work. Included constructs refer to features of the processes used to accomplish tasks (e.g., un/programmed coordination task structure; transaction costs); features of the environment in which tasks are accomplished (e.g., dependence; excess capacity); and features of the task (e.g., frequency of transactions; technology cycles).

Knowledge and Learning (n = 5) refers to the information available to an organization in pursuing its goals and the processes used to acquire the information. Included constructs relate to characteristics of knowledge (e.g., tacit and implicit knowledge) and approaches to acquiring knowledge (e.g., learning (sub)processes; sense making).

Interorganizational Relationships (n = 20) refer to characteristics of the interactions that an organization has with other institutions. In contrast to the Characteristics of a Population of Organizations domain, which refers to features of a population of organizations as a whole, the Interorganizational Relationships domain characterizes communal (e.g., communication) or exchange (e.g., monetary or other resource exchange) interactions (26). Included constructs characterize an organization's dependence on other institutions (e.g., interdependence; community interdependence); the pressure that organizations exert on each other (e.g., normative, mimetic, and coercive pressure; dominance; power).

Discussion

This study describes how we created the OTIS framework to increase implementation scientists' familiarity with and conceptualization of the diverse set of organizational influences on implementation. Increasing implementation scientists' conceptualization of organizational influences may contribute to more comprehensive understanding of the key drivers of implementation and, in turn, our ability to identify and select strategies to accelerate the translation of evidence into practice, as found in other industries, such as business and education (11, 12, 17, 18). Our efforts yielded six conceptually distinct domains, encompassing 70 constructs from nine organization theories with relevance to implementation. Distilling many constructs from several theories into a limited number of domains limits the burden on implementation scientists to account for the vast array of potentially important organizational influences on implementation. The six domains that we identified in this study reflect concepts that are central to organization theory, including power, structure, autonomy, control (20), but which are less commonly addressed in implementation science. The concepts reflected in the OTIS framework offer perspective on key questions in implementation science, such as how and why organizations adopt, implement, and sustain evidence-based practices—or resist doing so.

The OTIS framework considerably expands upon existing implementation determinant frameworks' conceptualization of organizational influences on implementation. OTIS includes constructs such as specialization, which is not explicitly captured in the CFIR or EPIS frameworks, but which may influence the decision to adopt an evidence-based practice. For example, a study of determinants of low-value use of computed tomography to evaluate microscopic hematuria found that, while urologists' evaluation practices changed following the American Urological Association's revised guidelines, primary care providers' evaluation practices often went unchanged, highlighting the need to tailor strategies for the various specialties involved in implementation (27, 28). OTIS greatly expands upon the TDF's organization-level environmental context and resources domain with more nuanced domains, such as interorganizational relationships. Future efforts should systematically map OTIS onto extant determinant frameworks to clearly articulate OTIS's unique contribution. For example, OTIS's Governance and Operations and Tasks and Processes domains include several constructs that may add critical nuance to EPIS's Funding/Contracting construct. Before systematic mapping of OTIS onto extant frameworks, OTIS may be used in its current form in conjunction with other frameworks, such as the CFIR and TDF, which are increasingly used in combination and already capture intra-organizational constructs, such as climate and leadership (28). For example, some OTIS domains or constructs that appear not to be captured in CFIR (e.g., tasks and processes; stability of the population of organizations; normative pressures) could be included in implementation determinant studies.

OTIS also expands upon commonly used implementation frameworks by allowing users to access organization theories articulate the mechanisms underlying relationships between included constructs and implementation. For example, the EPIS framework identifies sociopolitical influences on implementation (e.g., legislation; monitoring and review); however, EPIS does not articulate how or why these constructs influence implementation. In contrast, OTIS's basis in theory allows users to identify hypothesized relationships between included constructs and implementation, as clearly articulated in publicly available OTIS abstraction forms (28). Specifically, users may consult the propositions section of OTIS abstraction forms to identify mechanisms underlying included constructs. For example, OTIS describes how coercive influences of governments and accrediting bodies exert normative pressure (Interorganizational Relationships domain) on healthcare organizations to comply with legislation and monitoring by virtue of organizations' dependence on these governing bodies for permission to operate. Therefore, OTIS could be used in conjunction with extant frameworks to explain the mechanisms underlying constructs' influence on implementation (29). Clearly articulated mechanisms are critical for identifying strategies that are best-suited to influence the construct identified as influencing implementation.

Members of the CPCRN OTIS workgroup are currently applying the OTIS framework in the following projects: Project 1 a) tests the conceptual validity and applicability of the OTIS framework in community oncology practices and b) develops, tests, and disseminates tools using OTIS in implementation research, including a qualitative interview guide and codebook. Project 2 is an American Society of Clinical Oncology collaborative study on Sexual Orientation and Gender Identity data collection. OTIS will be used in this project to a) reanalyze data that have been previously analyzed using the CFIR, and b) compare results between CFIR and OTIS findings. Project 3 applies OTIS to a CDC-funded U01 cooperative agreement to reduce health inequities for cancer survivors in the District of Columbia. OTIS will be used to a) to build community coalitions of approximately 10 organizations to improve infrastructure and communication and b) to think consider power dynamics and the elimination of disparities and health inequalities. Some limitations of our study should be noted. Concept mapping requires participants to have pre-existing knowledge and experience with the topic they are mapping, limiting the pool of potential participants. There is a limited population of researchers with the required familiarity of organization theories and implementation science to participate in concept mapping. As a result, our purposive sampling approach was necessary to increase the likelihood that participants would understand included constructs enough to sort and rate them. However, it is possible that participants lacking refined expertise in implementation or organization science would have valuable perspective on included constructs. For example, hospital administrators may lack fluency in the terminology included in organization theories, but they may have unique insight into how, for example, normative pressure from professional organizations influences implementation. Future work should refine the language used in OTIS to increase its accessibility to an audience without expertise in organization science. Additionally, the clusters that Global Max generated, in many cases, lacked coherence as indicated by the stress value of 0.32, suggesting variation in concept mapping participants' interpretation of the constructs and their relationships. To address this concern, OTIS workgroup members used their expertise to reorganize many clusters in our consensus-building process, potentially suggesting the limited utility of concept mapping for developing the framework. We view the OTIS framework as a living document to be revised through application. For example, implementation scientists may find through qualitative interview data collection that study participants describe OTIS constructs in combinations not reflected in the domains identified in this study. Future iterations of the OTIS framework will be revised to reflect empirical evidence.

Despite these limitations and the need for continued development, OTIS may be used in its current form in implementation research. OTIS could be used to inform data collection or analysis. For example, OTIS could be used to develop guides for interviews with cancer program leadership to understand the potential influence of participation in quality improvement networks, professional norms, and the ability to recruit providers influence compliance with cancer program accreditation standards (30). We plan to use OTIS to analyze data that were previously collected regarding factors influencing cancer programs' implementation of exercise interventions. In each of these cases, OTIS offers researchers the tools necessary to understand the mechanisms underlying factors that influence implementation, pointing toward strategies to facilitate implementation (e.g., strengthening or reorganizing quality improvement networks to support compliance with accreditation standards).

Conclusions

We distilled 70 constructs from nine organization theories into six domains in the OTIS framework. The OTIS framework has several potential benefits. First, OTIS may enhance implementation scientists' consideration of organization-level constructs, which to date has been insufficient (17). Second, OTIS adds nuance to relatively limited conceptualizations of organizational influences in extant implementation determinant frameworks, such as the CFIR, EPIS, and TDF. Third, OTIS may increase the use of theories in implementation science. Evidence suggests that the use of theories, models, and frameworks in implementation science is inconsistent and often inappropriate (31). Unlike conceptual frameworks, which offer a menu of constructs thought to influence implementation, theoretical frameworks including OTIS are based on theories, which propose mechanisms underlying the relationship between constructs and implementation. OTIS links implementation scientists to theories that may explain the phenomena underlying complex implementation problems, such as slow uptake or poor sustainment. Future efforts should include expanding extant frameworks with OTIS's unique domains and constructs; refining OTIS's language to increase its accessibility to an audience without expertise in organization science; and revising OTIS to reflect empirical evidence.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation by reasonable request to the corresponding author.

Ethics statement

This study was approved by the Wake Forest University School of Medicine IRB (IRB00072134) on 6/2/21. Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.

Acknowledgments

We would like to thank all of those who took part in the concept mapping activities. Additionally, we would like to thank Jennifer Leeman and the members of the CPCRN OTIS workgroup for their contributions.

Some of the data analyzed during this study have been previously published and are available from (20).

Author contributions

SB: contributed to the study conception and design, drafted the manuscript, and participated in the data collection, analysis, and interpretation of results; CW and AP: drafted the manuscript and participated in the data collection, analysis, and interpretation of results; MK, JB, PA, MF, MM, TH, ML, MW, PN, MB, MC-B, GR, and AR: participated in the analysis and interpretation of concept mapping results; LK contributed to the study conception and design and participated in the data collection, analysis, and interpretation of results. All authors contributed to the article and approved the submitted version.

Funding

This publication was supported by the Centers for Disease Control and Prevention of the U.S. Department of Health and Human Services (HHS) as part of a financial assistance award with 100 percent funded by CDC/HHS (Cooperative Agreement Number U48 DP006400). The contents are those of the author(s) and do not necessarily represent the official views of, nor an endorsement, by CDC/HHS, or the U.S. Government.

Conflict of interest

The authors PN, TH, MEF, and PA declared that they were an editorial board member of Frontiers, at the time of submission. This had no impact on the peer review process and the final decision.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/frhs.2023.1142598/full#supplementary-material

Abbreviations

CFIR, consolidated framework for implementation research; CPCRN, cancer prevention and control research network; EPIS, exploration, preparation, implementation, sustainment; OTIS, organization theory for implementation science; TDF, theoretical domains framework.

References

1. Nilsen P, Potthoff S, Birken SA. Conceptualising four categories of behaviours: implications for implementation strategies to achieve behaviour change. Frontiers in Health Services. (2022) 1:795144. doi: 10.3389/frhs.2021.795144

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Greenwald AG, Krieger LH. Implicit bias: scientific foundations. Calif Law Rev. (2006) 94(4):945–67. doi: 10.2307/20439056

CrossRef Full Text | Google Scholar

3. Potthoff S, Kwasnicka D, Avery L, Finch T, Gardner B, Hankonen N, et al. Changing healthcare professionals’ non-reflective processes to improve the quality of care. Soc Sci Med. (2022) 298:114840. doi: 10.1016/j.socscimed.2022.114840

PubMed Abstract | CrossRef Full Text | Google Scholar

4. EPIS FRAMEWORK. Available at: https://episframework.com/ (Accessed January 24, 2023).

5. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. (2009) 4(1):1–15. doi: 10.1186/1748-5908-4-50

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Association AP. APA Dictionary of Psychology - Theoretical Construct (2022). Available at: https://dictionary.apa.org/theoretical-construct (Accessed January 24, 2023).

7. Smith LR, Ashok M, Dy SM, Wines RC, Teixeira-Poit S. Contextual frameworks for research on the implementation of complex system interventions (2014).

8. Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. (2004) 82(4):581–629. doi: 10.1111/j.0887-378X.2004.00325.x

PubMed Abstract | CrossRef Full Text | Google Scholar

9. EPIS Framework. Outer context. Available at: https://episframework.com/service-environment-1#:˜:text=Funding,system%20in%20which%20implementation%20occurs (Accessed January 24, 2023).

10. Atkins L, Francis J, Islam R, O’Connor D, Patey A, Ivers N, et al. A guide to using the theoretical domains framework of behaviour change to investigate implementation problems. Implement Sci. (2017) 12(1):1–18. doi: 10.1186/s13012-017-0605-9

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Bonner M, Koch T, Langmeyer D. Organizational theory applied to school reform: a critical analysis. Sch Psychol Int. (2004) 25(4):455–71. doi: 10.1177/0143034304048779

CrossRef Full Text | Google Scholar

12. Forrester JP, Adams GB. Budgetary reform through organizational learning: toward an organizational theory of budgeting. Adm Soc. (1997) 28(4):466–88. doi: 10.1177/009539979702800403

CrossRef Full Text | Google Scholar

13. Hunter DE. Using a theory of change approach to build organizational strength, capacity and sustainability with not-for-profit organizations in the human services sector. Eval Program Plann. (2006) 29(2):193–200. doi: 10.1016/j.evalprogplan.2005.10.003

CrossRef Full Text | Google Scholar

14. Trochim W, Spiegelman C. The relative assignment variable approach to selection bias in pretest-posttest group designs. Proceedings of the Survey Research Section. (1980) 1:376–80.

Google Scholar

15. Nilsen P. Making sense of implementation theories, models, and frameworks. Implement Sci. (2020) 30:53–79. doi: 10.1007/978-3-030-03874-8_3

CrossRef Full Text | Google Scholar

16. Schafersman SD. An introduction to science: Scientific thinking and the scientific method. Online lecture note) (1997). Available at: https://gehrcke.de/2014/06/steven-d-schafersmans-introduction-to-science/ (Accessed January 24, 2023).

17. Birken SA, Bunger AC, Powell BJ, Turner K, Clary AS, Klaman SL, et al. Organizational theory for dissemination and implementation research. Implement Sci. (2017) 12(1):1–15. doi: 10.1186/s13012-017-0592-x

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Payne J, Leiter J. Structuring agency: examining healthcare management in the USA and Australia using organizational theory. J Health Organ Manag. (2013) 27(1). doi: 10.1108/14777261311311825

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Daft RL. Organizational theory & design. Boston, MA: Cengage Learning (2016).

20. Birken SA, Ko LK, Wangen M, Wagi CR, Bender M, Nilsen P, et al. Increasing access to organization theories for implementation science. Front Health Serv. (2022) 2:1–13. doi: 10.3389/frhs.2022.891507

CrossRef Full Text | Google Scholar

21. Network CPaCR. (2020). Available at: https://cpcrn.org/ (Accessed January 24, 2023).

22. Concept Systems, Incorporated. 2022.273.21 B. The Concept System® Global MAX™ Ithaca, NY. (2021). Available at: http://www.conceptsystemsglobal.com (Accessed January 24, 2023).

23. Rosas SR, Kane M. Quality and rigor of the concept mapping methodology: a pooled study analysis. Eval Program Plann. (2012) 35(2):236–45. doi: 10.1016/j.evalprogplan.2011.10.003

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Péladeau N, Dagenais C, Ridde V. Concept mapping internal validity: a case of misconceived mapping? Eval Program Plann. (2017) 62:56–63. doi: 10.1016/j.evalprogplan.2017.02.005

CrossRef Full Text | Google Scholar

25. Hannan MT, Freeman J. The population ecology of organizations. Am J Sociol. (1977) 82(5):929–64. doi: 10.1086/226424

CrossRef Full Text | Google Scholar

26. Giacalone RA, Jurkiewicz CL. Handbook of workplace spirituality and organizational performance. New York, NY: M.E. Sharpe (2003).

27. Barocas DA, Boorjian SA, Alvarez RD, Downs TM, Gross CP, Hamilton BD, et al. Microhematuria: Aua/sufu guideline. J Urol. (2020) 204(4):778–86. doi: 10.1097/JU.0000000000001297

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Birken SA, Powell BJ, Presseau J, Kirk MA, Lorencatto F, Gould NJ, et al. Combined use of the consolidated framework for implementation research (CFIR) and the theoretical domains framework (TDF): a systematic review. Implement Sci. (2017) 12(1):1–14. doi: 10.1186/s13012-016-0534-z

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Sales AE, Barnaby DP, Rentes VC. Letter to the editor on “the implementation research logic model: a method for planning, executing, reporting, and synthesizing implementation projects”(Smith JD, Li DH, Rafferty MR. The implementation research logic model: a method for planning, executing, reporting, and synthesizing implementation projects. Implement Sci. 2020; 15 (1): 84. Implement Sci. (2021) 16(1):1–3. doi: 10.1186/s13012-021-01169-1

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Wahlen MM, Schroeder MC, Johnson EC, Lizarraga IM, Engelbart JM, Tatman DJ, et al. Identifying core functions of an evidence-based intervention to improve cancer care quality in rural hospitals. Front Health Serv. (2022) 2:891574. -. doi: 10.3389/frhs.2022.891574

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Moore GF, Evans RE. What theory, for whom and in which context? Reflections on the application of theory in the development and evaluation of complex population health interventions. SSM Popul Health. (2017) 3:132–5. doi: 10.1016/j.ssmph.2016.12.005

PubMed Abstract | CrossRef Full Text | Google Scholar