James W. Dearing- Michigan State University, East Lansing, MI, United States
The ability of non-governmental organizations, government agencies, and corporations to deliver and support the availability and use of interventions for improved global public health depends on their readiness to do so. Yet readiness has proven to be a rather fluid concept in global public health, perhaps due to its multidimensional nature and because scholars and practitioners have applied the concept at different levels such as the individual, organization, and community. This review concerns 30 publically available tools created for the purpose of organizational readiness assessment in order to carry out global public health objectives. Results suggest that these tools assess organizational capacity in the absence of measuring organizational motivation, thus overlooking a key aspect of organizational readiness. Moreover, the tools reviewed are mostly untested by their developers to establish whether the tools do, in fact, measure capacity. These results suggest opportunities for implementation science researchers.
Despite a common emphasis on the development of effective global health interventions, the greatest contemporary challenges in improving the health of populations rest with the delivery and utilization of interventions (1). Delivery relies on many human factors—such as communication, coordination, training, leadership and management, logistics, transportation, storage, and community outreach and behavioral campaigns—that function both independently of and interdependently with technical systems. Delivery is necessarily reliant on systems, comprised of different types of organizations, their histories, and current ways of working together. Strengthening these systems and the organizations that comprise them represents a global health priority (2). In some topical areas such as maternal, newborn, and child health, the availability of effective and simple interventions has shifted the challenge of achieving impact at scale from the development of new interventions to the delivery and uptake of these evidence-based interventions (3, 4).
Delivery is achieved through systems which commonly function as partnerships between governments (e.g., ministries of health), the non-profit sector, and private industry. Especially when pursued at large scale, delivery demands a degree of readiness to implement interventions, which reflects both organizational abilities and a desire to affect change (5).
In this article, I review the state of applied tools for assessing organizational readiness for global health intervention, and suggest how they might be improved through research and evaluation.
Readiness Means Motivation Combined with Capacity
The term readiness has often meant a psychological state (if measured at organizational and community levels, this represents a shared “state”) of commitment to a particular course of action (6). For example, individual readiness can refer to a person’s resolve to stop smoking, organizational readiness can be a shared belief by hospital staff that hospital acquired infections are unacceptably frequent, and community readiness may be represented by the degree to which community leaders are supportive of an effort to share patient health record data across competing health clinics. For global health work in less-developed countries, assessing the degree of motivation of organizations that are candidates to deliver or implement interventions such as bed nets to prevent malaria, biomedical interventions such as pre-exposure prophylaxis for HIV prevention, or inexpensive and clean burning cook stoves is important since many NGOs, aid organizations, and government agencies work on multiple challenges at once and sometimes relegate some interventions to a low priority. So success requires the presence of an important attitudinal component; organizations need to be appropriately motivated or willing for the organization to engage in a particular intervention (7–9).
In addition to motivated organizations, successful global health intervention in less-developed countries requires those organizations to have the skills, training, and resources to do a good job. Capacity is the ability to carry out stated objectives (10). The concept includes both the ability to produce an output, such as a functional community health outreach worker program, and the effectiveness of those outputs to produce desired health outcomes. Outcomes represent performance: How well do an organization’s activities induce the desired effect on outcomes such as individual behaviors and community or population health?
The concept of capacity has proven rather fluid in global health, perhaps due to its multidimensional nature and because scholars and practitioners have applied the concept at different levels; for example, capacity has been used to describe individual, team, organizational, and community abilities. At any level of analysis, capacity is also subject to exogenous inputs such as policy decisions and funding availability (11). As a result, the extent to which organizational capacity and outputs are responsible for observed outcomes is often difficult to accurately determine. This is one reason why investments in organizational capacity building (or when raised a level, system strengthening) can be controversial (12).
So organizational readiness assessment can be performed to (1) learn about the degree of motivation within a candidate organization for delivering and implementing a global health intervention, (2) assess the particular abilities within organizations, (3) help improve one or more organizational capacities, or (4) empower organizations to bring more value to their clients. Each objective can be useful. For example, a funder may want to compare which of a set of non-governmental organizations is best suited to deliver mosquito nets, conduct radio campaigns about them, and train community outreach workers in their correct use, with no intent to affect organizational capacities. Or an organizational leader may want to understand her organization’s capacities in order to set improvement or budgetary priorities. Or a policy maker may realize that a grassroots community-based organization has yet to fully develop its technical skills, but has strong and authentic access into those communities; thus, assessment can be used to help organizations from marginalized populations to better understand health risks and deliberate over alternative solutions, as well as help those organizations to work effectively with and for the community stakeholders they represent (13).
This review presents and analyzes readiness assessment tools that purportedly measure organizational readiness. We began with a systematic search of published and unpublished (gray) literature for tools (including decision aids and instruments such as questionnaires) that were designed to provide information about the capacity and motivation of organizations involved in global health.
Inclusion Criteria and Methods
For this review, publically available resources must have had:
• Addressed organizational capacity and/or motivation (tools that assessed general organizational assessment or performance were not included)
• Been relevant to the objectives of global health interventions (resources primarily assessing or addressing system- or individual-level factors were not included)
• Been created for organizations working in global health or international development in low-resource countries
• Contained a tool, instrument, or decision aid that:
⚬ Facilitates decision-making
⚬ Provides recommended measures or operationalized frameworks/questions to assess capacity and/or motivation
⚬ Enables quantitative or qualitative measurement of factors
We defined factors as attributes thought to contribute to organizational motivation and capacity to perform a service or function. A tool, instrument, or decision-aid was defined as a published collection of measures or factors meant to assist individuals in assessing the capacity or motivation of an organization.
Resources were identified using a systematic search of computerized databases (OVID, Web of Science, Academic Search Premier, CSA Sociological Abstracts), in-depth web-based searches, and bibliographic snowballing and back referencing. These search strategies were supplemented with targeted searches of relevant organizations, including World Bank Institute (WBI), the United Nations Development Programme (UNDP), John Snow International (JSI), United States Agency for International Development (USAID), and the UK Department for International Development (DFID), among others.
Our search identified 141 potentially relevant tools. An initial review was conducted by at least two members of the research team to assess each tool against the inclusion criteria. This process was done by reviewing the web sites for each tool and then the instructions and items specific to each tool. Each tool was assessed against each inclusion criterion. Tools were independently reviewed by two trained coders and coded for a range of variables. Differences of opinion were resolved through discussion and, in instances of continued disagreement, by the project manager who had trained the coders.
Results
Thirty tools met the inclusion criteria. These 30 tools are included in this analysis (see Table 1).
Table 1. Organization capacity assessment tools (N = 30).
The 30 tools are of different types (decision trees, questionnaires, checklists, matrices, etc.) and formats (paper, mobile, web-based, etc.) as listed in Box 1. Tools had a mean number of 60 questions or items grouped into a mean number of 13 factors.
Box 1. Types and formats of tools.
All tools addressed capacity; none addressed motivation. Each tool was coded for all factors it addressed and/or measured. Our team inductively developed a composite matrix of the capacity factors represented in the tools. Coders had been trained to familiarize themselves with barriers to or facilitators for the scale up of global health interventions in low-income countries (14–18). We then conducted an iterative analysis to identify those domains and factors most commonly addressed in the 30 tools. We grouped the factors into five domains associated with organizational capacity: (1) External Environment; (2) Organizational Attributes; (3) Management and Governance Capacity; (4) Collaboration; and (5) Organizational Performance. Each domain contains between two and eight factors (see Table 2). Analysis was completed based on totals by domain and factor, and by tool.
Table 2. Categorization of tool domains and factors.
The tools reviewed here assessed capacities of organizations by allowing users to enter qualitative and/or quantitative data, derived from expert judgment, interviews with staff and stakeholders, document review, workshops, and observation. Fourteen tools allowed both qualitative and quantitative data, 11 allowed only quantitative input, generally in the form of ordinal scales, and 5 allowed only qualitative input. Eleven tools provided scores or ratings by capacity factor and/or a composite capacity score or rating. A few tools also provide graphic output to provide a visual comparison of reported organizational strengths and weaknesses. Of the 30 tools, half addressed at least 40% of the factors identified in our review.
Of the five domains, Organizational Attributes, Management and Governance, and Collaboration were most consistently represented in the tools; 67% of tools included at least one factor in the Organizational Attributes and/or the Management and Governance domain. Seventy-three percent of tools included a least one factor in the Collaboration domain.
The Organizational Attributes domain includes tangible resources belonging or accessible to an organization (e.g., human, financial, technical, infrastructure), as well as intangible resources, such as the organization’s goals, knowledge, work and funding history, and culture. Approximately two-thirds of tools addressed these factors. Of these, mission and vision, human resources, financial resources, and infrastructure were most commonly addressed. Communication, leadership, and organizational culture are also commonly measured.
The Management and Governance domain addresses those systems, structures, and processes needed to effectively manage an organization. Financial management factors were most commonly addressed; also commonly measured were strategic management and administration, organizational structure factors, human resource management and measurement and evaluation.
Collaborations include relationships and communication with external partners—including governmental agencies, potential partner organizations, and stakeholders. These two factors appeared concurrently in more than half (57%) of all tools reviewed.
The External Environment and Organizational Performance domains were less consistently represented. Within the External Environment domain, 60 and 27% of tools operationalized political–legal/economic and sociocultural/geographic factors, respectively; 23% of tools addressed both factors and 37% did not include either factor. Forty percent of tools did not include any factors within the Organizational Performance domain; only 3% of reviewed decision aids included all three factors within this domain. Box 2 lists those capacity factors that are most prevalent in the 30 tools.
Box 2. Top 10 capacity factors.
Discussion
Assessing the readiness of organizations for global health intervention purposes should, according to the literature, involve measurement of both the capacity and the motivation of those organizations to engage in initiatives. Yet this review found only capacity assessment instruments. Measurement of an organization’s motivation or willingness to prioritize and engage in a global health intervention should be included and made available in such tools either in combination with existing capacity assessment tools or as stand-alone instruments.
Organizational capacity assessment tools with relevance for global health intervention measure many of the same domains and factors. This similarity may reflect a tendency by tool developers to employ common frameworks or orientations about what constitutes organizational capacity for global health intervention. It is possible that tool developers used a common evidence base to derive the measures included in the tools. These observed similarities may highlight an emerging consensus and convergence in scope regarding those factors that predict organizational effectiveness in the delivery of global health interventions.
The tools included in this review are focused at the organizational unit of analysis. As more organizational alliances active in efforts to scale-up health impact emerge, the most relevant level of analysis for estimating likelihood of success will be at the inter-organizational system or partnership level, reflecting the necessity that entire supply chains of collaborating and contracted organizations are involved when interventions are to be delivered to millions of people across large geographic areas (19). At this juncture, we did not find any tools that conceptualized and operationalized capacity or motivation measures at the level of inter-organizational systems or partnerships. While it can be expected that, because systems and inter-organizational partnerships are comprised of organizational actors, existing measures of organization-level capacities should be relevant, it is also the case that systems and partnerships require greater attention to working with heterophilous others (e.g., ministries of health, private organizations, non-governmental organizations, community health outreach workers). This requires coordination and contracting, and working across organizational boundaries where there may not be systems in place to seamlessly support such large-scale initiatives. Capacity assessment tools do not currently reflect the special challenges of this aggregate level of agency. Yet they could by focusing each partner organization’s experiences with the other organizations, for example, and measures focused on identification of complementary skills and resources across organizations.
Do Organizational Readiness Assessment Tools Work?
Given the current information available, we cannot conclude whether these assessment tools are effective. Many highly reputable organizations have sponsored the development of these tools; perhaps these tools have been broadly and enthusiastically applied in practice to good effect. Yet we were unable to find data for any of the tools regarding validity assessment or utilization evaluation. None specifically presented evidence supporting the inclusion or validity of specific measures. In general, it appears that developers have relied on expert opinion and structural measures—with an unproven relationship to outcomes—when developing indices and measures. We do not know if use of any of these tools is associated with improvement in organizational motivation or capacities, initiative performance, or efficient use of resources. Neither did we find information about how much any of these tools has been used.
The partial convergence of these tools on similar factors related to capacity suggests that these instruments tap into correct constructs. That is, the factors most commonly assessed are likely meaningfully related to organizational capacity. A number of the 21 factors in this review are well reflected in the literature about scaling up in low-income countries (17, 20). However, improvement and refinement of the composition of items contained in these tools is probably possible.
Utilization Considerations
Best practices for instrument and heuristic development suggests that tools or decision aids should be actively tested, refined, and improved with plausible potential users during a pre-testing stage conducted prior to release of the tool. Among other things, stakeholder feedback to prototype tools can provide exceedingly valuable insights into format preferences, optimum length, question order effects, response variance, use of graphics, and which types of potential users are best suited to provide different types of information. Though this may have occurred during development of the reviewed tools, the information available to us rarely included detail about such user-facing formative evaluation. Furthermore, stakeholders can differ considerably in their preferences for applied tools. Yet none of the 30 tools reviewed here was available in more than one interface or format.
Some tools reviewed here were developed in the 1990s, prior to remarkable developments in web-based applications. Nevertheless, the paucity of interactive capacity assessment tools for global health stakeholders is striking. It is likely that many users of these tools would find value in an instrument that is able to provide computations, comparative assessments, confidence intervals, or qualitative but tailored feedback based on data entered by the user. Of course, the utility of interactive formats would be limited to users with access to computers or electronic devices with internet connections.
An important implication for utilization lies in the understanding that much of the value in tools such as these lies beyond the data provided by the user and/or as computational or informational output. The process of engaging in use can be quite valuable for purposes of critical reflection, discussion, the development of a shared understanding among stakeholders, and the stimulation of a collective organizational will to improve processes and outputs. This type of enlightenment use has been shown in some research to be of more consequence in terms of learning and organizational improvement than is the instrumental data (the “answers”) themselves (21).
Next Steps in Tool Development
A previously published framework used to evaluate measurement systems in public health (22) employed four criteria to assess instruments:
• Clarity of measurement parameters and normative standards
• Balance between structural and process measures
• Evidence for effectiveness
• Specification of an accountable entity
These criteria, when applied globally to currently existing organizational capacity assessment tools, highlight challenges and opportunities for improvement. Many of the tools we reviewed employ clear standards; the best quantitative scales are tied to well-defined categories that reflect the continuum of organizational development. The newer and more robust tools strike a balance between structural and process measures, as well as specification of accountable entities.
This review suggests a need for robust, validated, and user-friendly tools to measure organizational capacity and, we suggest, organizational motivation; taken together, such tools can more fully represent the construct of organizational readiness for global health intervention. Identified strategies for instrument improvement include standardization, evaluation, validation, and application of an evidence base to inform tool development. This evidence base could, in part, be constructed using retrospective case studies of how global health intervention delivery fared to assess whether successes and failures were associated with certain factors. Information could also be gathered about the valence and weighting of those factors. An evidence base could then be applied prospectively, using predictive tests to determine whether tool use affects roll-out or scale-up of global health interventions, and how. These research validation steps, if applied in tandem with utilization study during formative development, could result in tools that not only work but also work well for users. As the evidence base supporting the identification of core domains, factors, and appropriate methodologies evolves, tools such as these will become more valid, reliable, and useful for the increasingly diverse range of stakeholders involved in global health interventions.
Author Contributions
The author originated the project and drafted and rewrote the manuscript.
Conflict of Interest Statement
The author has no commercial conflicts of interest regarding any of the assessment tools reviewed.
Acknowledgments
The author thanks Lauren K. Krause, Sarah D. Madrid, Quynh A. Le, Heather A. Nuanes, and Erica F. Morse for their work on the present study.
Funding
This work was supported by the Bill & Melinda Gates Foundation. The information provided in this article solely reflects the view of the author and not the views of the Bill & Melinda Gates Foundation.
References
1. Brownson RC, Colditz GA, Proctor EK. Dissemination and Implementation Research in Health. 2nd ed. Oxford, UK: Oxford University Press (2018).
2. Bosch-Capblanch X, Lavis JN, Lewin S, Atun R, Rottingen JA, Beck L, et al. Guidance for evidence-informed policies about health systems: rational for and challenges of guidance development. PLoS Med (2012) 9(3):e1001185. doi:10.1371/journal.pmed.1001185
3. Darmstadt GL, Bhutta ZA, Cousens S, Adam T, Walker N, de Bernis L, et al. Evidence-based, cost-effective interventions: how many newborn babies can we save? Lancet (2005) 365(9463):977–88. doi:10.1016/S0140-6736(05)71088-6
4. Bennett S, Ssengooba F. Closing the gaps: from science to action in maternal, newborn, and child health in Africa. PLoS Med (2010) 7(6):e1000298. doi:10.1371/journal.pmed.1000298
5. Bowen S, Zwi AB. Pathways to “evidence-informed” policy and practice: a framework for action. PLoS Med (2005) 2(7):e166. doi:10.1371/journal.pmed.0020166
6. Backer TE. Assessing and enhancing readiness for change: implications for technology transfer. NIDA Res Monogr (1995) 155:21–41.
7. Chilenski SM, Greenberg MT, Feinberg ME. Community readiness as a multidimensional construct. J Community Psychol (2007) 35(3):347–65. doi:10.1002/jcop.20152
8. Backer TE, David SL. Synthesis of behavioral science learnings about technology transfer. In: Backer TE, David SL, Soucy G, editors. Reviewing the Behavioral Science Knowledge Base on Technology Transfer. NIDA Research Monograph 155. Rockville, MD: National Institute on Drug Abuse (1992). p. 262–79.
9. Ruhe MC, Weyer AM, Zronek S, Wilkinson A, Wilkinson PS, Stange KC. Facilitating practice change: lessons from the STEP-UP clinical trial. Prev Med (2005) 40:729–34. doi:10.1016/j.ypmed.2004.09.015
10. Goodman RM, Speers MA, McLeroy K, Fawcett S, Kegler M, Parker E, et al. Identifying and defining the dimensions of community capacity to provide a basis for measurement. Health Educ Behav (1998) 25:258–78. doi:10.1177/109019819802500303
11. LaFond A, Brown L, MacIntyre BK. Mapping capacity in the health sector: a conceptual framework. Int J Health Plann Manage (2002) 17:3–22. doi:10.1002/hpm.649
12. Potter C, Brough R. Systemic capacity building: a hierarchy of needs. Health Policy Plan (2004) 19(5):336–45. doi:10.1093/heapol/czh038
13. Eade D. Capacity building: who builds whose capacity? Dev Pract (2007) 17(4–5):630–9. doi:10.1080/09614520701469807
14. Frost LJ, Reich MR. The access framework. In: Frost LJ, Reich MR, editors. Access: How do Good Health Technologies Get to Poor People in Poor Countries? Cambridge, MA: Harvard Center for Population and Development Studies (2008). p. 15–38.
15. Hanson K, Ranson MK, Oliviera-Cruz V, Mills A. Expanding access to priority health interventions: a framework for understanding the constraints to scaling-up. J Int Dev (2003) 15:1–14. doi:10.1002/jid.963
16. Simmons R, Shiffman J. Scaling up health service innovations: a framework for action. In: Simmons R, Fajans P, Ghiron L, editors. Scaling up Health Service Delivery. Geneva: World Health Organization (2007). p. 1–30.
17. Yamey G. Scaling up global health interventions: a proposed framework for success. PLoS Med (2011) 8(6):e1001049. doi:10.1371/journal.pmed.1001049
18. Cooley L, Kohl R. Scaling Up – From Vision to Large-Scale Change: A Management Framework for Practitioners. Washington, DC: Management Systems International (2006).
19. Naimoli JF. Global health partnerships in practice: taking stock of the GAVI Alliance’s new investment in health systems strengthening. Int J Health Plann Manage (2009) 24:3–25. doi:10.1002/hpm.969
20. Simmons R, Fajans P, Ghiron L. Scaling Up Health Service Delivery. Geneva, Switzerland: World Health Organization (2007).
21. Weiss CH. Have we learned anything new about the use of evaluation? Am J Eval (1998) 19(1):21–33. doi:10.1016/S1098-2140(99)80178-7
Keywords: organizational readiness tools, global public health, organizational capacity, organizational motivation, implementation science, scale up
Citation: Dearing JW (2018) Organizational Readiness Tools for Global Health Intervention: A Review. Front. Public Health 6:56. doi: 10.3389/fpubh.2018.00056
Received: 26 November 2017; Accepted: 14 February 2018;
Published: 02 March 2018
Edited by:
Ross Brownson, Washington University in St. Louis, United StatesReviewed by:
Shoba Ramanadhan, Dana–Farber Cancer Institute, United StatesLaura Kay Murray, Johns Hopkins Bloomberg School of Public Health, United States
Copyright: © 2018 Dearing. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: James W. Dearing, dearjim@msu.edu