The Community Health Worker (CHW) Common Indicators Project: Engaging CHWs in Measurement to Sustain the Profession

Despite progress in documenting the outcomes of Community Health Worker interventions, the lack of standardized measures to assess CHW practice has made it difficult for programs to conduct reliable evaluations, and impossible to aggregate data across programs and regions, impeding commitment to sustainable, long-term financing of CHW programs. In addition, while CHWs have sometimes been involved as data collectors, they have seldom been engaged as full partners in all stages of evaluation and research. This manuscript details the current work being done by the CI Project, demonstrating how CHWs are able to contribute to the integrity, sustainability, and viability of CHW programs through the collaborative development and adoption of a set of common process and outcome constructs and indicators for CHW practice and CHW program implementation.

Introduction

Measurement is inherently political. Who has the power and money to measure; what is measured and who chooses; how measurement occurs and who decides; whether or not measurement is a requirement for funding; how and by whom data are collected, collated, analyzed, interpreted and presented—all of these questions reflect ideologies and relationships to power. As scholars in the movement to decolonize research and practitioners of various strands of participatory research have made clear (1, 2), what were previously thought of (by those in power) as value-free, objective decisions, are anything but.

This is particularly true in the case of measurement and evaluation in the Community Health Worker (CHW) profession. CHWs are trusted community members who work with others in their community and use a range of approaches to improve health and equity (3). As predominantly members of marginalized communities where health inequities are greatest, CHWs (who use titles including Community Health Representatives in Native/American Indian communities and Promotores/as in Latinx communities) experience the same oppression and denial of power experienced by their broader communities. Historically and still today, this has included the denial of power to identify research and evaluation questions, and design, conduct, and disseminate research and evaluation studies.

While CHWs have sometimes been involved as data collectors, they have seldom been engaged as full partners in all stages of evaluation and research, from conceptualization to analysis to publication. A recent systematic review of CHW research found that only 23 articles out of 130 included CHW participation in five or more intervention research phases. Across the phases of research, 98.5% of studies employed CHWs to implement the health intervention. CHWs were frequently involved in participant eligibility screening and/or recruitment (57.6%) and data collection (49.2%). CHWs were much less frequently involved in identifying the research question (10.8%), data analysis (2.3%), and research dissemination/action (10.8%) (4).

Based on the authors' collective experience, because of CHWs' strong connections with community members, the data they collect are often more accurate and extensive than data collected by non-CHWs. In the context of the pandemic, for example, CHWs have been able to learn more about possible sources of transmission than epidemiologists who lack the same levels of trust and connection.

Consequences of alienation from the knowledge production process for CHWs have mirrored consequences for people in other marginalized communities and include characterization through a white/academic/bureaucratic/colonizer/medical gaze (1, 2). Additionally, CHW studies and evaluations have often lacked the crucial perspectives of those closest to and most informed about the work, which has led in turn to the use of process measures that do not adequately capture the contributions of CHWs, and outcome measures that do not emphasize the outcomes CHW are uniquely able to achieve (5).

Another outcome of CHWs' and their communities' social location is chronic underfunding, including underfunding of research and evaluation. One of the consequences of underfunding has been an inability to use common measures to conduct longitudinal studies such as those which have been conducted in fields like nursing (6). Despite progress in documenting the outcomes of interventions led by CHWs (7–12), a lack of standardized measures to evaluate CHW programs and policies has made it impossible to aggregate data across programs and regions, impeding commitment to sustainable, long-term financing of CHW programs and positions. Aggregated data could also facilitate inferences about which aspects of CHW practice and program inputs lead to improved outcomes. Lack of comprehensive and easy-to-use indicators hampers the ability of many community-based programs to reliably report outcomes to funders. Lack of attention to the processes by which CHWs achieve outcomes has made it difficult to conclusively demonstrate the importance and effectiveness of particular CHW roles, skills, and qualities, and identify the kinds of support that programs need to provide to CHWs (5).

To address these issues, building on work conducted by the Michigan CHW Alliance, in 2015 CHWs and non-CHW researcher collaborators from five states formed the national CHW Common Indicators (CI) Project. The purpose of the CI Project is to contribute to the integrity, sustainability, and viability of CHW programs through the collaborative development and adoption of a set of common process and outcome constructs and indicators for CHW practice. Since its organizing Summit in 2015, CHWs have been at the forefront of the CI Project. Five of 16 attendees at the organizing Summit were CHWs, three of whom co-facilitated the Summit. CHWs have been actively involved in presenting about the project, participating on the project Leadership Team and Advisory Group, and publishing blogs and peer-reviewed journal articles about the project (13–15).

Between 2015 and 2019, the CI Project achieved several important goals, including engaging more than 100 CHWs, researchers and others from around the country who are committed to identifying and implementing common indicators through a participatory process (13), and compiling a robust set of 20 process and outcome constructs (see Table 1). Based partly on a strong track record of accomplishments, in 2019 the CI Project received an initial year of funding from the Centers for Disease Control and Prevention (CDC) via the National Association of Chronic Disease Directors (NACDD)¹.

TABLE 1

Table 1. Full list of recommended constructs with definitions.

The purpose of the CDC-funded 2019-2020 scope of work was the collaborative selection of 10 priority constructs (from the list of 20 in Table 1) and development of associated indicators for evaluation of programs, systems, and investments involving CHWs. If adopted, these indicators will illuminate (1) the processes by which CHWs achieve positive outcomes at multiple levels (individual, community, and system), (2) the outcomes themselves, and (3) the key kinds of support that CHWs need to be successful, across programs and diseases or conditions. In addition, there are currently few specific indicators available to measure process and outcome constructs across CDC CHW programs and initiatives. The CI Project's 2019–2020 work addressed this gap, thus strengthening the evidence regarding CHW contributions to improving health and reducing inequities. The project did not require IRB approval as it did not include research participants.

This article describes how Project leaders were able to enhance the engagement of CHWs and achieve project objectives during the first year of CDC funding and in the midst of the COVID-19 pandemic through a series of activities that were based in popular education methodology. The article then discusses findings and lessons learned. After exploring conceptual and methodological constraints of the project, the manuscript concludes with a summary of next steps and some persistent questions posed by a project of this type.

Context

The setting for the CI Project is the CHW field in the United States. Organizational settings include community health centers; community-based organizations; academic health centers; universities; health plans; state, local, and national CHW associations; and a range of other organizations that are led by and employ CHWs. Principal actors (the population) are CHWs, who by definition are members of the communities they serve. Other stakeholders and constituents include university- and community-based researchers and evaluators; CHWs' colleagues in their places of work; program administrators; CHW supervisors; and those involved in making policy regarding CHWs at the state and national levels.

The phase of the CI Project covered by this manuscript occurred in the midst of at least four cataclysmic events which strongly influenced the project. These included the emergence of the COVID-19 pandemic; the uprising for racial justice that followed the police killings of George Floyd, Breanna Taylor, and Tony McDade; the largest economic crisis since the Great Depression; and an upsurge in white supremacist violence. As a project committed to health justice, the CI Project sought to respond to these crises in a variety of ways, some of which are detailed below.

At the beginning of the period described in this manuscript, there was one CHW on the five-person Leadership Team, and multiple CHWs in the Advisory Group. At the end of this period, the Leadership Team comprised three CHWs and three non-CHW collaborators, and a CHW Council consisting of four CHW leaders had been formed through a national recruitment process.

The CI Project uses popular education as a theoretical framework, an organizing philosophy, and an educational methodology. Also referred to as “people's education,” popular education creates settings in which people most affected by inequities can share what they know, learn from others in their community, and use their knowledge to create a more just and equitable society (16). Popular education and the CHW model grew out of many of the same historical roots and share key principles, such as the ideas that people most affected by inequity are the experts about their own lives, and that experiential knowledge is just as important as (and sometimes more important than) academic knowledge (17).

Collaboration With the CDC

This project benefited from the close collaboration of the CHW Work Group at CDC (the Work Group). The Work Group first convened in 2011 as an informal multidisciplinary group, composed of volunteers from CDC's National Center for Chronic Disease Prevention and Health Promotion. Since then, the Work Group has expanded to include representatives from across CDC. The mission of the Work Group is to facilitate, support, and advance CHW initiatives and policies to help accomplish public health goals.

The 2019–2020 CI work plan was created jointly by the CI Project Leadership Team and colleagues at CDC and NACDD. CDC and NACDD colleagues provided input into two major aspects of the Project: the choice of priority indicators and the development of indicator profiles. CDC colleagues prioritized development of indicators for policy and systems change at the program and state levels, since they had identified this as a gap in previous work and it is central to CDC's CHW Sustainability Strategy (18). The other constructs prioritized by CDC colleagues included: Participant Health and Social Needs; Participant Self-Reported Health Status; CHW-Facilitated Referrals; and CHW Integration into Teams.

The CI Leadership Team prioritized five constructs that had been highlighted by stakeholders: CHW Compensation, Benefits, and Promotion; CHW Enactment of the 10 Core Roles (as identified in the CHW Core Consensus or C3 Project) (19); CHW Involvement in Policy Making; Participant Empowerment; and Participant Social Support.

CDC colleagues were involved in the development of profiles (detailed documents including a definition of the construct, purpose and rationale of the indicator, a description of the indicator, recommendations for how to operationalize it, and other information) for the 10 priority constructs. CDC and NACDD colleagues also helped to develop criteria to identify and select key stakeholders who would provide feedback for the content and operationalization of the indicators. They participated actively in bi-monthly Advisory Group meetings and the 2020 Summit (see below) and met with the Leadership Team to discuss the dissemination plan.

Key Programmatic Elements

The collaborative methods used in this project included a review of peer-reviewed and non-peer reviewed literature, and comprehensive stakeholder engagement culminating in a 1.5-day online Summit held in 2020. Findings from all methods were triangulated to produce results and identify lessons learned. Both the findings of this project and popular education suggest that equitable engagement of marginalized individuals and communities including CHWs depends on thoughtful and diligent work before, during and after engagement opportunities. For this reason, the processes used and how they were influenced by popular education are described in some detail.

Literature Review

A review of peer-reviewed and non-peer reviewed literature about measurement of the 10 priority constructs was conducted to identify existing measurement approaches and promising paths forward for indicator development and validation. The Leadership Team divided the 10 priority constructs among four members of the team (including one CHW), and each member undertook the literature reviews for their respective constructs, consulting each other as needed.

It was difficult to conduct a truly systematic literature review across all 10 priority indicators. Leadership Team members were tasked with reviewing literature for a total of 10 constructs, some of which have been studied for decades. In some cases, Leadership Team members were able to build on literature reviews they had begun as much as 20 years earlier, for constructs that have been well-defined (e.g., empowerment and social support). In these cases, they used academic databases such as EBSCO host to update searches in other databases including Academic Search Complete, Academic Search Premier, E-Journals, Health Source: Nursing/Academic Edition, Fuente Académica, MasterFILE Premier, MedicLatina, Medline, and Psychology and Behavioral Sciences Index.

For the self-reported health status construct, an initial search on general health-related quality of life revealed two widely accepted measures: the SF-12 (20) and the CDC Healthy Days measure. Once these measures had been identified, a search was conducted in PubMed, filtering for publications in the U.S. from 2010 to 2020. For other well-defined constructs (i.e., teamness, a construct contained within CHW Integration into Teams), research reviews exist and were consulted.

In other cases (notably CHW Enactment of the 10 Core Roles, Policy and Systems Change, and CHW Compensation, Benefits, and Advancement), reviewers were unable to identify peer-reviewed literature. Leadership Team members used their networks and general abilities to search for various kinds of literature to conduct what they felt were sufficiently thorough reviews of both peer-reviewed and non-peer-reviewed literature. They consulted CDC documents, reports, presentations, and parallel literature in other fields and/or current practice in the CHW field.

Some indicators (e.g., Participant Health and Social Needs), while not well-defined as constructs, are frequently measured in the CHW field, and Leadership Team members were able to base indicators on existing, widely used measures. This was also the case with CHW-Facilitated Referrals, where activity tracking forms and individual patient self-management assessments from programs in Oregon and Michigan were used. Reference lists from journal articles were also searched. The most useful tools were derived from other organizations and measurement efforts.

While most of the literature review occurred before the formal stakeholder engagement described in the next section, stakeholders including CHWs have been engaged in choosing and defining constructs and identifying potential indicators since the CI Project began. Stakeholders provided input into the project at multiple times and venues, including the organizing Summit in 2015; 2016 and 2019 pre-conference workshops at the Annual Meeting of the American Public Health Association (APHA); multiple interactive workshops at state and local conferences; and bi-monthly Advisory Group calls. Since 2015, a substantial portion of several Advisory Group calls has been dedicated to specific constructs, eliciting how Advisory Group members have defined and measured the construct in their own programs and settings.

Stakeholder Engagement

A formal process of stakeholder engagement, emphasizing engagement of CHWs, was the centerpiece of the 2019–2020 scope of work. Input was sought from stakeholders at various times, in various venues, and on various questions about both overarching issues and specific constructs and indicators.

APHA Pre-conference Workshop, November 2019

In November 2019, the Leadership Team held a 2.5 h by-invitation workshop at the APHA Annual Meeting. Invitees included all current members of the Advisory Group as well as key partners from CDC and the National Association of Community Health Workers (NACHW). The primary goal of the workshop was to invite feedback on: (1) the tentative list of 10 priority constructs, (2) the stakeholders who would help guide indicator development for these constructs, and (3) the methods for engaging these stakeholders. The Leadership Team also aimed to develop community through face-to-face interaction. Eighteen people participated in the workshop, including CHWs, supervisors, researchers/evaluators, program directors, and others.

After a welcome and opening dinámica, facilitators reviewed objectives and action steps for the 2019–2020 work plan. They then divided participants into cooperative learning groups and elicited feedback on the criteria for choosing stakeholders, the initial list of stakeholders, and the proposed list of priority constructs. The workshop concluded with a large group report back, a brainstorm of next steps, and a group evaluation of the meeting.

Regular Meetings of the Project Advisory Group

Since the organizing Summit in 2015, the Advisory Group has met monthly or bi-monthly. The Advisory Group distribution list has grown from 16 to 170+ individuals from 30 states and the District of Columbia. Much of this growth occurred during 2019-2020, thanks at least in part to funding from the CDC. Many regular Advisory Group participants are active members of their state's or region's CHW association or network. Multiple researchers and CHW program evaluators also regularly attend. Attendance at meetings has climbed steadily from ~10 to more than 50 attendees.

Advisory Group meetings use popular education methodology and building community is the first objective of every meeting. When participant numbers allowed, facilitators set aside time at the beginning of every meeting for all participants to introduce themselves. Recently, facilitators have begun meetings with short breakout groups to further relationship building. Meetings always include an update from the Leadership Team members, who rotate facilitation responsibilities. In addition to focused discussion on specific indicators or topics, facilitators ensure time for participants to provide meaningful feedback on the information shared.

Individual Interviews and Focus Groups

Before the COVID-19 pandemic, the Leadership Team developed a plan for obtaining stakeholder feedback on the priority indicators that relied heavily on volunteers from the Advisory Group conducting in-person focus groups with CHWs and other stakeholders in their area. The Leadership Team developed a detailed lesson plan, PowerPoint, note-taking template and indicator grid (Table 2), and provided guidance to volunteers on how to use the materials. However, due to the pandemic, only two in-person focus groups took place. As the breadth of the pandemic became clear, the Leadership Team adapted the plan to rely on remote, web-based focus groups and one-on-one interviews. Ultimately, five focus groups were conducted with CHWs and other staff representing a state CHW association, a community-based organization, a state health department, the CI Advisory Group, and the research and outcomes arm of an urban health institute.

TABLE 2

Table 2. Indicator grid.

Leadership Team members also conducted seven individual interviews, as well as several informal conversations using an interview guide (see Figure 1). Combining focus groups and one-on-one interviews, 46 people were reached. While the Leadership Team had hoped to reach more people, they were able to reach most major stakeholder groups. By the end of the process, based on iterative analysis after each interview and conversation, Leadership Team members felt they had reached saturation on several questions and concepts, providing confidence in the findings.

FIGURE 1

Figure 1. Individual interview guide.

Early in the development of the Stakeholder Engagement Plan, the Leadership Team developed a system for analyzing the feedback received. This plan consisted of at least one Leadership Team member (and sometimes two) doing a line-by-line analysis of the transcript and/or notes taken by the facilitators and interviewers. (Some interviews and focus groups were audio recorded; during others, facilitators took careful notes.) Subsequently, summaries of the feedback on each indicator were created and a list of cross-cutting themes (identified in the Results section below) was compiled and added to the previously developed indicator grid (Table 2). This allowed the Leadership Team member responsible for developing each indicator to quickly see the individual feedback as well as the cross-cutting themes, which were identified through a process that combined both inductive and deductive coding (21, 22). Leadership Team members used these materials to discuss and make changes to the indicators and resulting grid.

Online Summit, May 2020

The second CI Summit was the major culminating stakeholder engagement activity to solidify the indicators and identify next steps for the Project. The Leadership Team began planning in January of 2020 and moved toward inviting participants to a 2-day Summit in Portland, Oregon. As the severity of the COVID-19 pandemic became increasingly clear, the Leadership Team made iterative shifts in the planning, moving first to the idea of a hybrid Summit, and then, by March 20, deciding to conduct the Summit entirely online. The Summit took place on May 14-15, 2020.

The Leadership Team sought to invite a diverse group of ~20 CHWs, researcher/evaluators, CHW program staff, health system staff, state and local health department staff, and colleagues from CDC and NACDD. Initial invitations were sent in early March, when an in-person gathering was still planned. This led to an overrepresentation of people from the West Coast, for whom travel costs would have been lower. Once organizers decided to make the Summit entirely virtual, they expanded the group to include more people from other parts of the country. A total of 39 people (including facilitators and staff from NACDD and CDC) participated in the Summit, of whom 16 identified as CHWs. Further information about planning for the Summit and how popular education was used in the online environment, as well as a final report from the Summit are available on the CI Project webpage (23).

A systematic approach was used to document the process and outcomes of the Summit. First, a 21-page document that included notes from all the plenary sessions at the Summit was prepared. In addition, 10 individual documents with notes on each indicator and three individual documents with notes on the piloting process were created. All these documents informed the Indicator Profiles. Finally, a checklist of important considerations for the piloting process was developed.

Findings and Discussion

Literature Review

A primary finding from the literature review was that there was a great deal of literature about some constructs and a paucity of literature about other constructs. While some constructs had not been measured in CHW programs, they had been measured in other settings. For some of the indicators, it made sense to adopt and adapt published and validated scales, whereas for other indicators, there were no published, validated measurement approaches. Thus, indicators had to be developed “from the ground up,” by proposing questions/items that have been used in CHW program evaluations but not necessarily published and validated, and/or by developing brand new questions/items that were suggested and endorsed by our stakeholders. Another general finding was that some construct names needed to be changed to align with what is used in peer-reviewed literature and in community health practice. This was the case with Participant Health and Social Needs (which was originally titled, Participant Access to Health and Social Services).

Another important finding from the literature review was that a project with similar goals to the CHW Common Indicators Project—yet focused on low- and middle-income countries rather than the United States—had recently been conducted (24). The existence of this project, known as the Frontline Health Project and funded by the Bill and Melinda Gates Foundation, further confirmed the timeliness and potentially global significance of the CHW Common Indicators Project. There was a great deal of overlap between the two projects in overall measurement frameworks and in respective lists of recommended evaluation constructs. In this case, the literature review informed the stakeholder engagement by prompting the Leadership Team to engage leaders within the Frontline Health Project, who provided input on the CI Project indicators, shared resources, and joined the Advisory Group. One important distinction is that while the CI Project has focused on engaging CHWs as leaders in the work, publications by the Frontline Health Project do not provide evidence of such engagement. This likely leads to important differences between the projects in overall measurement frameworks and recommended constructs and indicators, a review of which is beyond the scope of this article.

Stakeholder Engagement

The most important findings from the stakeholder engagement are reflected in Table 2. Specific findings from specific stakeholder engagement activities are highlighted below.

2019 APHA Pre-conference Workshop

Participants in the workshop endorsed the tentative list of 10 priority constructs and strongly endorsed selected constructs including Participant Empowerment, CHW Integration into Teams, and CHW Compensation, Benefits, and Promotion. In addition, they urged Leadership Team members to make important additions to the stakeholder list, including CHW employers, payers/health plans, FQHCs, and state health department representatives. They identified a need to clarify and further develop the CHW-Facilitated Referrals construct, so as not to make CHWs responsible for “failed” referrals (when, for example, services don't exist and/or are substandard, not culturally competent, inaccessible, etc.). Finally, they identified a need to acknowledge the importance of health care utilization and cost measures but make clear it is impossible to create one utilization/cost measure that will work in all settings.

Advisory Group Meetings

In the meeting evaluation, participants regularly provide useful corrective feedback, for example: “Sometimes we need a space to discuss things that aren't on the agenda, so we have some time to discuss and organize.” They also frequently express appreciation for the way meetings are conducted, such as the following from the May 2020 meeting: “As usual, I appreciate the level of organization that gets us through a lot of stuff. I continue to be very excited about sitting on this group.” The trust and community building processes inherent in the Advisory Group meetings and other CI Project activities have been essential to the broad consensus and growing, nationwide uptake of common indicators for evaluating CHW practice.

Focus Groups, Individual Interviews, and May 2020 Summit

Overall, this phase of the Project demonstrated the broad acceptability of the measurement framework developed within the CI Project. That framework centers CHWs' 10 core roles with an explicit goal to ensure that all 10 roles are understood and practiced within CHW programs. The framework also highlights key kinds of support that CHWs need to be successful in all 10 roles, and outcomes that CHWs are particularly capable of bringing about—not only at the level of individual participants in CHW programs (e.g., wellbeing and social support), but also at the level of policy and systems change to address structural determinants of health inequities. Notably, the framework defines and forefronts a multi-level indicator of empowerment, which both the literature and our experience suggest is among the most significant and emblematic outcomes of CHW work. This framework reflects the deep participation and leadership of CHWs within the Project, who aim to protect the integrity and advance the self-determination of the workforce. Non-CHW stakeholders generally affirmed the importance of these key features of the measurement framework, as well as the principle that CHWs must be deeply involved in evaluating their own profession.

Generally, stakeholders expressed enthusiasm and approval for the choice of indicators as well as the proposal to operationalize the indicators in existing tools such as encounter forms, CHW surveys, CHW employer surveys, participant surveys, and state performance reports. Stakeholders reinforced the importance of complementing quantitative indicators with qualitative, narrative, and ethnographic assessment methods. They pointed out that narrative methods and storytelling are uniquely powerful and effective ways to document how and why CHW practice is effective in various cultural settings, and to clarify the kinds of changes that are necessary to improve systems, provide adequate support to CHWs and their communities, and achieve health equity.

Lessons Learned

An important lesson reaffirmed during this phase of the CI Project was that obtaining meaningful input from a diverse group of stakeholders about a project primarily concerned with measurement and evaluation in the midst of a pandemic is challenging and requires thoughtful planning, skillful use of popular education methodology, and a team that is aligned around common goals and principles.

During the early stakeholder focus groups, it was found that many stakeholders who were not trained as evaluators commented not on how to measure concepts like social support, but rather how to increase social support among program participants and what they need to be successful in their work. Making the distinction between doing the work and measuring the work was challenging for volunteer focus group facilitators. This was partly because most CHW program staff are focused on doing, not measuring, their work and CHWs remain largely marginalized from evaluation and research processes.

Similarly, based on the summaries of indicator-specific input created after the Summit, it was clear that the way the indicators were explained by the small group facilitators affected the feedback group members provided on those indicators. As the CI Project moves to piloting, it will be important to assure that indicators are explained in a consistent way.

Interestingly, many stakeholders, including CHWs, resisted shortening the length of (i.e., reducing the number of items within) the indicators, even when the goal of keeping the indicators relatively quick and easy to use had been emphasized. This was generally because stakeholders felt that cutting proposed items would eliminate important aspects of CHW practice and participant outcomes. At the same time, stakeholders readily appreciated that program capacity must be developed to measure the work, so that measurement and evaluation help rather than hinder CHWs in doing the work. As we move forward with the piloting process, it will therefore be important to carefully communicate with CHWs and other stakeholders the pros and cons of shorter surveys, and the fact that funding is necessary to pay for CHWs' and others' time involved in new data collection and reporting. This is a reflection of how the goals of the CHW Common Indicators Project are intricately tied to the issue of securing sustained funding for CHW programs generally.

Other lessons learned included the importance of clearly communicating project assumptions at the outset of any engagement activities. Specifically, stakeholders expressed the importance of making it clear that:

• the indicators will be accompanied by a manual that will carefully explain the meaning and intent of each indicator;

• the Project will recommend that indicators be operationalized using existing data collection and/or case management tools, whenever possible, to reduce the burden on CHWs and other data management staff;

• quantitative indicators are proposed because they are easiest to implement in a consistent and reliable way, and not because they are of higher value than qualitative methods;

• health care utilization and cost measures are important but not included, because it is impossible to create one utilization or cost measure that will work in all cases and not all CHW programs have access to this data; and

• while the CI Project is unable to control the fact that CHWs are often given multiple different titles, the project's recommended indicators can help bring about more consistency in CHW job descriptions, given the indicators' built-in emphasis on the APHA definition of a CHW as well as the C3 Project's definition of the 10 core roles of CHWs (19).

Though it was enforced by the pandemic, heavier reliance on individual interviews was beneficial to the goal of collecting constructive feedback on the proposed and evolving indicators. The experience doing focus groups revealed that the kind of input needed was easier to obtain one-on-one.

Future Applications

With assistance from colleagues at CDC and NACDD and participants in the 2020 Summit, the Leadership Team has identified several next steps, in which the CI Project is currently engaged. The first step is piloting the indicators developed during 2019-2020 and developing a manual/toolkit with information crucial to piloting, including definitions, intent, background, and methods of calculation. A participatory and developmental evaluation for the pilot will also be conducted, with pilot sites actively involved in the on-going development of the evaluation plan. Other next steps include developing an indicator for reflective supervision, continuing to build project infrastructure, and strengthening the CI Project's CHW-led and community-based methodology. In pursuit of the final goal, following a national recruitment process, in August of 2020 the CI Leadership Team expanded and is now 50% CHW and 50% CHW ally. In addition, the team created a CHW Council composed of four CHWs with experience in research and evaluation. Finally, the Leadership Team chose and adapted a racial equity tool to guide future project decisions.

Conceptual or Methodological Strengths and Constraints

This phase of the CI Project had several strengths, including a well-developed network of 180+ individuals who possess a variety of skills, perspectives, and knowledge based on lived experience and are committed to improving measurement in CHW programs; and the Leadership Team's dedication to and capacity in using popular education methodology. This methodology allows the Leadership Team to operationalize our commitments to community-based participatory research; shared power; racial, social, and health justice; and a non-hierarchical approach. All members of the Leadership Team are deeply committed to the project, and possess a well-rounded set of skills as CHWs, researcher/evaluators, and program managers. An additional strength is our strong relationship with NACHW and various state CHW organizations. The CI Project benefited from the dedication and knowledge of the CDC CHW Work Group members and partners at NACDD. Also, using an online platform for the 2020 Summit facilitated inclusion of a larger and more diverse number of stakeholders.

The work discussed in this article had several constraints and limitations. A major limitation was the COVID-19 pandemic, which required project partners to be creative in how they collected stakeholder feedback and built community. Funding constraints and lack of full-time staff on the project limited the number of hours staff could dedicate to various project activities. Despite attempts to mitigate its effects, limited racial/ethnic diversity on the Leadership Team, the fact that all three members with doctoral degrees were white, and that only one member was a CHW unquestionably influenced power dynamics and meant that the Leadership Team lacked crucial perspectives. Recognizing this led to an intentional process of increasing both racial/ethnic diversity and CHW representation on the Leadership Team and creating a CHW Council to provide additional CHW input into decision-making.

Conclusion

The CDC-funded 2019-2020 work plan was pivotal for the CI Project. Having funds to pay salary and stipends allowed Leadership Team members and Summit facilitators to set aside time to focus on this project, leading to substantial progress on the overall development of the CI Project. With the participation of multiple stakeholders, particularly CHWs, 11 profiles were developed for 10 priority constructs. The profiles include information about why the indicators are important, why they should be measured in particular ways, and how data should be calculated. In addition, largely based on the success of the Summit, the CI Project substantially expanded the constituency committed to the Project. In 2020–2021, Leadership Team members look forward to piloting the indicators, developing materials and methods to support the piloting, strengthening the work through the application of an explicit racial equity lens, and continuing to expand CHW involvement and capacity as the researchers and evaluators for and about their field.

Some persistent and important questions remain. The CI Project is premised partly on the idea that policy makers and funders require additional data about CHW outcomes before they will agree to sustainably finance CHW programs. Yet the same does not seem to be true of professions and programs staffed by more privileged people. They do not have to produce data about their value, at least not in the same way. The CI Project will continue to problematize that fundamental issue while continuing to develop indicators, based on the conviction that if CHWs and dedicated allies do not do so, someone else will, with potentially dangerous consequences for CHW autonomy and self-determination.

Data Availability Statement

The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author/s.

Author Contributions

KR and NW contributed equally to the work plan and writing of this manuscript. KM contributed to the work plan and writing of this manuscript. TC-D contributed to the work plan and reviewing of this manuscript. VA, PJ, and SM-J contributed to the review of this manuscript. All authors contributed to the article and approved the submitted version.

Funding

This publication was supported in part by Federal Award Number: 5-NU38OT000286-02, CFDA Number: 93.421, from the Center for Disease Control and Prevention, via the National Association of Chronic Disease Directors.

Disclaimer

The findings and conclusions of this article are those of the authors and do not necessarily represent the official position of the U.S. Department of Health and Human Services or the Centers for Disease Control and Prevention. Furthermore, this article is not intended to promote any particular legislative, regulatory, or other action.

Conflict of Interest

NW was employed by Wiggins Health Consulting, LLC.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

The authors would like to thank the members of the Community Health Worker Work Group at the Center for Disease Control and Prevention (CDC) (Diane Manheim, Refilwe Moeti, Betsy Rodriguez, Magon Saunders, and Sharada Shantaram), the National Association of Chronic Disease Directors (NACDD) (Kayla Craddock), and the participants in our stakeholder engagement activities including members of our Advisory Group. The work represented in this article is the result of all of your contributions.

Footnotes

1. ^This publication was supported in part by Federal Award Number: 5-NU38OT000286-02, CFDA Number: 93.421, from the Center for Disease Control and Prevention, via the National Association of Chronic Disease Directors.

2. ^Please note that in the CHW Common Indicators Project, the term “Community Health Workers” (CHWs) is inclusive of Promotores/as de Salud and Community Health Representatives.

References

1. Tuhiwai Smith L. Decolonizing Methodologies: Research and Indigenous Peoples. New York, NY: St. Martin's Press (1999).

Google Scholar

2. Wallerstein N, Duran B. The conceptual, historical, and practice roots of community based participatory research and related participatory traditions. In: Minkler M, Wallerstein, N, editors. Community Based Participatory Research for Health. San Francisco, CA: Jossey-Bass (2003). p. 27–52.

3. Farquhar SA, Michael YM. Wiggins N. Building on leadership and social capital to create change in two urban communities. Am J Public Health. (2005) 95:596–601. doi: 10.2105/AJPH.2004.048280

CrossRef Full Text | Google Scholar

4. Coulter K, Ingram M, McClelland DJ, Lohr A. Positionality of community health workers on health intervention research teams: a scoping review. Front Publ Health. (2020) 8:208. doi: 10.3389/fpubh.2020.00208

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Kieffer E, Yankey N, Mitchell K, Allen CG, Janevic MR, Thomas C, et al. The role of evaluation in developing and sustaining community health worker coalitions: the example of the Michigan Community Health Worker Alliance. J Ambul Care Manage. (2015) 38:283–96. doi: 10.1097/JAC.0000000000000113

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Old DL. The Nurse–family Partnership: an evidence-based preventive intervention. Infant Mental Health. (2006) 27:5–25. doi: 10.1002/imhj.20077

PubMed Abstract | CrossRef Full Text | Google Scholar

7. The Community Guide. Cardiovascular Disease: Engaging Community Health Workers. Centers for Disease Control and Prevention Website (2015). Available online at: https://www.thecommunityguide.org/findings/cardiovascular-disease-prevention-and-control-interventions-engaging-community-health (accessed September 18, 2020).

Google Scholar

8. The Community Guide. Diabetes Management: Interventions Engaging Community Health Workers. Centers for Disease Control and Prevention Website (2017). Available online at: https://www.thecommunityguide.org/findings/diabetes-management-interventions-engaging-community-health-workers (accessed September 18, 2020).

Google Scholar

9. The Community Guide. Diabetes Prevention: Interventions Engaging Community Health Workers. Centers for Disease Control and Prevention Website (2016). Available online at: https://www.thecommunityguide.org/findings/diabetes-prevention-interventions-engaging-community-health-workers (accessed September 18, 2020).

Google Scholar

10. The Community Guide. Cancer Screening: Interventions Engaging Community Health Workers – Breast Cancer. Centers for Disease Control and Prevention Website (2019). Available online at: https://www.thecommunityguide.org/findings/cancer-screening-interventions-engaging-community-health-workers-breast-cancer (accessed September 18, 2020).

11. The Community Guide. Cancer Screening: Interventions Engaging Community Health Workers – Cervical Cancer. Centers for Disease Control and Prevention Website (2019). Available online at: https://www.thecommunityguide.org/findings/cancer-screening-interventions-engaging-community-health-workers-cervical-cancer (accessed September 18, 2020).

12. The Community Guide. Cancer Screening: Interventions Engaging Community Health Workers – Colorectal Cancer. Centers for Disease Control and Prevention Website (2019). Available online at: https://www.thecommunityguide.org/findings/cancer-screening-interventions-engaging-community-health-workers-colorectal-cancer (accessed September 18, 2020).

13. Wiggins N, Maes K, Palmisano G, Rodela K, Rodriguez Avila L, Kieffer E. A community participatory approach to identify common evaluation indicators for Community Health Worker practice. Prog Community Health Partnersh (In press).

14. Rodela K, Wiggins N, Maes K. Using Popular Education in an Online Environment: Lessons Learned From the Community Health Worker Common Indicators Summit [Blog Post]. (2020). Retrieved from: https://www.internationalhealthpolicies.org/featured-article/using-popular-education-in-an-online-environment-lessons-learned-from-the-community-health-worker-common-indicators-summit/ (accessed December 7, 2020).

15. Rodela K, Maes K, Wiggins N, Kieffer E, Palmisano G. Community Health Workers and the Power of Measurement [Blog Post]. Health Systems Global: Featured Article Series (2020). Retrieved from: https://www.internationalhealthpolicies.org/featured-article/community-health-workers-and-the-power-of-measurement/ (accessed May 12, 2020).

Google Scholar

16. Wiggins N. Popular education for health promotion and community empowerment: a review of the literature. Health Promot Int. (2012) 27:356–71. doi: 10.1093/heapro/dar046

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Wiggins N, Hughes A, Rios-Campos T, Rodriguez A, Potter C. La Palabra es Salud (The Word is Health): combining mixed methods and CBPR to understand the comparative effectiveness of popular and conventional education. J Mixed Meth Res. (2014) 8:278–98. doi: 10.1177/1558689813510785

CrossRef Full Text | Google Scholar

18. Jayapaul-Philip B, Shantharam SS, Moeti R, Kumar GS, Barbero C, Rohan EA, et al. Community Health Worker initiatives: an approach to design and measurement. J Publ Health Manag Pract. (2020). doi: 10.1097/PHH.0000000000001183. [Epub ahead of print].

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Rosenthal EL, Menking P, St John J. The Community Health Worker Core Consensus (C3) Project: A Report of the C3 Project Phase 1 and 2, Together Leaning Toward the Sky. A National Project to Inform CHW Policy and Practice. El Paso, TX: Texas Tech University Health Sciences Center (2018).

20. Ware J, Kosinski M, Keller SD. A 12-Item Short-Form Health Survey: construction of scales and preliminary tests of reliability and validity. Med Care. (1996). 34:220–33.

PubMed Abstract | Google Scholar

21. Miles MB, Huberman AM. Qualitative Data Analysis: An Expanded Sourcebook. Thousand Oaks, CA: Sage (1994).

Google Scholar

22. Strauss A, Corbin J. Basics of Qualitative Research: Grounded Theory Procedures and Techniques, Newbury Park, CA: Sage (1990).

Google Scholar

23. Michigan Community Health Workers Alliance. CHW Common Indicators Project (2021). Available online at: https://www.michwa.org/news/common-indicators-project (accessed February 28, 2021).

24. Agarwal S, Sripad P, Johnson C, Kirk K, Bellows B, Ana J, et al. A conceptual framework for measuring community health workforce performance within primary health care systems. Hum Resour Health. (2019) 18:1–20. doi: 10.1186/s12960-019-0422-0

PubMed Abstract | CrossRef Full Text | Google Scholar