Real World Feedback: How Well Did the Virtual Training Academy Prepare California's COVID-19 Contact Tracing Workforce?

To effectively respond to the COVID-19 pandemic, California had to quickly mobilize a substantial number of case investigators (CIs) and contact tracers (CTs). This workforce was comprised primarily of redirected civil servants with diverse educational and professional backgrounds. The purpose of this evaluation was to understand whether the weeklong, remote course developed to train California's CI/CT workforce (i.e., Virtual Training Academy) adequately prepared trainees for deployment. From May 2020 to February 2021, 8,141 individuals completed the training. A survey administered ~3 weeks post-course assessed two measures of overall preparedness: self-perceived interviewing proficiency and self-perceived job preparedness. Bivariate analyses were used to examine differences in preparedness scores by education level, career background, and whether trainees volunteered to join the COVID-19 workforce or were assigned by their employers. There were no significant differences in preparedness by education level. Compared to trainees from non-public health backgrounds, those from public health fields had higher self-perceived interviewing proficiency (25.1 vs. 23.3, p < 0.001) and job preparedness (25.7 vs. 24.0, p < 0.01). Compared to those who were assigned, those who volunteered to join the workforce had lower self-perceived job preparedness (23.8 vs. 24.9, p = 0.02). While there were some statistically significant differences by trainee characteristics, the practical significance was small (<2-point differences on 30-point composite scores), and it was notable that there were no differences by education level. Overall, this evaluation suggests that individuals without bachelor's degrees or health backgrounds can be rapidly trained and deployed to provide critical disease investigation capacity during public health emergencies.

Introduction

Case investigation and contact tracing are core public health strategies used to interrupt transmission of SARS-CoV-2, the virus that causes the coronavirus disease 2019 (COVID-19) Case investigation is the identification, investigation, and support of individuals with confirmed and probable diagnoses of COVID-19, and contact tracing is the subsequent identification, monitoring, and support of their close contacts who may have been exposed to the virus (1). Comprehensive and timely case investigation and contact tracing can reduce morbidity and mortality by providing education and resources for cases and contacts to adhere to isolation and quarantine guidance and therefore break the chain of infection (2, 3). While public health departments in the United States (U.S.) maintain contact tracing programs for ongoing prevention and control of transmissible diseases such as tuberculosis, the size of this workforce was quickly deemed insufficient to respond to the COVID-19 pandemic (4, 5). The National Association of County and City Health Officials estimated that the national contact tracing workforce capacity would need to double, from approximately 15 professionals to 30 professionals per 100,000 population in order to adequately contain COVID-19 (5). In California (CA), the contact tracing workforce gap was estimated to be ~10,000 professionals.

To bridge this workforce gap, public health experts at the California Department of Public Health (CDPH), the University of California, San Francisco (UCSF) and the University of California, Los Angeles (UCLA) were mobilized to develop and scale a virtual training program to rapidly train a large, competent workforce of case investigators (CIs) and contact tracers (CTs). This workforce was primarily redirected civil servants from county or state departments. Thus, trainees had diverse educational and professional backgrounds, and many had no public health knowledge or experience. In addition, while some staff volunteered to support the COVID-19 response, some were mandated to do so in their obligated role as Disaster Service Workers during State emergencies (6). As a result, trainees may have had varying levels of willingness to work as CIs/CTs. The purpose of this evaluation is to determine whether the program was effective in training a contact tracing workforce diverse in professional backgrounds and motivations to join the workforce.

Methods

Program Description

The COVID-19 Virtual Training Academy (VTA) for Case Investigators & Contact Tracers is a weeklong, remote learning course that combines self-study modules, live didactic webinars, and interactive skill development labs through which trainees build knowledge and practice skills required to engage in effective, culturally responsive communication with cases and contacts. Didactic webinars provide (a) fundamental knowledge of technical topics including the epidemiology of COVID-19, comprehensive containment strategies for preventing the spread of COVID-19, and the purpose and practice of case investigation and contact tracing; and (b) a basic understanding of skills important for effective communication with members of diverse communities, including cultural humility, health coaching, and motivational interviewing (MI). Skill development labs allow trainees to actively practice case investigation and contact tracing interviews and build skills in health coaching and MI. Initially (through November 2020), trainees would enroll in either a 16-h CT track or a 20-h CT/CI track, depending on their anticipated job assignment. The core course content was the same, with CI-track learners completing additional didactic and skill development sessions to learn how to conduct case investigation interviews. The CT-only track was discontinued in December 2020 as the state experienced an increased demand for CIs. Additional details about training development and content (7) and evaluation of the impact of the program on trainees' knowledge and self-perceived skills (8) are published elsewhere.

Purpose of the Evaluation

The purpose of this evaluation was to assess whether trainees felt adequately prepared to perform CI/CT duties after deployment in the field to inform best practices for rapidly scaling and training a workforce to respond to public health emergencies. Given the disparate backgrounds of the new workforce, the primary research question was whether the VTA adequately prepared trainees for deployment regardless of their personal and professional characteristics. Specific aims were to understand trainees' (1) self-perceived proficiency in interviewing skills, and (2) self-perceived job preparedness, and whether these varied by (a) education level, (b) professional background, or (c) volunteer status.

Population and Setting

From May 18, 2020 to February 26, 2021, 8,141 individuals (29 cohorts of trainees) completed the VTA training. Course completers were defined as trainees who passed a post-training knowledge assessment with a score of 70% or higher within two attempts. The course was offered weekly from May through September to rapidly build workforce capacity, then semi-monthly thereafter as enrollment decreased. Because this was a CDPH-funded initiative, course participation was limited to individuals who were assigned to conduct case investigation and/or contact tracing for a local health department (LHD) or LHD partner organization (e.g., a university or tribal organization), and included primarily county staff (i.e., LHD staff and redirected non-LHD county government staff), and redirected staff from over 100 State departments. Eligibility was verified by the VTA training team and CDPH.

Data Sources and Measures

A “Real World Feedback” survey was distributed to trainees ~3 weeks following completion of the training. This timeline was intended to be long enough so that trainees were likely to have been deployed and working as CIs/CTs, but not so long to cause significant concern about the probability of recall bias. Initially, an email announcement was sent to all course enrollees ~2.5 weeks after the final day of the course, inviting them to take the optional survey in the course Learning Management System (LMS). Due to concerns that some trainees might have forgotten their LMS log-in credentials and been unable to access the survey, survey administration was moved to the Qualtrics survey platform in September 2020. Another benefit of this transition was the ability for invitations to be targeted only to trainees who received a record of course completion, to ensure feedback was gathered from trainees who fully engaged with the course. Additionally, to improve the likelihood that trainees had begun working as a CI or CT before taking the survey, the timing of the invitation was moved to ~3.5 weeks after course completion. The survey was voluntary, and no incentives were offered for completion. The UCLA Office of the Human Research Protection Program determined that the evaluation project did not meet the definition of human participants research, so did not require institutional review board approval.

The survey was developed by the VTA Monitoring and Evaluation Working Group to reflect the unique content of the training and job responsibilities of COVID-19 CIs and CTs in California. Two measures were used to assess overall preparedness among those who had been deployed to perform CI/CT duties: self-perceived interviewing proficiency and self-perceived job preparedness.

Self-Perceived Interviewing Proficiency

Survey respondents were asked to assess their level of comfort and proficiency in the following CI/CT interview areas using a 5-item Likert scale (1 = not proficient, 2 = somewhat proficient, 3 = neutral, 4 = proficient, and 5 = extremely proficient): (a) assessment of symptoms and medical history; (b) assessment of COVID-19 exposure; (c) categorizing a contact's risk; (d) providing guidance on self-isolation and quarantining; (e) referring contacts for testing; and (f) referring contacts for social support services. In addition, a composite interviewing proficiency score was created by summing the six items, with scores possibly ranging from 6 to 30.

Self-Perceived Job Preparedness

Survey respondents were asked to indicate their level of agreement with the following six statements about job perception and preparedness using a 5-item Likert scale (1 = strongly disagree; 2 = somewhat disagree; 3 = neither agree nor disagree; 4 = somewhat agree; 5 = strongly agree): (a) I was prepared to do this work; (b) I can do this work well; (c) I am making a difference through this work; (d) I am prepared to use MI skills; (e) I am prepared to use my health coaching skills; (f) I am prepared to handle difficult situations. In addition, a composite job preparedness score was created by summing the six items, ranging from 6 to 30.

Survey data were matched to registration data to obtain age, race and ethnicity, gender, education level, employer, career background, and volunteer status. Education level was classified as less than bachelor's degree (high school, some college, or associate degree); college graduate; or graduate school (some graduate school or graduate degree). Career background was obtained by gathering registrants' professional roles prior to the pandemic and was dichotomized as public health related (disease investigator, public health nurse, public health administrator, other allied health professional) or non-public health related (e.g., librarian, city attorney, etc.). As some state and LHD staff volunteered to perform CI/CT duties, volunteer status was gathered at registration and dichotomized as volunteer (trainees who volunteered to be trained as CIs/CTs) or assigned (trainees who were assigned to be trained through their job); those who reported not knowing their volunteer status were classified as assigned.

Analysis

We examined descriptive characteristics of the course completers, the sample of survey completers, and the sample of survey completers who had been deployed to perform CI/CT duties at the time of survey completion. Differences between course completers and survey completers were explored using Pearson chi-squared tests. Job preparedness measures were analyzed among survey completers who had been deployed. Self-perceived interviewing proficiency and job preparedness were summarized using means. Bivariate analyses using Wilcoxon rank sum or Kruskal Wallis tests were used to examine differences in individual and composite interviewing proficiency and job preparedness scores by trainee characteristics. Sample sizes vary slightly due to non-response. Analyses were conducted using R version 4.0.3 and Rstudio Version 1.4.1103.

Results

Among the 8,141 course completers, n = 1,522 completed the survey (19% response rate). Of those, n = 439 (29%) had been deployed to perform contact tracing duties and represent the primary sample for analyses. Table 1 summarizes the demographic characteristics of course completers, survey completers, and those who had been deployed.

TABLE 1

Table 1. Demographic characteristics of course participants and study sample.

Survey completers were primarily redirected State of California staff (60%), from non-public health-related careers (78%), and assigned to work in the COVID-19 response (59%). Approximately one-third of survey respondents had completed less than a bachelor's degree (32%). Just over half of the respondents were White (53%) and two-thirds were female (68%). Nearly two-thirds of survey respondents were age 45 years and older (63%). Compared to the full sample of course completers, survey respondents were more likely to be State of California staff (60% vs. 46%, p < 0.001), from non-public health-related fields (78% vs. 71%, p < 0.001), and volunteers (42% vs. 34%, p < 0.001).

Table 2 summarizes self-perceived interviewing proficiency and self-perceived job preparedness. Mean scores for most interviewing skills were at least 4, corresponding to “proficient” on the Likert scale. Trainees were most confident in their ability to provide guidance on self-isolation and quarantining (proficiency score = 4.2). The only item with a mean score below 4 related to self-perceived proficiency referring contacts for social support services (3.7). Similarly, mean scores for most job preparedness items were at least 4. Trainees had strong convictions in their ability to do contact tracing work well (preparedness score = 4.3) and their ability to make a difference through their work (preparedness score = 4.4). Trainees felt least prepared to handle difficult situations, with a mean score of 3.8.

TABLE 2

Table 2. Self-perceived interviewing proficiency (n = 409) and job preparedness (n = 412).

Figure 1 summarizes differences in self-perceived interviewing proficiency by education (A), career background (B) and volunteer status (C).

FIGURE 1

Figure 1. Differences in Self-Perceived Interviewing Proficiency by Education Level (A, n = 383), Career Background (B, n = 353), and Volunteer Status (C, n = 380).

There were no significant differences in composite interviewing proficiency scores between trainees with less than bachelor's degree (24.2), college (24.2), or graduate school (23.5) education (p = 0.38). Similarly, there were no differences in self-perceived interviewing proficiency between those who volunteered to be trained as CIs/CTs (23.5) and those who were assigned by their employer (24.1) (p = 0.16). However, there were significant differences in interviewing proficiency by career background. Trainees with public health career backgrounds had significantly higher self-perceived proficiency in interviewing skills (25.1) compared to those with non-public health backgrounds (23.3) (p < 0.001). In examining the six interview skills individually, trainees with public health backgrounds had higher self-perceived proficiency on all six items compared to those with non-public health backgrounds (p < 0.05, data not shown).

Figure 2 summarizes differences in self-perceived job preparedness by education (A), career background (B), and volunteer status (C).

FIGURE 2

Figure 2. Differences in Self-Perceived Job Preparedness by Education Level (A, n = 386), Career Background (B, n = 356), and Volunteer Status (C, n = 383).

There were no differences in composite job preparedness scores between trainees with less than bachelor's degree (24.9), college (24.5), or graduate school (24.4) education (p = 0.70). However, there were differences in job preparedness by career background and volunteer status. Trainees with public health backgrounds reported greater preparedness to perform contact tracing job duties (25.7) than those with non-public health backgrounds (24.0) (p < 0.01). Examination of the individual job preparedness items revealed there was no significant difference in self-perceived preparedness to use MI skills between those with public health and non-public health backgrounds (p = 0.06, data not shown). However, those with public health backgrounds had significantly higher self-perceived preparedness to handle difficult situations, use health coaching skills, do the work and do it well, and make a difference (p < 0.05, data not shown).

Volunteers had significantly lower job-preparedness scores (23.8) compared to those who were assigned (24.9) (p = 0.02). In examining the individual job preparedness items, there were three key indicators driving this significant difference. There was no difference between volunteers and non-volunteers in their perceived ability to use health coaching skills in their work, or their perception that they can do the work well and are making a difference through the work (p > 0.05, data not shown). However, volunteers scored significantly lower on preparedness to do the work (p < 0.01), to use MI skills (p = 0.04), and to handle difficult situations (p < 0.01) compared to CI/CT staff who did not volunteer (data not shown).

Discussion and Lessons Learned

Federal, state, and local health agencies had to act swiftly to train a competent workforce of case investigators and contact tracers to effectively fight the spread of COVID-19 in the U.S. Recruitment and training efforts varied widely (9), and few studies to date have described the workforce or evaluated the effectiveness of these training programs. This evaluation suggests that individuals without health backgrounds and those without bachelor's or advanced degrees can be rapidly trained and deployed to conduct disease investigation duties, and provides support for expanding and diversifying the public health workforce pipeline.

One key finding was that there was no difference in preparedness by educational background. Approximately one-third of all VTA trainees and one-quarter of survey respondents who had been deployed as CIs/CTs had less than a bachelor's degree. However, they were similarly confident in their interviewing proficiency and their ability to perform CI/CT duties as well as more diverse as those with college or graduate degrees. This has important implications for strategic development of the public health workforce, one of which is the value of non-4-year college degree training programs (i.e., accelerated “pipeline” programs) to train entry-level public health workers. A recent survey of staff in State Health Agencies found that 75% of the workforce had college degrees (10), which reflects a significant barrier to entry. Accelerated pipeline programs not only increase the speed at which the public health workforce can be trained, but also can increase the racial and ethnic diversity of the workforce to reflect and represent the diverse populations served by public health departments, better equipping them to address health disparities (11, 12). In addition, these findings provide a compelling case for health departments to leverage current federal funding opportunities—such as the American Rescue Plan—to invest in community health workers who may not have advanced education but who have community knowledge and shared lived experience (13).

While there were no differences in preparedness by education, there were statistically significant differences by career background, with trainees who entered the workforce from public health or allied health fields feeling more prepared to perform CI/CT work. There are several potential factors that may contribute to these differences. First, health professionals may work in settings where they perform job duties similar to CIs/CTs, including assessing medical histories and providing referrals to social support services. In addition, there are several “soft skills” that are critical to the success of disease investigation work which may not be core functions of some non-health careers, including establishing rapport and building trust. The importance of these skills are demonstrated by a survey conducted by the California Health Care Foundation which found that about 4 in 10 Californians were unwilling or potentially unwilling to share personal information about their health, movements, and contacts with public health officials (14). However, though differences were statistically significant, the practical significance was small (<2-point differences on 30-point composite scores), so the implications for public health practice and outcomes may be minimal. Furthermore, certain non-health fields do have considerable transferrable skills that make them particularly adept at performing disease investigation duties. As an example, anecdotal evidence from the COVID-19 pandemic response suggests that redirected staff such as librarians and tax assessors are skilled in contact tracing work because they are accustomed to having conversations about sensitive and difficult topics (15). Thus, for future public health emergencies, these findings suggest that when the non-health workforce is critical to expanding public health capacity, there may be benefit to targeted recruitment of fields with key transferrable skills relevant to the emergency response duties.

This evaluation revealed two areas in which deployed trainees were slightly less confident in their abilities. The first was handling difficult situations, which was the only job preparedness item with a score that trended toward neutral, or uncertainty of whether they were prepared. While the VTA skill development labs provided opportunities for trainees to practice interviews, additional sessions with more complex scenarios may be warranted to improve self-efficacy before deployment. The second area with comparatively lower self-perceived proficiency was the ability to refer contacts for social support services. While the VTA provided training on how to elicit information to determine whether cases may need social support services, the actual availability of such services and the process by which to refer could vary widely across jurisdictions and is therefore contextual knowledge better suited for local training post-deployment. In addition, this may reflect the reality that the capacity of social safety net programs were often stretched during the COVID-19 pandemic due to increased demand, as well as the need for programs to adapt their service provision models to protect their workers and clients (16).

One unexpected finding was that individuals who reported that they volunteered to join the contact tracing workforce had lower self-perceived preparedness compared to those who were assigned to the role. Our a priori hypothesis was that volunteers may have greater enthusiasm about their role and more active engagement in the training program, which may translate into greater preparedness to perform contact tracing work. In contrast, we actually found that volunteers had lower self-perceived job preparedness than non-volunteers, driven by significantly lower perceptions of their overall preparedness to do CI/CT work, to apply MI skills, and to handle difficult situations. One hypothesis, derived from theories of cognitive bias (17), is that volunteers may have underestimated their competence because they placed greater importance on the work. Because individuals who volunteered are likely to be highly motivated and committed to performing the job well, they may have higher expectations for themselves than those who were assigned. Another important consideration is that redirected state department staff, who were primarily assigned, had additional opportunities for support provided by CDPH and the VTA, including access to a Mentorship Team of experienced CIs, as well as virtual Communities of Practice to continue to build knowledge and skills.

Limitations

One limitation of this study is potential non-response bias due to the low response rate. It is possible that those who responded to the survey had different perceptions about their preparedness compared to those who did not. One challenge was the inability to incentivize survey completion, as employees of the State of California are not permitted to receive gift cards or other forms of incentives. In addition, there were limitations with both methods of survey administration. While we moved to Qualtrics to limit barriers to accessing the survey, in using Qualtrics to administer subsequent program evaluation surveys there were reports that some LHD e-mail servers block or quarantine e-mails sent from the Qualtrics platform, suggesting that some trainees may not have received the survey invitation. Finally, California experienced a significant surge in COVID-19 cases between November 2020–February 2021, with CTs in some counties receiving up to 4,000 call assignments per day (18, 19). Increased caseloads and subsequent burnout likely reduced their capacity to respond to the survey in a timely manner.

Another limitation is the potential for recall bias, namely that as length of CI/CT deployment increases, trainees' knowledge and skills improve and they have difficulty recalling their preparedness at initial deployment. However, we attempted to limit the potential for recall bias by administering the survey within 3 weeks post course completion. Finally, this evaluation reflects self-perceived preparedness, not competency, so more research is necessary to assess how the VTA training impacted California's response to the COVID-19 pandemic. Additional priorities for future research include understanding how demographic characteristics of trainees, including race/ethnicity and languages spoken, impacted the perceived preparedness and performance of California's CI/CT workforce, particularly given the disparate impact of COVID-19 on communities of color (20). Despite potential limitations, this evaluation provides valuable insight into trainees' perception of their preparedness to work as CIs/CTs following a weeklong virtual training program.

Conclusion

The COVID-19 pandemic emphasized the critical need to reimagine who delivers essential public health services in the U.S. In order to engage effectively with community members, a contact tracing workforce needs to be empathetic, language concordant, and culturally responsive (21, 22). To achieve this goal, there must be focused efforts to diversify the public health workforce, including recruitment of professionals from the communities they will serve and from backgrounds underrepresented in the field. This evaluation provides evidence that individuals outside of the traditional public health pipeline, including those without bachelor's or advanced degrees and those without health or public health experience, can be rapidly trained and deployed and feel confident and prepared to perform disease investigation duties, lending critical surge capacity at an essential time.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Author Contributions

KW, MR, AD, and MP contributed to the conception and design of the project. MW, MF, OG, and SS contributed to the conceptualization of the paper and review and synthesis of the literature. OG and SS contributed to acquisition of data. MW, OG, KT, and SS contributed to analysis and/or interpretation of the data. MW and MF wrote the first draft of the manuscript. All authors contributed to critical review and revision of the manuscript.

Funding

This work was supported by the California Department of Public Health [agreement number 19-11102].

Author Disclaimer

The contents may not necessarily reflect the official views or policies of the State of California.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

We wish to acknowledge the many VTA staff and learners who have been committed to improving the lives of Californians affected by COVID-19.

References

1. Centers for Disease Control Prevention. Interim Guidance on Developing a COVID-19 Case Investigation & Contact Tracing Plan (2022). Available online at: https://www.cdc.gov/coronavirus/2019-ncov/php/contact-tracing/contact-tracing-plan/overview.html (accessed May 4, 2022).

Google Scholar

2. Spencer KD, Chung CL, Stargel A, Shultz A, Thorpe PG, Carter MW, et al. COVID-19 case investigation and contact tracing efforts from health departments — United States, June 25–July 24, 2020. MMWR Morb Mortal Wkly Rep. (2021) 70:83–7. doi: 10.15585/mmwr.mm7003a3

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Kretzschmar ME, Rozhnova G, Bootsma MCJ, van Boven M, van de Wijgert JHHM, Bonten MJM. Impact of delays on effectiveness of contact tracing strategies for COVID-19: a modelling study. Lancet Public Heal. (2020) 5:e452–9. doi: 10.1016/S2468-2667(20)30157-2

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Ruebush E, Fraser MR, Poulin A, Allen M, Lane JT, Blumenstock JS. COVID-19 case investigation and contact tracing: early lessons learned and future opportunities. J Public Heal Manag Pract. (2021) 27:S87–97. doi: 10.1097/PHH.0000000000001290

PubMed Abstract | CrossRef Full Text | Google Scholar

5. National Association of County & City Health Officials. NACCHO Position Statement: Building COVID-19 Contact Tracing Capacity in Health Departments to Support Reopening American Society Safely (2020). Available online at: https://www.naccho.org/uploads/full-width-images/Contact-Tracing-Statement-4-16-2020.pdf (accessed April 30, 2021).

Google Scholar

6. California Department of Public Health California California Department of Human Resources. Contact Tracer Assignment Information for State Departments (2020). Available online at: https://www.calhr.ca.gov/Documents/contact-tracer-assignment-information-for-state-departments.pdf (accessed May 12, 2021).

Google Scholar

7. Brickley DB, Forster M, Alonis A, Antonyan E, Chen L, DiGiammarino A, et al. California's COVID-19 virtual training academy: rapid scale-up of a statewide contact tracing and case investigation workforce training program. Front Public Heal. (2021) 9:1151–61. doi: 10.3389/fpubh.2021.706697

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Golston O, Prelip M, Brickley DB, Cass A, Chen L, Dorian A, et al. Establishment and evaluation of a large contact tracing and case investigation virtual training academy. Am J Public Health. (2021) 111:1934–8. doi: 10.2105/AJPH.2021.306468

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Lent M, Higgins E, Rosenthal J. How States Are Ramping Up Their COVID-19 Contact Tracing Capacity – The National Academy for State Health Policy (2020). Available online at: https://www.nashp.org/how-states-are-ramping-up-their-covid-19-contact-tracing-capacity/ (accessed May 13, 2021).

Google Scholar

10. Leider JP, Harper E, Bharthapudi K, Castrucci BC. Educational attainment of the public health workforce and its implications for workforce development. J Public Heal Manag Pract. (2015) 21(Suppl. 6):S56–68. doi: 10.1097/PHH.0000000000000306

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Bouye KE, McCleary KJ, Williams KB. Increasing diversity in the health professions: reflections on student pipeline programs. J Heal Sci Humanit. (2016) 6:67–79.

PubMed Abstract | Google Scholar

12. Coronado F, Beck AJ, Shah G, Young JL, Sellers K, Leider JP. Understanding the dynamics of diversity in the public healthworkforce. J Public Heal Manag Pract. (2020) 26:389–92. doi: 10.1097/PHH.0000000000001075

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Chokshi DA. People and places for the future of public health. JAMA Heal Forum. (2021) 2:e210423. doi: 10.1001/jamahealthforum.2021.0423

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Stremikis K. 75% of Californians Support Shelter in Place “As Long as Needed.” (2020). Available online at: https://www.chcf.org/wp-content/uploads/2020/04/CACOVID19StatewideSurveySupportShelterInPlace04242020.pdf (accessed May 19, 2021).

Google Scholar

15. Dembosky A. California Taps Librarians, Other City Workers To Trace Coronavirus Cases : Shots - Health News : NPR. Natl Public Radio (2020). Available online at: https://www.npr.org/sections/health-shots/2020/06/14/874088597/as-california-trains-20-000-contact-tracers-librarians-and-tax-assessors-step-up (accessed May 19, 2021).

Google Scholar

16. Holliday S, Hunter S, Dopp A, Chamberlin M, Iguchi M. Exploring the Impact of COVID-19 on Social Services for Vulnerable Populations in Los Angeles: Lessons Learned from Community Providers. RAND Corporation (2020).

Google Scholar

17. Dunning D. The dunning-kruger effect. On being ignorant of one's own ignorance. Adv Exp Soc Psychol. (2011) 44:247–96. doi: 10.1016/B978-0-12-385522-0.00005-6

CrossRef Full Text | Google Scholar

18. Becker J. This contact tracer is fighting two contagions: the virus and fear - The New York Times. The New York Times (2020). Available online at: https://www.nytimes.com/2020/08/09/us/california-contact-tracing.html (accessed December 10, 2021).

Google Scholar

19. California Department of Public Health. COVID Data and Tools. Available online at: https://www.cdph.ca.gov/Programs/CID/DCDC/Pages/COVID-19/CovidDataAndTools.aspx. (accessed December 22, 2021).

Google Scholar

20. Karaca-Mandic P, Georgiou A, Sen S. Assessment of COVID-19 hospitalizations by race/ethnicity in 12 states. JAMA Intern Med. (2021) 181:131–4. doi: 10.1001/jamainternmed.2020.3857

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Reid M, Enanoria W, Stoltey J, Philip S, Fuchs J, Lockwood A, et al. The SARS-CoV-2 pandemic: the race to trace: contact tracing scale-up in San Francisco—early lessons learned. J Public Health Policy. (2021) 42:211. doi: 10.1057/s41271-021-00285-y

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Celentano J, Sachdev D, Hirose M, Ernst A, Reid M. Mobilizing a COVID-19 contact tracing workforce at warp speed: a framework for successful program implementation. Am J Trop Med Hyg. (2021) 104:1616–19. doi: 10.4269/ajtmh.20-1665

PubMed Abstract | CrossRef Full Text | Google Scholar