Evaluation of online videos to engage viewers and support decision-making for COVID-19 vaccination: how narratives and race/ethnicity enhance viewer experiences

Background: Vaccine hesitancy has hampered the control of COVID-19 and other vaccine-preventable diseases.

Methods: We conducted a national internet-based, quasi-experimental study to evaluate COVID-19 vaccine informational videos. Participants received an informational animated video paired with the randomized assignment of (1) a credible source (differing race/ethnicity) and (2) sequencing of a personal narrative before or after the video addressing their primary vaccine concern. We examined viewing time and asked video evaluation questions to those who viewed the full video.

Results: Among 14,235 participants, 2,422 (17.0%) viewed the full video. Those who viewed a personal story first (concern video second) were 10 times more likely to view the full video (p < 0.01). Respondent–provider race/ethnicity congruence was associated with increased odds of viewing the full video (aOR: 1.89, p < 0.01). Most viewers rated the informational video(s) to be helpful, easy to understand, trustworthy, and likely to impact others' vaccine decisions, with differences by demographics and also vaccine intentions and concerns.

Conclusion: Using peer-delivered, personal narrative, and/or racially congruent credible sources to introduce and deliver vaccine safety information may improve the openness of vaccine message recipients to messages and engagement.

Introduction

Vaccine hesitancy and its role in vaccine uptake and also the subsequent control of vaccine-preventable diseases have become a major focus of research and practice (1). Since the convening of WHO's Strategic Advisory Working Group (SAGE) on vaccine hesitancy (2012) and the group's published definition of vaccine hesitancy in 2015 (2), researchers and practitioners in the vaccine community have continued to propose frameworks for measuring (3) and testing strategies to address vaccine hesitancy (3–7). In light of rising attention and efforts to address vaccine hesitancy over the past decade, vaccine hesitancy was formally recognized—prior to the start of the COVID-19 pandemic—as one of the top 10 threats to global health and security (8, 9).

The acceptance or refusal of COVID-19 vaccination—despite the widespread availability of vaccines—has been hampered by beliefs that COVID-19 does not present a serious health risk and a variety of concerns related to vaccine effectiveness and safety (10, 11). Along with sociodemographic factors (e.g., sex, education, race/ethnicity, and age), political affiliation, trust in public health authorities, and receiving the influenza vaccine in the prior year have been identified factors of COVID-19 vaccination (10–13). Despite a plethora of publications on vaccine hesitancy and the piqued interest of both experts and the public, to the best of our knowledge, there are few examples of rigorously tested vaccine communication strategies and interventions that have increased vaccine acceptance (14–16).

Experts agree that addressing vaccine hesitancy is context-specific, requiring tailored interventions that include a range of vaccine communication strategies (7). Patient–provider race/ethnicity concordance—defined as the occurrence of matching patient race/ethnicity and provider race/ethnicity—has been associated with an increased likelihood of care-seeking and continued care-seeking behaviors (17), as well as better patient–provider communication (18). Moreover, in science communication, listeners have been found to delay or not develop counter-arguments when listening to peer/personal narratives (19).

Our study was conducted as a part of the CDC-funded COVID-19 Vaccines Information Equity and Demand Creation (COVIED) program (20–22), a body of work designed to increase COVID-19 vaccination through the use of evidence-based, context-specific/tailored messaging. Based on our previous study on tailored vaccine education using racial/ethnic and gender diverse clinicians as credible sources and animation as a vehicle for conveying vaccine information (23), we conducted an internet-based, quasi-experimental study to evaluate the performance of 11 animated informational vaccine animation videos to address common vaccine attitudes and beliefs. We aimed to (1) evaluate the effect of using peer/personal narrative introductions (24) and (2) examine the role of race/ethnic congruence between the survey participant and a credible source (25) on the viewer engagement and their subsequent evaluation of the animated vaccine video intervention, randomizing on both video characteristics. We hypothesized that (1) introducing COVID-19 vaccine information with a personal narrative and (2) race/ethnic congruence between the survey participant and a credible source would be associated with an improvement in survey participant engagement, including an increase in the time spent viewing video content and positive ratings of video content. Additionally, we explored whether a credible source or participant's race/ethnicity was independently associated with viewer engagement and their evaluation of the intervention.

Methods

Study design

Using real-time interactive worldwide intelligence's (RIWI) patented Random Domain Intercept Technology (RDIT) (20–22), we implemented a national-level quasi-experimental design to evaluate 11 animated vaccine information videos with three variations of each based on different credible sources (i.e., clinical providers differentiating by race/ethnicity) who introduced and concluded each of the videos. A personal story video—narrated by an average peer of survey respondents, i.e., not a clinical provider—was created to precede or succeed eight (of the 11) videos that provided information on a common vaccine concern.

Development of intervention

We developed content tailored to each of the following relevant sub-populations of COVID-19 vaccine decision-makers: primary caregivers of children (i.e., <18 years of age) who have concerns about COVID-19 vaccines for their children, primary caregivers who do not have concerns about COVID-19 vaccines, non-caregivers who have concerns about COVID-19 vaccines for themselves, and non-caregivers who do not have concerns about COVID-19 vaccines. Formative research for our study was conducted using analyses of RIWI RDIT-derived data from two other national-level rapid response surveys designed to ascertain the public's COVID-19 vaccination attitudes and beliefs (21, 22), rapid formative ethnography to elicit insights from sub-populations on the root causes and other related influencing factors of reluctance (and the intention) to receive a SARS-CoV-2 vaccine, and continuous message development and testing. Based on a similar process described elsewhere (23), we collaborated with a scriptwriter to develop evidence-driven video content grounded in the insights gained for each sub-population during our formative phase. We developed 11 animated vaccine informational videos that were refined through an iterative process between the scriptwriter and scientists to ensure content appropriateness, messaging true to current scientific knowledge, and application of defined behavioral theories (23).

All videos included an introduction and concluding message by a clinical provider as well as an animated informational video. Introduction and concluding recordings were performed by three clinicians representing different racial and ethnic backgrounds (Black, White, and Hispanic). The use of a personal story to introduce (or conclude) eight informational videos on COVID-19 vaccine concerns was developed based on the theory of change (26) that by establishing empathy and credibility and briefly addressing specific concerns followed by conveying disease risk and vaccine effectiveness, attitudes toward COVID-19 vaccination would become more positive. The effect is hypothesized to be greater if the message is introduced by a strong and personalized recommendation from a clinical provider (27).

Data collection

We implemented the quasi-experiment from 06 December 2021 to 01 January 2022. We collected participant sociodemographic characteristics and determined video intervention eligibility based on answers to five questions regarding (1) caregiver status, (2) COVID-19 vaccination status, (3) race and ethnicity, (4) presence of COVID-19 vaccination concerns, and (5) intention to get the COVID-19 vaccine among the unvaccinated respondents. Based on answers to these questions, participants were assigned to appropriate message pathways (Supplementary Figure 1). Supplementary Table 1 presents all 11 possible response-specific videos.

Video assignment

Child caregivers were stratified by those with any COVID-19 vaccine concern and those without a concern. All caregivers without any concern were assigned to view a video discussing benefits for the child (Child Benefit video). Caregivers who had a concern about the COVID-19 vaccine and infertility were randomly (3:1) assigned to view a concern video addressing their concern about infertility or the child benefit video. The remaining concerned caregivers were assigned to the child benefit video.

Among non-caregivers, any unvaccinated participant without COVID-19 vaccine concerns was assigned to a video discussing benefits for adults (adult benefit video). Vaccinated non-caregivers with a previous COVID-19 vaccine concern as well as any unvaccinated non-caregivers with a COVID-19 vaccine concern were asked a multiple-choice question—“what are/were your main concerns about the COVID-19 vaccine?”—and could select all that applied. Based on anticipated sample size limits, any participant indicating concern about vaccine ingredients (specifically, fetal cell lines) was automatically assigned to the concern video addressing this specific concern. Participants selecting only one concern were assigned to the associated concerned video. Participants who selected more than one concern (not about fetal cell lines) were randomly assigned to a concern video addressing one of the selected concerns. Participants who selected “other” concerns were assigned to view the adult benefit video. In order to test our theory of change using personal narrative to introduce vaccine risk and safety messages, one out of every eight participants assigned to view a concern video was randomly selected to view a personal story video before viewing the concern video (i.e., seven out of every eight viewed the personal story after).

Unvaccinated non-caregivers without concern about the COVID-19 vaccine were assigned to view the adult benefit video. Clinical provider race/ethnicity was randomly assigned (with equal probability) for all videos. Vaccinated non-caregivers without previous COVID-19 vaccine concerns were asked to participate in a brief survey about COVID-19 vaccine boosters.

Outcomes of interest

We used a continuous measure of the length of time (in seconds) each respondent spent viewing their assigned video and a standardized measure of the proportion of the video viewed based on the total length of each video. The total length included the introduction, informational animation, and concluding message. We then created a dichotomous outcome variable identifying those who fully viewed their assigned video and those who did not. For respondents viewing a concern video, the total length of viewing time included both the concern video and the personal story video.

Participants who completed viewing their assigned video were asked to provide their level of agreement or disagreement using a five-point Likert scale (strongly agree, agree, neither agree nor disagree, disagree, and strongly disagree) with three statements to evaluate their assigned video:

(1) The video was helpful in making a vaccine decision.

(2) I trusted the information in the video.

(3) The video was easy to understand.

Viewers were asked to answer a fourth evaluation question (“how would this video influence others to get vaccinated?”) using a five-point Likert scale (much more likely, somewhat more likely, no impact, somewhat less likely, and much less likely). We opted for a neutral phrasing of the question to avoid any appearance of favor toward or against vaccination. Likert scales were dichotomized for analysis by combining “strongly agree” and “agree” or “much more likely” and “somewhat more likely” to form agree vs. disagree and likely vs. unlikely categories, respectively.

Exposures of interest

The main exposure of interest was a categorical variable classifying respondents according to their assigned video type. For those assigned to any of the concern videos, separate categories were used to indicate whether the personal story video or concern video was viewed first. Sociodemographic characteristics of respondent age, sex, race and ethnicity, and COVID-19 vaccination status (vaccinated vs. unvaccinated), as well as provider's race and ethnicity, were measured. We created a dichotomous variable to indicate racial/ethnic congruence between the provider and the respondent. Respondent location was categorized according to nine regions using Centers for Disease Control and Prevention (CDC) definitions (28).

Statistical analysis

For descriptive analyses, we compared distributions using chi-square and Fisher's exact tests for proportions, Wilcoxon rank-sum for non-parametric data, and Student's t-test for normally distributed data. No covariates had missingness of data exceeding 5%. Analyses were performed using the two-sided significance level (0.05). All analyses were conducted using Stata 16.1 (StataCorp, College Station, TX).

We used multivariable logistic regression to model the log odds of each of our dichotomous outcomes of interest: (1) viewing the assigned video in its entirety vs. not, and the respondent agreed (or disagreed) the following of the video; (2) is easy to understand; (3) is helpful for making vaccination decisions; (4) information is trustworthy; and (5) will influence others to get vaccinated. Table 1 shows video and respondent characteristics included in a backward stepwise selection process using 0.2 as the level of significance. We reviewed both the statistical significance of a fixed term used to control for survey date and interaction terms between provider–respondent racial congruence and provider race (and secondly, respondent race), and the results of a likelihood ratio test comparing extended and nested models that included (and subsequently excluded) a survey data fixed term. We reviewed the residual plots as well as Hosmer–Lemeshow chi-square estimates to evaluate the model goodness of fit. We used a robust variance estimator to adjust for clustering on respondent location (region).

TABLE 1

Table 1. Demographic, survey, and COVID-19 vaccination characteristics among non-caregivers, stratified by completely viewing the assigned video or not.

Results

Study population characteristics—descriptive analysis

Among 117,750 individuals who initially reached for participating, 75,616 (64.2%) completed the first five qualifying questions required for video assignment and 14,235 (18.8%) started to view the assigned video, allowing us to evaluate the length of time spent viewing the assigned video (e.g., the proportion of viewers who completed viewing their assigned video). Among those who started, 2,422 (17.0%) completed viewing the full video and answered at least one video and content quality or potential video utility evaluation question.

Among respondents assigned to view a concern and personal story video (4,043 non-caregivers with a COVID-19 vaccine concern), distributions of age, sex, race and ethnicity, COVID-19 vaccination status, COVID-19 booster status among the vaccinated, and intention to get the COVID-19 vaccine among the unvaccinated differed significantly by video viewing completion status (Table 1, left; all p-values < 0.01). Specifically, we observed higher proportions of video completion compared to demographic counterparts among respondents aged 46–55 and 56–64 years of age (19 and 21%, respectively), women (15.8%), self-reporting race and ethnicity as Hispanic/Latinx or Multiple (15–21%), those who were COVID-19 unvaccinated (19%), those who were COVID-19 vaccinated who are planning to get a COVID-19 booster (14%), and those who were COVID-19 unvaccinated and hesitant (delaying or refusing vaccination) (22%).

Among adult benefit video viewers (4,116 non-caregivers with or without a COVID-19 vaccine concern), we observed statistically significant differences in the distributions of age, race and ethnicity, and COVID-19 booster status among the vaccinated by video viewing completion status (Table 1, right; all p-values < 0.01). Mainly, those who were 36–74 years of age (~17%), self-identified race and ethnicity as Multiple (18%), Black (17%), and White or Hispanic/Latinx (15%) and were vaccinated but had not yet received the first COVID-19 booster but planned to (19%) had higher proportions of viewing compared to their younger (12–13%), Asian (9%), Alaskan Native (12%), American Indian (12%), and COVID-19 vaccinated (and have received a booster) (12%) counterparts (Table 1, right).

In Table 2 (left), among caregivers assigned to view the infertility concern video (n = 324), mostly 5–11-year-old children (36%) completed viewing than those aged 0–4 (23%) or 12–17 (18%) years (p = 0.02). No other significantly different distributions were found by video viewing completion status. Distributions of age (p < 0.01), sex (p = 0.01), and race and ethnicity (p < 0.01) differed significantly by child benefit video viewing completion status among 5,752 respondents where greater proportions of 36–64-year-olds (~9%), female (8%), and White (9%), Black (7%), Multiple (7%), Hispanic (7%), or American Indian (7%) completed viewing the video compared to those younger and older than the middle aged (~5%), male (7%), and Asian, Alaskan Native, or American Indian (all 5% completed viewing; Table 2, right).

TABLE 2

Table 2. Demographic, survey, and COVID-19 vaccination characteristics among caregivers, stratified by completely viewing the assigned video or not.

Evaluation of viewing completion

In Table 3, the odds of completing the view of the assigned video are adjusted for the assigned video, provider race and ethnicity, and respondent race and ethnicity, sex, age, and COVID-19 vaccination status (vaccinated vs. unvaccinated). When compared to those who viewed the assigned concern video first (personal story second), viewers of the personal story first (concern video second) were ~10 times (p < 0.01) more likely to watch the complete informational animation. Viewers of the personal story first (concern video second) were also ~4.5 times (p < 0.01) more likely to watch the complete informational animation compared to viewers of the infertility concern video, as well as 20 times (p < 0.01) and nine times (p < 0.01) more likely to watch than the viewers of the child benefit and adult benefit videos, respectively. Unadjusted estimates of viewing time measures are provided in Supplementary Tables 2, 3.

TABLE 3

Table 3. Association between fully viewing the assigned video or positive opinions of the viewed informational video and video and viewer sociodemographic characteristics.

Those viewing introduction/concluding material presented by the Black or Hispanic physician had lower odds of fully viewing the assigned video (58%, p < 0.01; 82%, p < 0.01) than those viewing videos presented by the White physician (Table 3). There was title difference between the adjusted and crude odds ra8os for respondent sex, age, race and ethnicity, and COVID-19 vaccina8on status except that viewers who self-iden8fied as American Indian (AI), Alaskan Na8ve (AN), or Other had lower odds of fully viewing the assigned video (27%, p < 0.01) compared to their White counterparts as did Asian viewers (40% lower, p < 0.01). Those reporting a multi-racial and/or ethnic identity had greater odds of completely viewing the assigned video (aOR 1.30, 95% CI 0.99, 1.71) though the relationship had borderline statistical significance (p = 0.06). Women aged 36–74 years and unvaccinated viewers had increased odds of fully viewing the assigned video compared to their men aged 18–35 years and vaccinated counterparts (aOR_female: 1.29; aOR_{36−74−years−old} range: 1.44–1.76; aOR_Unvaccinated: 1.32; all p < 0.01).

When adjusting for racial congruence between the provider and the viewer/respondent (Table 4), the odds of fully viewing the assigned video were lower for all videos compared to personal story video first (concern video second) viewers (all p-values < 0.01). Racial congruence was associated with increased odds of fully viewing a video (aOR: 1.89, p < 0.01).

TABLE 4

Table 4. Association between fully viewing the assigned video or positive opinions of the viewed informational video and video and viewer sociodemographic characteristics that include provider-respondent racial congruence.

Evaluation of video content (post-viewing)

Overall, the odds of evaluating the video positively (easier to understand, helpful for making vaccination decisions, providing trusted information, and influencing others to get vaccinated) were greater among those who viewed the personal story first (concern video second) though the adjusted odds ratio was statistically significant only when asking about the video's influence on others to get vaccinated (1.6 times greater odds, p < 0.01; Table 3). Independently, the provider's race was not significantly associated with a positive evaluation of the videos, whereas respondent race/ethnicity was significantly associated with positive evaluation, and mainly Black, Asian, and Hispanic/LatinX have greater odds (aOR: 1.4–2.1 where p < 0.01) of evaluating the videos positively [and American Indian or Alaskan Native (AI/AN)] or others have a lower odds (≤46% lower odds where p < 0.01) than their White counterparts (Table 3). Racial congruence was associated with increased odds of evaluating the video as influencing others to get vaccinated (Table 4; aOR: 1.14, p = 0.03). When adjusting for racial congruence, female viewers had greater odds (1.3 greater odds compared to males, all p ≤ 0.03) of positively evaluating their assigned videos whereas older adults (75+ years of age; ≤53% lower odds where p ≤ 0.02) and the unvaccinated ones had lower odds (≤84% lower odds where p < 0.01; Table 4).

Sub-analyses—potential confounding and effect modifying effects

There was no evidence of a confounding effect of survey time on the primary relationship of interest (p = 0.97). Moreover, we found no evidence that provider race (or respondent race) acts as an effect modifier on the relationship between provider–respondent racial congruence and our outcomes of interest (p > 0.40 and p > 0.30 for provider and respondent race, respectively).

Discussion

Using peer-delivered, personal narratives to introduce vaccine safety information may increase the likelihood that viewers will engage with informational vaccine videos. Despite fewer clear benefits in the likelihood of videos receiving positive evaluations, personal story videos had a consistently improved effect on the likelihood that viewers thought their respective videos would influence others to get vaccinated. Emotional engagement—an important part of communication strategies developed to engage the public for fostering vaccine confidence—has been a central part of health behavior change research and practice (23, 29, 30). Emotional engagement and transparent communication likely serve as important tools for messaging and vaccination program administration, particularly during periods of heightened collective and diverse emotions among the public such as the COVID-19 pandemic (30). Moreover, the use of personal narratives may foster learning environments of openness (open-mindedness) (31) for vaccine messages designed to build general confidence and understanding of evidence-based medicine (32). In light of these findings and previous research that has highlighted the polarization of social media content between positive and negative-toned content on vaccines, especially vaccine-hesitant topics (33), our findings on the influence of peer-delivered, personal narrative to introduce vaccine safety information may be an important area of further research.

Our findings suggest the importance of racial congruence between the patient and the provider in vaccine safety communication and also further support the importance of identifying sub-population attitudes (e.g., by race and ethnicity) and tailoring messages. Race and ethnicity have been identified as factors associated with COVID-19 (and other) vaccine uptake and vaccine hesitancy (25, 34–36). In general, healthcare providers who communicate effectively with patients are known influencers of vaccine uptake (25, 37, 38). However, some evidence supports that persons self-identifying as non-Hispanic White are more likely to receive a healthcare provider's recommendation than racial and ethnic minorities (37). Studies have shown that patients who are racially and/or ethnically concordant with their provider report greater satisfaction, levels of trust, and perceived quality of care (17). Although concordance has been found to affect patients' clinical encounter experiences and relates to better patient–physician communication, there is no general consensus on the positive effect of racial/ethnic patient–provider concordance on patient outcomes (nor specifically on effective vaccine communication) (18). Our study indicates support for this theory.

We found that while unvaccinated respondents were more likely to fully view their assigned video compared to vaccinated respondents, they were also less likely to give positive feedback on video content and usefulness. The majority of unvaccinated respondents (62%) agreed or strongly agreed that the video content was easy to understand, but greater proportions of unvaccinated respondents disagreed or strongly disagreed that the videos were helpful for making vaccine decisions or influencing others to get vaccinated or that the information was trusted than agreed or strongly agreed (69, 68, and 74%, respectively). Even before the COVID-19 pandemic, vaccine hesitancy and refusal were identified as public health concerns (4, 8), with emotion-driven vaccine beliefs spreading across (and even flourish) during the pandemic (30, 39). Regardless of their interest to view the content of our informational videos, deliberate efforts to engage viewers in a positive manner may rather activate emotions that decrease the likelihood of positive feedback (30). Resistance to vaccination is complex, and positive vaccine messages may have unintended and undesirable consequences (25, 40). Rigorous approaches to both measuring latent vaccination attitudes and beliefs and testing interventions for their effect on vaccination behavior (i.e., uptake) are needed and must take the psychology behind health decision-making into consideration (23, 25, 41, 42).

Limitations

Selection bias may have been introduced due to the opt-in, unincentivized design of the study. We explored the distribution of sociodemographic characteristics (Supplementary Table 4) among those who were lost to follow-up (i.e., early drop-out and did not get assigned to a video or did not start viewing the assigned video), noting that the proportion of these individuals are more vaccinated (77.3 vs. 55.0%, p < 0.01) and have fewer reported COVID-19 vaccine concerns than those described in our study sample (26.3 vs. 52.7%, p < 0.01). This may strengthen our findings that suggest the usefulness of our vaccine communication videos among specific sub-populations of the American public, noting that the distribution of intentions to complete the COVID-19 vaccine series or booster were similar between those lost to follow-up and those in our study (Supplementary Table 4). We note that our sample size may have limited our ability to identify statistically significant differences in the odds of viewing the entire assigned video among the multi-racial and/or ethnic subgroup compared to their racial/ethnic counterparts. The focus of this study was on the evaluation of the whole package of information and associations with demographic characteristics and vaccine intentions and concerns. Thus, we were not able to determine if vaccination rates improved in persons who viewed full videos compared to others. Further research is needed to evaluate this question.

Public health implications

Our study findings further support the importance of tailoring vaccine communication strategies to sub-population vaccine attitudes by delivering vaccine messages through trusted, race/ethnicity-congruent providers or other trusted health authorities. Introducing vaccine safety information with peer-delivered, personal narratives may improve the openness of vaccine message recipients to vaccine messages and engagement. Further research is needed to evaluate the effect of vaccine safety informational video packages on vaccine uptake. Additionally, further formative work is needed to explore message engagement among sub-populations that maintain fewer positive views of vaccine safety information.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

This study was reviewed by the Institutional Review Board at John Hopkins Bloomberg School of Public Health. Written informed consent and approval was not required as the study was determined to not be human subject research.

Author contributions

HS, RR, RB, PO, MD, L-SK, RS, SL, LW, JL, WO, JB, AJ, JS, RJ, DG, and DS contributed to the conception and/or the design of the work represented in this paper. RS and SL were a part of the acquisition of data. HS, RB, MD, RS, LW, WO, and DS contributed to the analysis and interpretation of data. HS and DS drafted the work. All authors revised the work critically and gave final approval of the version to be published.

Funding

Our work was conducted as a part of the CDC-funded COVID Vaccines Information Equity and Demand creation (COVIED) program, 20–22 a body of work designed to increase COVID-19 vaccination through the use of evidence-based, context-specific/tailored messaging. JS was supported by Ministry of Health of the Czech Republic, grant no. NU20-09-00454. All rights reserved.

Acknowledgments

The authors thank Ric LaGrange (6).

Conflict of interest

HS served as a (paid) health advisor to the University of Roehampton that provided guidance on recovery-building and future pandemic preparedness and understanding citizen engagement in the G7 in 2021–22 (during the presented study). MD reports research support from Merck. At the time of conducting this research, SL was an employee of, and RS was a consultant to, RIWI Corp—the company that owns the technology that was used to conduct the surveys. JL is a member of the Bioethics Advisory Council of Pfizer, Inc., New York. WO is an uncompensated member of the Moderna Scientific Advisory Board. JS is a consultant for Pfizer on meningococcal B vaccine. DS has served as a consultant and receives grant support from Merck, and served on advisory boards for Moderna, Sanofi, and Merck. L-SK was employed by Ideas42.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpubh.2023.1192676/full#supplementary-material

References

1. Larson HJ, Jarrett C, Eckersberger E, Smith DMD, Paterson P. Understanding vaccine hesitancy around vaccines and vaccination from a global perspective: a systematic review of published literature, 2007-2012. Vaccine. (2014) 32:2150–9. doi: 10.1016/j.vaccine.2014.01.081

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Macdonald NE, Group W. Vaccine hesitancy: definition, scope and determinants. Vaccine. (2015) 33:4161–4. doi: 10.1016/j.vaccine.2015.04.036

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Larson HJ, Jarrett C, Schulz WS, Chaudhuri M, Zhou Y, Dube E, et al. Measuring vaccine hesitancy: the development of a survey tool. Vaccine. (2015) 33:4165–75. doi: 10.1016/j.vaccine.2015.04.037

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Dubé E, Gagnon D, MacDonald NE, Hesitancy SWGoV. Strategies intended to address vaccine hesitancy: review of published reviews. Vaccine. (2015) 33:4191–203. doi: 10.1016/j.vaccine.2015.04.041

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Jarrett C, Wilson R, O'Leary M, Eckersberger E, Larson HJ, Hesitancy SWGoV. Strategies for addressing vaccine hesitancy—a systematic review. Vaccine. (2015) 33:4180–90. doi: 10.1016/j.vaccine.2015.04.040

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Butler R, MacDonald NE. Diagnosing the determinants of vaccine hesitancy in specific subgroups: the Guide to Tailoring Immunization Programmes (TIP). Vaccine. (2015) 33:4176–9. doi: 10.1016/j.vaccine.2015.04.038

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Macdonald NE, Butler R, Dubé E. Addressing barriers to vaccine acceptance: an overview. Hum Vaccin Immunother. (2018) 14:218–24. doi: 10.1080/21645515.2017.1394533

PubMed Abstract | CrossRef Full Text | Google Scholar

8. World Health Organization. Ten Threats to Global Health in 2019. World Health Organization News (2019). Available online at: https://www.who.int/newsroom/spotlight/ten-threats-to-global-health-in-2019 (accessed July 28, 2023).

Google Scholar

9. Hickler B, MacDonald NE, Senouci K, Schuh HB. Efforts to monitor Global progress on individual and community demand for immunization: Development of definitions and indicators for the Global Vaccine Action Plan Strategic Objective 2. Vaccine. (2017) 35:56. doi: 10.1016/j.vaccine.2017.04.056

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Wang Q, Yang L, Jin H, Lin L. Vaccination against COVID-19: a systematic review and meta-analysis of acceptability and its predictors. Prev Med. (2021) 150:106694. doi: 10.1016/j.ypmed.2021.106694

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Dudley MZ, Schwartz B, Brewer J, Kan L, Bernier R, Gerber JF, et al. COVID-19 vaccination status, attitudes, and values among US adults in September 2021. J Clin Med. (2022) 11:1–22. doi: 10.3390/jcm11133734

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Fridman A, Gershon R, Gneezy A. COVID-19 and vaccine hesitancy: a longitudinal study. PLoS ONE. (2021) 16:1–12. doi: 10.1371/journal.pone.0250123

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Dhanani LY, Franz B. A meta-analysis of COVID-19 vaccine attitudes and demographic characteristics in the United States. Public Health. (2022) 207:31–8. doi: 10.1016/j.puhe.2022.03.012

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Glanz JM, Wagner NM, Narwaney KJ, Shoup JA, McClure DL, McCormick EV, et al. A mixed methods study of parental vaccine decision making and parent—Provider trust. Acad Pediatr. (2013) 13:481–8. doi: 10.1016/j.acap.2013.05.030

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Drokow EK, Effah CY, Agboyibor C, Sasu E, Amponsem-Boateng C, Akpabla GS, et al. The impact of video-based educational interventions on cervical cancer, pap smear and HPV vaccines. Front Public Heal. (2021) 9:1–12. doi: 10.3389/fpubh.2021.681319

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Kaim A, Siman-Tov M, Jaffe E, Adini B. Effect of a concise educational program on COVID-19 vaccination attitudes. Front Public Heal. (2021) 9:767447. doi: 10.3389/fpubh.2021.767447

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Ma A, Sanchez A, Ma M. The impact of patient-provider race/ethnicity concordance on provider visits: updated evidence from the medical expenditure panel survey. J Racial Ethn Heal Disparities. (2019) 6:1011–20. doi: 10.1007/s40615-019-00602-y

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Shen MJ, Peterson EB, Costas-Muñiz R, Hernandez MH, Jewell ST, Matsoukas K, et al. The effects of race and racial concordance on patient-physician communication: a systematic review of the literature. J Racial Ethn Heal Disparities. (2018) 5:117–40. doi: 10.1007/s40615-017-0350-4

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Dudley MZ, Squires GK, Petroske TM, Dawson SBJ. The use of narrative in science communication: a scoping review. Patient Educ Couns. (2023) 112:107752. doi: 10.1016/j.pec.2023.107752

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Sargent RH, Laurie S, Moncada L, Weakland LF, Lavery JV, Salmon DA, et al. Masks, money, and mandates: a national survey on efforts to increase COVID-19 vaccination intentions in the United States. PLoS ONE. (2022) 17:1–15. doi: 10.1371/journal.pone.0267154

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Sargent RH, Laurie S, Weakland LF, Lavery JV, Salmon DA, Orenstein WA, et al. Use of random domain intercept technology to track COVID-19 vaccination rates in real time across the United States: survey study. J Med Internet Res. (2022) 24:37920. doi: 10.2196/37920

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Salmon DA, Schuh HB, Sargent RH, Konja A, Harvey SA, Laurie S, et al. Impact of vaccine pause due to Thrombosis with thrombocytopenia syndrome (TTS) following vaccination with the Ad26COV2S vaccine manufactured by Janssen/Johnson & Johnson on vaccine hesitancy and acceptance among the unvaccinated population. PLoS ONE. (2022) 17:e0274443. doi: 10.1371/journal.pone.0274443

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Salmon DA, Dudley MZ, Brewer J, Shaw J, Schuh HB, Proveaux TM, et al. LetsTalkShots: personalized vaccine risk communication. Front Public Health. (2023) 11:1195751. doi: 10.3389/fpubh.2023.1195751

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Hudson A, Montelpare WJ. Predictors of vaccine hesitancy: implications for COVID-19 public health messaging. Int J Environ Res Public Health. (2021) 18:158054. doi: 10.3390/ijerph18158054

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Dubé È, Ward JK, Verger P, Macdonald NE. Vaccine hesitancy, acceptance, and anti-vaccination: trends and future prospects for public health. Annu Rev Public Health. (2021) 42:175–91. doi: 10.1146/annurev-publhealth-090419-102240

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Reinholz DL, Andrews TC. Change theory and theory of change: what's the difference anyway? Int J Stem Educ. (2020) 7:3. doi: 10.1186/s40594-020-0202-3

CrossRef Full Text | Google Scholar

27. Salmon DA, Limaye RJ, Dudley MZ, Oloko OK, Church-Balin C, Ellingson MK, et al. MomsTalkShots: an individually tailored educational application for maternal and infant vaccines. Vaccine. (2019) 37:6478–85. doi: 10.1016/j.vaccine.2019.08.080

PubMed Abstract | CrossRef Full Text | Google Scholar

28. (CDC) C for DC and P. National Center for Chronic Disease Prevention and Health Promotion (NCCDPHP) Regions.

Google Scholar

29. Perugini M, Bagozzi RP. The role of desires and anticipated emotions in goal-directed behaviours: broadening and deepening the theory of planned behaviour. Br J Soc Psychol. (2001) 40:79–98. doi: 10.1348/014466601164704

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Chou WYS, Budenz A. Considering emotion in COVID-19 vaccine communication: addressing vaccine hesitancy and fostering vaccine confidence. Health Commun. (2020) 35:1718–22. doi: 10.1080/10410236.2020.1838096

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Cui Z. The impact of EFL teachers' open-mindedness and immediacy on their social intelligence: a theoretical review. Front Psychol. (2022) 13:1–5. doi: 10.3389/fpsyg.2022.872250

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Browne M, Thomson P, Rockloff MJ. Going against the herd: psychological and cultural factors underlying the ‘vaccination confidence gap.' PLoS ONE. 10:e0132562. doi: 10.1371/journal.pone.0132562

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Cascini F, Pantovic A, Al-Ajlouni YA, Failla G, Puleo V, Melnyk A, et al. Social media and attitudes towards a COVID-19 vaccination: a systematic review of the literature. eClinicalMedicine. (2022) 48:101454. doi: 10.1016/j.eclinm.2022.101454

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Viswanath K, Bekalu M, Dhawan D, Pinnamaneni R, Lang J, McLoud R. Individual and social determinants of COVID-19 vaccine uptake. BMC Public Health. (2021) 21:1–10. doi: 10.1186/s12889-021-10862-1

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Shih SF, Wagner AL, Masters NB, Prosser LA, Lu Y, Zikmund-Fisher BJ. Vaccine hesitancy and rejection of a vaccine for the novel coronavirus in the United States. Front Immunol. (2021) 12:1–8. doi: 10.3389/fimmu.2021.558270

PubMed Abstract | CrossRef Full Text | Google Scholar

36. He K, Mack WJ, Neely M, Lewis L, Anand V. Parental perspectives on immunizations: impact of the COVID-19 pandemic on childhood vaccine hesitancy. J Community Health. (2022) 47:39–52. doi: 10.1007/s10900-021-01017-9

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Hernandez ND, Daley EM, Young L, Kolar SK, Wheldon C, Vamos CA, et al. HPV vaccine recommendations: does a health care provider's gender and ethnicity matter to Unvaccinated Latina college women? Ethn Heal. (2019) 24:645–61. doi: 10.1080/13557858.2017.1367761

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Opel DJ, Mangione-Smith R, Robinson JD, Heritage J, DeVere V, Salas HS, et al. The influence of provider communication behaviors on parental vaccine acceptance and visit experience. Am J Public Health. (2015) 105:1998–2004. doi: 10.2105/AJPH.2014.302425

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Sear RF, Velasquez N, Leahy R, Restrepo NJ, Oud SE, Gabriel N, et al. Quantifying COVID-19 content in the online health opinion war using machine learning. IEEE Access. (2020) 8:91886–93. doi: 10.1109/ACCESS.2020.2993967

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Nyhan B, Reifler J. Does correcting myths about the flu vaccine work? An experimental evaluation of the effects of corrective information. Vaccine. (2015) 33:459–64. doi: 10.1016/j.vaccine.2014.11.017

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Buttenheim AM, Asch DA. Leveraging behavioral insights to promote vaccine acceptance: one year after disneyland. J Am Med Assoc Pediatr. (2016) 170:635–6. doi: 10.1001/jamapediatrics.2016.0192

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Rossen I, Hurlstone MJ, Lawrence C. Going with the grain of cognition: applying insights from psychology to build support for childhood vaccination. Front Psychol. (2016) 7:1–6. doi: 10.3389/fpsyg.2016.01483

PubMed Abstract | CrossRef Full Text | Google Scholar