Changes in COVID-19 vaccine hesitancy at different times among residents in Guangzhou, China

Background: Vaccination as a fundamental pillar of promoting public health and interest is critical to limiting the COVID-19 pandemic. However, many citizens are still hesitant about this epidemic prevention measure. This article aimed to understand the COVID-19 vaccination and hesitancy rates among Guangzhou residents at different points in time as well as to explore the relevant factors that cause vaccination hesitancy.

Methods: We conducted a total of nine cross-sectional surveys by enrolling 12,977 questionnaires among Guangzhou residents through the online survey software called “WenJuanXing” between April 2021 and December 2022, and residents made their choices by judging their willingness to vaccinate. These surveys collected data on the participants' sociodemographic characteristics, vaccination status, vaccine hesitancy, and factors influencing this hesitancy. The Chi-squared test was used for univariate analysis and the multivariate logistic regression model was used to further adjust the influence of the confounding factors to evaluate the main factors affecting the hesitancy of the COVID-19 vaccine at different periods.

Results: Over the course of 2021–2022, a total of 12,977 residents in the study area were surveyed. The vaccine hesitancy rates fluctuated over time. From April to June 2021, the vaccine hesitancy rate decreased from 30% to 9.1% and then increased to 13.7% in November. However, from April to December 2022, the hesitancy rate continued to rise from 13.4% to 30.4%. Vaccination rates, the epidemic waves of COVID-19, and changes in policies may all be possible factors that contributed to these fluctuations in vaccine hesitancy rates. We found statistically significant correlations between factors, such as residence, education, and occupation, and vaccine hesitancy at certain points of time. The results of the surveys in April and June 2021 showed that rural residents showed higher vaccine hesitancy rate than urban residents. Their lower education level was associated with higher vaccine hesitancy. Workers and farmers are more likely to have vaccine hesitancy than people with other occupations. The univariate analysis showed that people with underlying medical conditions and lower perceived health status were more likely to experience vaccine hesitation. Logistic regression analysis revealed that the health status of individuals is the most important factor leading to vaccine hesitancy, and residents' underestimation of domestic risks and overconfidence in personal protection measures were also contributing factors. At different stages, vaccine hesitancy among residents was related to vaccine side effects, safety and efficacy, convenience fluctuation, and various factors.

Conclusion: In the present study, we found that vaccine hesitancy did not display a consistent downward trend but it fluctuated over time. Higher education, residing in urban areas, lower perceived disease risk, and concerns about the vaccine's safety and side effects were risk factors for vaccine hesitancy. Implementing appropriate interventions and educational programs tailored to address these risk factors may prove to be effective in enhancing public confidence on vaccination.

Introduction

Severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2), which caused the COVID-19 pandemic in December 2019, has caused great harm to the physical and mental health of people as well as incurred financial losses around the world (1). In addition to preventive measures (such as social distancing, washing hands, and wearing face-masks) and treatment, vaccination has been proven to be an effective tool for managing public health as essential to curbing the spread of SARS-CoV-2/COVID-19 (2). Research has shown that (3), for a COVID-19 vaccine, assuming that the prescribed vaccine has the highest possible efficacy, the vaccination rate of the general public must be above 70% to achieve herd immunity (4). To achieve herd immunity, a sufficient proportion of the population must be vaccinated to prevent the spread of diseases among the population. Therefore, the willingness of the public to vaccinate is decisive in achieving herd immunity (5).

On 30 December 2020, the State Food and Drug Administration of China approved the conditional launch of Sinopharm's COVID-19 inactivated vaccine (6). However, with the launch of the COVID-19 vaccine, various negative reports concerning the vaccine have also been propagated on the internet, making people hesitant about the vaccine and even refusing to be vaccinated (7). According to the survey, public trust in vaccination has declined globally, for both COVID-19 and other vaccines, in general, leading to an increase in vaccine hesitancy (8, 9). Vaccine hesitancy was listed by the WHO in 2019 as one of the top ten threats to global health (10). There are many reasons for vaccine hesitancy, including a lack of knowledge about the importance and necessity of vaccines, doubts about vaccine safety and efficacy, mistrust of health providers and vaccine strategists, availability of vaccination, geographic location, and concerns for costs (11).

Vaccination rates may vary by time, region, and specific brand of vaccine; however, vaccine hesitancy will lead to a decline in overall vaccination rate inevitably, making it impossible to achieve or maintain herd immunity to protect those with vaccine contraindications or individuals who failed to develop an immune response (12). Declining vaccination rates increase the risk of vaccine-preventable disease outbreaks (13). An online survey of people over the age of 18 in France found that 26% of respondents would refuse a vaccine even if it were available (14). In surveys conducted in the UK and Canada, 9 and 14%, respectively, of respondents refused to be vaccinated against COVID-19 (15, 16). According to a national cross-sectional survey report by Peking University, 67.1% of participants in China are willing to receive the COVID-19 vaccine, while 9.0% of the participants declined vaccination altogether and 35.5% of participants reported vaccine hesitancy (5). Even with guaranteed vaccine availability, it is still challenging to convince a sufficient number of individuals to receive the vaccine due to people's hesitation about vaccines (17). A recent survey conducted among healthcare professionals and students identified fear of unforeseen future effects as the main reason for their hesitation to vaccinate (18). Additional studies exploring the acceptability of the anti-SARS-CoV-2 vaccine co-administered with the influenza vaccine found that healthcare workers were more motivated than other groups to receive the vaccine, and the anti-SARS-CoV-2 vaccine was more acceptable when administered in combination with the influenza vaccine (19).

Although the Chinese COVID-19 vaccination rate has exceeded 80%, the mutation of the virus remains a concern as studies have shown that people can still contract SARS-CoV-2 even after completing the complete vaccination process. Therefore, investigating vaccine hesitancy and its influencing factors is crucial to prevent another outbreak of the disease and ensure high vaccination rates for booster doses (4). Tang et al. (20) explored the protective effect of the vaccine against COVID-19 pneumonia caused by virus mutation in Henan Province. The results demonstrated that the protective effect of the vaccine began to decline 6 months after the initial vaccination, but the protective effect was restored through homologous vaccination upon vaccine readministration (20). Evidence shows that, while COVID-19 vaccines are less effective in preventing the spread of the disease, vaccines are effective in preventing symptomatic SARS-CoV-2 infection from developing into severe stages (21–23). From 2021 to 2022, we investigated changes in COVID-19 vaccine hesitancy and vaccination status among 12,977 residents in Guangzhou across different periods and analyzed the relevant factors leading to vaccine hesitancy. This main aims of the study are to provide guidelines for the government to develop targeted vaccination programs, reduce the rate of vaccine hesitation, and increase the acceptance and enthusiasm for vaccination among the population.

Materials and methods

Study design

We conducted a population-based cross-sectional online survey between April 2021 to December 2022 aimed at assessing the COVID-19 vaccine hesitancy and its influencing factors among Guangzhou residents. Nine rounds of surveys were conducted in this study and distributed to all residents living in Guangzhou. The questionnaire includes three aspects: demographic characteristics, COVID-19 vaccination status and hesitancy, and influencing factors. This study has a non-duplicate cross-sectional survey that randomly sampled mobile phone numbers registered in Guangzhou. The staff of the Guangzhou Center for Disease Control and Prevention sent questionnaires to residents' mobile phones through WeChat, SMS, and other forms from the 1st to the 15th of each month during the survey period. The participants were required to complete the survey questionnaires through mobile phones. For those residents who have already filled out questionnaires, they were be given questionnaires in the subsequent survey. Data were collected on Wen Juan Xing (like Amazon Mechanical Turk, Qualtrics, SurveyMonkey, or CloudResearch), which provides the function of designing questionnaires and surveys online.

Calculation of sample size

According to the formula for calculating the sample size of enumerating data from the current situation survey in epidemiological studies,

$\begin{array}{l}\text{N }=\text{ p}\times \text{q}/\text{S}{\text{p}}^2,\end{array}$

where N represents the sample size required for the investigation, p is the expected positive rate or prevalence rate of the investigation, q = 1-p, S_p = d/ Z_α, d is the allowable error, z is the boundary value of the standard normal distribution, and Z_α is the significance test statistic. For instance, when α = 0.05, Z_α = 1.96, and d = 0.1p is generally used, the formula for calculating the sample size can be rewritten as:

$\begin{array}{l}\text{N }=400\times \text{q}/\text{p}\end{array}$

Based on the available data, the ratio of the herd immunity level (pc) required to prevent transmission for a vaccine with 100% efficacy and lifelong protection is (1-1/R₀) in the population, where R₀ represents the basic reproductive number (24). Given that most countries had pre-lockdown R₀ values between 2.5 and 3.5, the required herd immunity would be approximately 60–72% (25). Since the vaccine is unlikely to be 100% effective, much higher vaccination rates are needed to ensure herd immunity. Therefore, a minimum vaccination rate of above 60% is required, and this value is further impacted by vaccine efficacy. Therefore, p = 0.5 is substituted into the formula, and this yields a sample size of 400. However, in consideration of the large number of urban residents in the survey, their strong health awareness, and their high willingness to vaccinate, the sample size was doubled. In addition, the sample size of each sample should not be <800 people.

Data collection

The nine questionnaires were completed by different people in a time series. We adopted the method of simple random sampling without replacement (i.e., each respondent had only one opportunity to participate) and randomly distributed the questionnaires to the mobile phone numbers registered by Guangzhou residents. Data were collected during the months of April to June and November in 2021 as well as April to June and November to December in 2022. We collected demographic data, including age, gender, place of residence, education, monthly income, and occupation, as well as information relating to vaccination status and vaccine hesitancy. We also investigated factors that contribute to vaccine hesitancy.

Vaccine hesitancy

A report from the Strategic Advisory Group of Experts (SAGE) on immunization of the WHO defines vaccine hesitancy as “delay in acceptance or refusal of vaccination despite availability of vaccinations services. Vaccine hesitancy is complex and context specific, varying across time, place, and vaccines. It is influenced by factors such as complacency, convenience, and confidence (26).” Vaccine hesitancy is a continuum between complete acceptance and full rejection. This study divided vaccine hesitancy into seven categories: fully accepted, accepted but unsure, partially accepted, delayed vaccination, partially rejected, rejected but unsure, and completely rejected. Individuals who chose the two categories, fully accepted and accepted but unsure, were considered to have no vaccine hesitancy. Those who selected the other five were classified as having vaccine hesitancy. This classification provides a framework for understanding the various degrees of vaccine hesitancy that individuals may exhibit.

Vaccine hesitancy rate

The vaccine hesitancy rate refers to the proportion of the total number of people who participated in the survey and were classified as having vaccine hesitancy toward COVID-19 vaccine during the survey period.

Vaccination rate

The survey on vaccination rate examines both the vaccination status and the actual vaccination status of the survey participants. The vaccination rates are calculated based on the number of individuals who have received at least one dose of vaccine and those who have completed the full three-dose course (considered as fully vaccinated). The vaccination rate is the percentage of people who have been vaccinated out of the total number of people surveyed each month (the samples drawn each month are different). The actual cumulative vaccination rate of the population refers to the proportion of the cumulative number of vaccinated people during a particular month, relative to the permanent resident population of Guangzhou.

Determinants of vaccine hesitancy

There are many potential reasons as to why the residents are hesitant to receive vaccination against COVID-19, including underlying diseases (chronic diseases such as hypertension and diabetes, and COPD), self-assessed health scores, unpleasant experiences with vaccination, consultation with professionals about COVID-19 vaccination, and less knowledge of vaccines (including whether the mutant strain affects the effectiveness of the vaccine in preventing disease and whether vaccination reduces the likelihood of developing severe stage in the future).

Knowledge and beliefs about vaccination

To investigate residents' knowledge and beliefs about vaccination and the influencing factors of vaccine hesitancy, we designed 18 questions from three aspects, namely, self-confidence, complacency, and convenience. Self-confidence includes trust in the efficacy and safety of vaccines, reliability and competence of health services and health professionals, and motivations of vaccine decision-makers. Complacency is mainly due to the lack of sufficient understanding of the necessity and importance of vaccines. Convenience mainly includes vaccine availability, geographic location, and willingness to pay. Each item is scored on a scale of “1 = strongly disagree” to “5 = strongly agree” with reference to the Likert scale. The higher the score, the greater the impact this factor has on residents' hesitation in receiving the COVID-19 vaccine. We tested the reliability and validity of the scale, respectively, and found that the Cronbach's α values in April 2021, May 2021, and June 2021 were 0.667, 0.736, and 0.876. Overall results with values >0.6 were accepted.

Statistical analysis

Descriptive analyses of demographic characteristics were performed. Categorical variables are expressed as the number of cases and percentages (%) in a different group. The survey participants were divided into four categories according to their age, namely, <25, 25–34, 35–44, and >45 years. The subjects were divided into the vaccine-hesitant group and the vaccine non-hesitant group according to the above definition of vaccine hesitancy. The vaccine hesitancy rate for each month was calculated individually. Univariate analysis was performed on the data from April to June 2021 using the chi-squared test. Internal consistency of the scale scores was evaluated with Cronbach's α. The vaccine-hesitant individuals were regarded as the case group and the vaccine non-hesitant individuals as the control group, and the odds ratio (OR) and 95% confidence interval (CI) were calculated by the binary logistic regression model. A two-sided test was used for statistical analysis in this study, and the test level was α = 0.05. Taking “1 = strongly disagree” as a reference, for each increment of 1, the risk of vaccine hesitancy due to this factor increases OR times.

Results

A total of 15,000 questionnaires were distributed during the course of this survey, of which 13,521 of them were completed. We performed quality control on the questionnaires, excluding questionnaires with missing information and questionnaire completion time of <60 s. Finally, 12,977 valid questionnaires were collected, and the response rate was 95.9%.

Demographic characteristics

Table 1 describes in detail the date of the survey, the number of people in each round of surveys, and the distribution of respondents' age, gender, region, education level, monthly income, and occupation. The population participating in this survey predominantly consists of individuals aged between 20 and 50 years. There are more urban residents than rural residents. In addition, most participants have attained an undergraduate degree or higher.

TABLE 1

Table 1. Demographic characteristics.

Vaccine hesitancy rate

As shown in Table 2, between April 2021 and June 2021, the vaccine hesitancy rate showed a significant downward trend, i.e., from 30% to the lowest value of 9.1%. However, in the November 2021 survey, we found a small increase in the vaccine hesitancy rate (13.7%). The survey showed that there has been a continuous upward trend in the vaccine hesitancy rates (five consecutive surveys from April 2022 to December 2022), which peaked to 30.4% in December 2022.

TABLE 2

Table 2. Vaccine hesitancy rate.

Factors influencing vaccine hesitancy

COVID-19 vaccination rate

Since March 2021, when the Guangzhou Center for Disease Control and Prevention first announced its COVID-19 vaccination program, the vaccination rate in Guangzhou has shown an obvious upward trend. The survey shows a sharp increase in the number of people vaccinated against COVID-19 from April to June 2021, according to Guangzhou residents' vaccination data provided by the Guangzhou Municipal Health Commission. The data, including the vaccination rate for at least one dose and the full course of vaccination, reveal that the vaccination rate has reached more than 90% in November 2021 (Table 3). It is consistent with the results of our investigation, indicating that this study is representative. As shown in Figure 1, the chi-squared test revealed that the P-values of the survey results at each stage were all <0.001, indicating that there was a statistically significant difference between the two groups in vaccination.

TABLE 3

Table 3. COVID-19 vaccination rate in Guangzhou.

FIGURE 1

Figure 1. COVID-19 vaccination rate and hesitation rate among Guangzhou residents at different stages. The vaccination policy issued by Guangzhou city and the fluctuation of the epidemic situation.

Epidemic wave of COVID-19 in Guangzhou

Since the emergence of COVID-19, Guangzhou has experienced a total of five local waves of the epidemic. The first wave occurred from January to April 2020, followed by the second wave from May to June 2021, during when the Delta virus variant was prevalent. The third wave was from March to May 2022, when the virus variant was Omicron. The fifth wave emerged from October to December 2022 (Figure 2). The correlation coefficients between the vaccine hesitancy rate and the number of cases during the second, third, and fourth waves were r₁ = -0.783, r₂ = 0.996, and r₃ = 1.000.

FIGURE 2

Figure 2. The new local COVID-19 cases report in Guangzhou form January 2021 to June 2022 (as the national health ccommission announced that data will not be released from December 14, 2022, the data was collected until December 21, 2022).

Sociodemographic factors

In the April 2021 survey, individuals living in urban areas (31.6%) were more likely to develop vaccine hesitancy than those living in rural areas (23.6%). However, in the subsequent May and June surveys, residence could no longer be considered a statistically significant factor associated with vaccine hesitancy. In both the April and June 2021 surveys, people with primary school education and below showed higher rates of hesitation. and the results were statistically significant. In the May 2021 survey, healthcare workers exhibited no vaccine hesitancy, while workers and farmers had a higher likelihood of vaccine hesitancy than individuals did in other occupations. The results of the May–June 2021 surveys revealed that there are more negative attitudes toward vaccination among those with underlying medical conditions. After examining the self-assessed health scores, three different outcomes emerged. The results of the April 2021 survey showed that residents with moderate self-assessed health were more likely to be hesitant toward vaccinations. As time went on, however, hesitation increased significantly among residents who self-assessed their health as poor in June, following adverse side effects and various negative reports concerning the vaccine. The June study also found that people with unpleasant past vaccination experiences were more likely to be hesitant to get vaccinated again, and, as evidenced by the April study, those who did not believe that the COVID-19 vaccine would reduce COVID-19 symptoms were more likely to be hesitant to vaccinate (Table 4).

TABLE 4

Table 4. Determinants of vaccine hesitancy.

Influencing factors of vaccine hesitancy on knowledge and belief

Figure 3 shows the adjusted odds ratios and 95% confidence intervals for factors that contributed to vaccine hesitancy between April 2021 and June 2021. Among these 18 influencing factors, Q1–Q5 belong to complacency, Q6–Q11 and Q15–Q18 belong to confidence, and Q12–Q14 belong to convenience. The main influencing factors of vaccine hesitancy among individuals in April, May, and June 2021 are given as follows: a lower perceived risk of contracting COVID-19 in China (April: OR = 2.191, 95% CI = 1.809–2.653; May: OR = 1.703, 95% CI = 1.347–2.153; and June: OR = 2.441, 95% CI = 1.972–3.020), sufficiency with existing treatments (April: OR = 1.978, 95% CI = 1.583–2.470; May: OR = 1.722, 95% CI = 1.344–2.206; and June: OR = 2.280, 95% CI = 1.830–2.842), and disallowing themselves of the need for vaccination due to their current health status (April: OR = 1.925, 95% CI = 1.573–2.356; May: OR = 1.506, 95% CI = 1.152–1.969; and June: OR = 2.545, 95% CI = 2.029–3.192). However, all three items belong to the complacency factor. In addition to these three influencing factors, the second related factor leading to vaccine hesitancy is the fear of vaccine side effects (April: OR = 1.631, 95% CI = 1.417–1.876; May: OR = 1.560, 95% CI = 1.310–1.857; and June: OR = 2.327, 95% CI = 1.876–2.886). There was no statistically significant difference between April and June 2021 in terms of vaccine availability. The following factors all led to vaccine risk hesitation among individuals during the 3 months of the survey: vaccines cannot effectively prevent new coronavirus infection (April: OR = 1.608, 95% CI = 1.349–1.917; May: OR = 1.416, 95% CI = 1.131–1.772; and June: OR = 1.983, 95% CI = 1.664–2.363), b vaccines are risky (April: OR = 1.314, 95% CI = 1.129–1.529; May: OR = 1.250, 95% CI = 1.039–1.505; and June: OR = 1.718, 95% CI = 1.399–2.109), worries that vaccination personnel are not standardized (April: OR = 1.309, 95% CI = 1.128–1.519; May: OR = 1.265, 95% CI = 1.038–1.541; June: OR = 1.520, 95% CI = 1.255–1.841), the vaccine effectiveness is not high (April: OR = 1.425, 95% CI = 1.221–1.663; May: OR = 1.443, 95% CI = 1.180–1.764; and June: OR = 2.403, 95% CI = 1.952–2.958), lacked the time to vaccinate (April: OR = 1.181, 95% CI = 1.035–1.348; May: OR = 1.267, 95% CI = 1.059–1.516; and June: OR = 1.706, 95% CI = 1.431–2.034), and an hesitant attitude toward vaccine if colleagues or classmates were hesitant (April: OR = 1.230, 95% CI = 1.077–1.405; May: OR = 1.222, 95% CI = 1.021–1.464; and June: OR = 1.498, 95% CI = 1.257–1.785). Two factors were newly added in June 2021 as contributing factors to vaccine hesitancy, namely the belief that personal protection protects against COVID-19 (OR = 1.271, 95% CI = 1.091–1.481) and vaccine hesitancy among healthcare workers (OR = 1.418, 95% CI = 1.197–1.680).

FIGURE 3

Figure 3. Multivariate logistic regression analysis of factors associated with COVID-19 vaccine hesitancy. The OR value and 95% CI were obtained after adjusting for gender and age. Graphs in black with “*” inidcate that the item is satistically significant.

Discussion

This study highlights the importance of understanding the dynamic changes in residents' hesitancy since the COVID-19 vaccination program began in Guangzhou at the end of March 2021 and explored the main factors leading to vaccine hesitancy to propose targeted measures to increase residents' confidence and willingness to vaccinate and increase vaccination rates, particularly during times of increased risk or changes in vaccination policies. The attitudes and factors influencing vaccination hesitancy are not constant and may change over time and according to circumstances (27). For example, research in the United States showed that the hesitancy rates for a COVID-19 vaccine fell from 46.0% at the start of the survey to 35.2% 3 months after the survey (28). Conversely, in Hong Kong, the vaccination hesitancy rate during the two waves of the epidemic in February and August 2020 increased from 55.8 to 65.2% (29). Another long-term cross-sectional survey in Hong Kong reported that, along with the fluctuation of the coronavirus epidemic, people's hesitation concerning vaccination also showed fluctuation at different stages (30). Similar to the fluctuation in the previous study, the vaccine hesitancy rate among Guangzhou residents dropped from 30% to 9.1% between April and June 2021. However, in the November 2021 survey, we found a slight increase in vaccine hesitancy (13.7%) rather than a sustained decline. The Guangzhou Center for Disease Control and Prevention issued notices on vaccination for residents aged 12–17 years and over 60 years old from June to November 2021. Considering that children and the older adult are more likely to have side effects (such as dizziness, fatigue, etc.) owing to their poor physical fitness (31), this may have contributed to the increase in vaccine hesitancy in November. Since 8 April 2022, Guangzhou has ushered in a new wave of SARS-CoV-2 infections, which primarily infected vaccinated individuals. This may have caused vaccine hesitancy to increase from 13.4% to 23.0% between April and June 2022. Later, the fourth wave of the epidemic in Guangzhou in October 2022, and the alterations in various epidemic prevention and control policies may have caused residents to be lax in adhering to epidemic prevention and control policies and may have further affected vaccine hesitation. According to the survey, the vaccine hesitancy rate of Guangzhou residents reached 30.4% in December 2022, which was higher than the rate recorded at the beginning of the survey.

The residents' knowledge of vaccines was insufficient, such as failing to acknowledge that the COVID-19 vaccine can effectively mitigate the severity and mortality of COVID-19 (32) or having misconceptions such as believing that vaccination can completely prevent the infection of the disease. In light of this, we should actively educate the public, popularize vaccine-related knowledge, and disseminate correct vaccination information. Our research also found that unpleasant vaccination experiences and distrust of healthcare personnel (who administers vaccines) were also the key factors leading to hesitancy. Therefore, during the vaccination process, we should conduct uniform and standardized training for staff to alleviate residents' doubts concerning the professionalism of vaccination of those administering vaccines to reduce the public's resistance (5).

Since the organization and implementation of the booster vaccination program in Guangzhou in October 2022, more than 80% of the residents completed the full course of vaccination in November of the same year, but the situation of vaccine hesitancy continues to persist. The extent of vaccination hesitation cannot be fully reflected by the vaccination rate (33). Moreover, we found that a small percentage of residents who had received the first dose of vaccination had not completed the full course of vaccination. Consequently, addressing and overcoming residents' hesitancy remains an inevitable and ongoing challenge.

Our survey coincides with the timing of the second, third, and fourth waves of the COVID-19 epidemic in Guangzhou. The obtained correlation coefficient obtained from our analysis of the survey from April to June 2021 showed that the vaccine hesitancy rate decreases with the emergence of the epidemic (r <0). However, the surveys taken in April–June 2022 and November–December 2022 showed that the correlation coefficient was positive (r>0), indicating that the epidemic has led to an increase in the rate of vaccine hesitancy. In May 2021, Guangzhou City launched its vaccination campaign for the first time, and with the assistance and extensive publicity effort from the state and government, residents began to vaccinate against COVID-19. The emergence of new COVID-19 cases that month may have sparked panic about the disease and increased the demand for vaccinations among residents, thereby reducing the vaccine hesitancy rates. In the third and fourth waves of the epidemic, even though most people have been fully vaccinated, some of them were still infected. This led to questions among some people who do not understand the characteristics of vaccine-preventable diseases to question the effectiveness of the vaccine, thereby increasing the vaccination hesitancy rate (11).

Residence and education are important factors affecting vaccination. Consistent with the findings of Israel and the UK (3, 34), people living in urban areas tend to have a higher hesitation rate, which may be related to factors such as population density, income, etc. Compared to rural areas, urban areas have better medical resources and protective measures in place. Studies have shown that people in urban areas are more susceptible to negative information about vaccination and thus show distrust of WHO, while people in rural areas tend to be more receptive to government's calls to action and arrangements (35). Attitudes toward vaccine hesitancy vary across occupations, which may be due to differing levels of medical knowledge and their varying needs for preventive measures (36). Medical workers, for instance, will have a more accurate understanding of vaccine knowledge, and the thus demand for vaccines will be higher (36). In contrast, farmers or workers may be more likely to refuse vaccination due to lower health literacy or perceived low risk of contracting a disease (37). Furthermore, chronic diseases were also associated with vaccination hesitancy, as evidenced by a cross-sectional survey conducted in Hong Kong (29). A survey of COVID-19 vaccine hesitancy among chronically ill children aged 5–11 years in Italy showed that 26.3% of chronically ill patients were highly hesitant. The low perceived risk of children being infected with SARS-CoV-2 coupled with the fact that the children had not yet been vaccinated further exacerbated the degree of vaccine hesitancy (38). However, one particular finding of our study was that people with moderate self-rated health scores exhibited higher rates of vaccination hesitancy. This suggests that refusing vaccination because of physical health is also a major factor leading to hesitation.

Since the outbreak of the COVID-19 epidemic in late 2020, China has implemented a series of stringent measures and public health interventions to control the spread of COVID-19 (39, 40), which has also caused some members of the public to underestimate the risk of the disease (5). Our study found that the main reasons for refusing vaccination include: confidence in one's own robust immune system, confidence in the treatment of the disease, and the belief that existing preventive measures are sufficient to prevent COVID-19 without vaccination. Numerous studies have demonstrated that the perceived risk of contracting the disease also affects the public's vaccination intentions (41–44). We must increase public awareness of the new coronavirus and underscore the importance and necessity of vaccination in order to reduce vaccine hesitation rates and maximize the number of people vaccinated.

Consistent with findings from other countries (29, 45, 46), concerns about the safety and side effects of vaccination are also one of the main reasons for vaccine hesitancy. COVID-19 is an emerging infectious disease, and uncertainty about a new vaccine may further heighten public concerns about vaccination (47). A cross-sectional study of people in Italy who had completed initial vaccination showed that those who reported reluctance and uncertainty about getting the booster shot were mainly concerned about vaccine safety (48). To enhance the public's confidence in vaccination, national or government authorities should regularly monitor and disclose scientific information concerning the safety of vaccines and conduct timely health education and communication to alleviate the public's concerns about vaccine safety and side effects (33). Social media dissemination of information about vaccine safety is as important as that of vaccine efficacy (49). Research by Betsch et al. (50) showed that disproportionally widespread negative coverage of vaccines on the Internet may increase people's distrust of vaccines. Therefore, healthcare authorities should strengthen the control over the dissemination of vaccine-related information on social media platforms and prevent the dissemination of information that denigrates or exaggerates the safety and effectiveness of vaccines (51). Social media can also play a positive role as timely dissemination of accurate information about the COVID-19 vaccines and effective control of fake news related to COVID-19 vaccines can eliminate public hesitation about vaccination and help increase confidence in vaccination (52, 53). Vaccines are more likely to be accepted if the information about vaccines is endorsed by medical professionals, as well as their family, friends, and colleagues (54). Other studies have also demonstrated that medical practitioners' advice on vaccination is more likely to be adopted by the public (55). Therefore, medical workers should establish a good relationship of trust with the population, thereby alleviating people's concerns and addressing vaccine hesitancy (56). Previous studies have shown that organizing health education among professionals on immunization strategies can have a positive effect on disease prevention and is essential for good adherence to vaccination (57). In response to challenges in vaccine access and distribution, such as not having time to get vaccinated and not knowing where to receive vaccination, China has taken various corresponding measures, such as mobile vaccination vehicles and setting up mobile vaccination locations in various communities. These measures have brought great convenience to residents and promoted the vaccination campaign.

The limitation of this study is that it is a cross-sectional study, which makes it difficult to infer the influence of an individual or a set of factors on vaccine-hesitant behavior. However, this study adopted a random sampling method for different populations to guarantee the representativeness of the sample. Although the study only conducted three surveys on knowledge and belief, the results were largely consistent. Self-report questionnaires are susceptive to subjective interpretation and bias, which may affect the reliability and validity of the survey results. Therefore, we still need more research studies to confirm our findings. The study cannot provide conclusive evidence of a causal relationship between changes in background factors (including changes in news coverage of COVID-19 vaccines and policies) and changes in public risk perception-related vaccine hesitancy. Therefore, the inferences drawn from this study must be considered tentative.

Conclusion

In this study, we found that vaccine hesitancy did not exhibit a steady decline over time but it rather fluctuated. Risk factors for vaccine hesitancy included higher education, urban residency, a lower perceived risk of contracting the disease, and concerns about the vaccine's safety and side effects. Appropriate interventions and education initiatives would be effective and are necessary to improve public confidence in vaccination.

Data availability statement

The original contributions presented in the study are included in the article/Supplementary material, further inquiries can be directed to the corresponding authors.

Ethics statement

The studies involving human participants were reviewed and approved by the Institutional Review Board of Guangzhou Medical University - approval: GZMU2020-07-0676. Written informed consent to participate in this study was provided by the participants' legal guardian/next of kin.

Author contributions

LC analyzed and interpreted the data and completed the writing. ZL, XL, ZW, and LC took part in the investigation. JL, YD, and ZW made contributions to conceptualization, participant recruitment, and project management. JL, XL, TL, and LL guided the methodology and revised the manuscript. All authors read and approved the final manuscript.

Funding

This study was supported by the following grants: the National Natural Science Foundation of China (grant numbers 81872694 and 82173609) (JL) and the Local Innovative and Research Teams Project of Guangdong Pearl River Talents Program (grant number 2017BT01S155) (JL).

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpubh.2023.1164475/full#supplementary-material

References

1. Yamey G, Schäferhoff M, Hatchett R, Pate M, Zhao F, McDade KK. Ensuring global access to COVID-19 vaccines. Lancet. (2020) 395:1405–6. doi: 10.1016/S0140-6736(20)30763-7

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Chou WS, Budenz A. Considering emotion in COVID-19 vaccine communication: addressing vaccine hesitancy and fostering vaccine confidence. Health Commun. (2020) 35:1718–22. doi: 10.1080/10410236.2020.1838096

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Fisher KA, Bloomstone SJ, Walder J, Crawford S, Fouayzi H. Mazor KM. Attitudes toward a potential SARS-CoV-2 vaccine : a survey of US adults. Ann Intern Med. (2020) 173:964–73. doi: 10.7326/M20-3569

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Wang C, Han B, Zhao T, Liu H, Liu B, Chen L, et al. Vaccination willingness, vaccine hesitancy, and estimated coverage at the first round of COVID-19 vaccination in China: a national cross-sectional study. Vaccine. (2021) 39:2833–42. doi: 10.1016/j.vaccine.2021.04.020

PubMed Abstract | CrossRef Full Text | Google Scholar

5. The The first new coronavirus vaccine is listed on conditional conditions (State Council Joint Prevention and Control Mechanism Press Conference) in People's Daily (2021). Available online at: https://news.china.com/social/1007/20210101/39137891.html

Google Scholar

6. Harrison EA, Wu JW. Vaccine confidence in the time of COVID-19. Eur J Epidemiol. (2020) 35:325–30. doi: 10.1007/s10654-020-00634-3

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Shetty P. Experts concerned about vaccination backlash. Lancet. (2010) 375:970–1. doi: 10.1016/S0140-6736(10)60421-7

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Larson HJ, de Figueiredo A, Xiahong Z, Schulz WS, Verger P, Johnston IG, et al. The state of vaccine confidence 2016: global insights through a 67-country survey. EBioMedicine. (2016) 12:295–301. doi: 10.1016/j.ebiom.2016.08.042

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Geoghegan S, O'Callaghan KP, Offit PA. Vaccine safety: myths and misinformation. Front Microbiol. (2020) 11:372. doi: 10.3389/fmicb.2020.00372

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Callaghan T, Moghtaderi A, Lueck JA, Hotez P, Strych U, Dor A, et al. Correlates and disparities of intention to vaccinate against COVID-19. Soc Sci Med. (2021) 272:113638. doi: 10.1016/j.socscimed.2020.113638

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Karafillakis E, Larson HJ. ADVANCE consortium. The benefit of the doubt or doubts over benefits? A systematic literature review of perceived risks of vaccines in European populations. Vaccine. (2017) 35:4840–50. doi: 10.1016/j.vaccine.2017.07.061

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Olive JK, Hotez PJ, Damania A, Nolan MS. The state of the antivaccine movement in the United States: a focused examination of nonmedical exemptions in states and counties. PLoS Med. (2018) 15:e1002578. doi: 10.1371/journal.pmed.1002578

PubMed Abstract | CrossRef Full Text | Google Scholar

13. COCONEL Group. A future vaccination campaign against COVID-19 at risk of vaccine hesitancy and politicisation. Lancet Infect Dis. (2020) 20: 769–0. doi: 10.1016/S1473-3099(20)30426-6

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Sherman SM, Smith LE, Sim J, Amlôt R, Cutts M, Dasch H, et al. COVID-19 vaccination intention in the UK: results from the COVID-19 vaccination acceptability study (CoVAccS), a nationally representative cross-sectional survey. Hum Vaccin Immunother. (2021) 17:1612–21. doi: 10.1080/21645515.2020.1846397

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Frank KAR. Canadians' willingness to get a COVID-19 vaccine: Group differences and reasons for vaccine hesitancy. Statistics Canada Catalogue 45280001: No. 00043.

Google Scholar

16. Omer SB, Salmon DA, Orenstein WA, deHart MP, Halsey N. Vaccine refusal, mandatory immunization, and the risks of vaccine-preventable diseases. N Engl J Med. (2009) 360:1981–8. doi: 10.1056/NEJMsa0806477

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Tomietto M, Simonetti V, Comparcini D, Stefanizzi P, Cicolini G. A large cross-sectional survey of COVID-19 vaccination willingness amongst healthcare students and professionals: reveals generational patterns. J Adv Nurs. (2022) 78:2894–903. doi: 10.1111/jan.15222

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Stefanizzi P, Martinelli A, Bianchi FP, Migliore G. Tafuri S.Acceptability of the third dose of anti-SARS-CoV-2 vaccine co-administered with influenza vaccine: preliminary data in a sample of Italian HCWs. Hum Vaccin Immunother. (2022) 18:1–2. doi: 10.1080/21645515.2021.2011652

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Porru S, Monaco MGL, Spiteri G, Carta A, Pezzani MD, Lippi G et al. SARS-CoV-2 breakthrough infections: incidence and risk factors in a large European multicentric cohort of health workers. Vaccines. (2022) 10:1193. doi: 10.3390/vaccines10020272

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Tang L, Zhang Y, Wang F, Wu D, Qian ZH. Relative vaccine effectiveness against Delta and Omicron COVID-19 after homologous inactivated vaccine boosting: a retrospective cohort study. BMJ Open. (2022) 12:e063919. doi: 10.1136/bmjopen-2022-063919

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Al Kaabi N, Zhang Y, Xia S. Effect of 2 inactivated SARS-CoV-2 vaccines on symptomatic COVID-19 infection in adults: a randomized clinical trial. JAMA. 2021;326(1):35–45. 10.1001/jama.2021.8565 doi: 10.1001/jama.2021.8565

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Tartof SY, Slezak JM, Fischer H. Effectiveness of mRNA BNT162b2 COVID-19 vaccine up to 6 months in a large integrated health system in the USA: a retrospective cohort study. Lancet. (2021) 398:1407–16. doi: 10.1016/S0140-6736(21)02183-8

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Singanayagam A, Hakki S, Dunning J. Community transmission and viral load kinetics of the SARS-CoV-2 delta (B. 1617 2) variant in vaccinated and unvaccinated individuals in the UK: a prospective, longitudinal, cohort study. Lancet Infect Dis. (2022) 22:183–95. doi: 10.1016/S1473-3099(21)00648-4

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Anderson RM, May RM. Infectious Diseases of Humans: Dynamics and Control. Oxford, New York: Oxford University Press (1991).

Google Scholar

25. Anderson RM, Vegvari C, Truscott J, Collyer BS. Challenges in creating herd immunity to SARS-CoV-2 infection by mass vaccination. Lancet. (2020) 396:1614–6. doi: 10.1016/S0140-6736(20)32318-7

PubMed Abstract | CrossRef Full Text | Google Scholar

26. MacDonald NE. Vaccine hesitancy: definition, scope and determinants. Vaccine. (2015) 33:4161–4. doi: 10.1016/j.vaccine.2015.04.036

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Xiao X, Wong RM. Wong, Vaccine hesitancy and perceived behavioral control: a meta-analysis. Vaccine. (2020) 38:5131–8. doi: 10.1016/j.vaccine.2020.04.076

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Daly M, Jones A, Robinson E. Public trust and willingness to vaccinate against COVID-19 in the US From October 14, 2020, to March 29, 2021. JAMA. (2021) 325:2397–9. doi: 10.1001/jama.2021.8246

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Wang K, Wong EL, Ho KF, Cheung AW, Yau PS, Dong D, et al. Change of willingness to accept COVID-19 vaccine and reasons of vaccine hesitancy of working people at different waves of local epidemic in Hong Kong, China: repeated cross-sectional surveys. Vaccines. (2021) 9:62. doi: 10.3390/vaccines9010062

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Xiao J, Cheung JK, Wu P, Ni MY, Cowling BJ, Liao Q. Temporal changes in factors associated with COVID-19 vaccine hesitancy and uptake among adults in Hong Kong: Serial cross-sectional surveys. Lancet Reg Health West Pac. (2022) 23:100441. doi: 10.1016/j.lanwpc.2022.100441

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Menni C, Klaser K, May A, Polidori L, Capdevila J, Louca P. et al.Vaccine side-effects and SARS-CoV-2 infection after vaccination in users of the COVID Symptom Study app in the UK: a prospective observational study. Lancet Infect Dis. (2021) 21:939–49. doi: 10.1016/S1473-3099(21)00224-3

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Baden LR, El Sahly HM, Essink B, Kotloff K, Frey S, Novak R, et al. Efficacy and Safety of the mRNA-1273 SARS-CoV-2 Vaccine. N Engl J Med. (2021) 384:403–16. doi: 10.1056/NEJMoa2035389

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Larson HJ, Jarrett C, Eckersberger E, Smith DM, Paterson P. Understanding vaccine hesitancy around vaccines and vaccination from a global perspective: a systematic review of published literature, 2007-2012. Vaccine. (2014) 32:2150–9. doi: 10.1016/j.vaccine.2014.01.081

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Green MS, Abdullah R, Vered S, Nitzan D, A. study of ethnic, gender and educational differences in attitudes toward COVID-19 vaccines in Israel - implications for vaccination implementation policies. Isr J Health Policy Res. (2021) 10:26. doi: 10.1186/s13584-021-00458-w

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Hak E, Schönbeck Y, De Melker H, Van Essen GA, Sanders EA. Negative attitude of highly educated parents and health care workers towards future vaccinations in the Dutch childhood vaccination program. Vaccine. (2005) 23:3103–7. doi: 10.1016/j.vaccine.2005.01.074

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Gudi SK, Chhabra M, Undela K, Venkataraman R, Mateti UV, Tiwari KK, et al. Knowledge and beliefs towards universal safety precautions during the coronavirus disease (COVID-19) pandemic among the Indian public: a web-based cross-sectional survey. Drugs Ther Perspect. (2020) 36:413–20. doi: 10.1007/s40267-020-00752-8

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Dodd RH, Cvejic E, Bonner C, Pickles K, McCaffery KJ. Sydney Health Literacy Lab COVID-19 group. Willingness to vaccinate against COVID-19 in Australia Lancet Infect Dis. (2021) 21:318–9. doi: 10.1016/S1473-3099(20)30559-4

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Miraglia Del Giudice G, Napoli A, Corea F, Folcarelli L, Angelillo IF. Evaluating COVID-19 Vaccine willingness and hesitancy among parents of children aged 5-11 years with chronic conditions in Italy. Vaccines. (2022) 10:396. doi: 10.3390/vaccines10030396

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Leung K, Wu JT, Liu D, Leung GM. First-wave COVID-19 transmissibility and severity in China outside Hubei after control measures, and second-wave scenario planning: a modelling impact assessment. Lancet. (2020) 395:1382–93. doi: 10.1016/S0140-6736(20)30746-7

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Kraemer MUG, Yang CH, Gutierrez B, Wu CH, Klein B, Pigott DM, et al. The effect of human mobility and control measures on the COVID-19 epidemic in China. Science. (2020) 368:493–7. doi: 10.1126/science.abb4218

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Alsuhaibani M. Barriers and beliefs among health-care workers regarding seasonal influenza vaccine in Al-Qassim region, Saudi Arabia. Hum Vaccin Immunother. (2020) 16:313–20. doi: 10.1080/21645515.2019.1656020

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Abdelhafiz AS, Mohammed Z, Ibrahim ME, Ziady HH, Alorabi M, Ayyad M, et al. Knowledge, perceptions, and attitude of Egyptians towards the novel coronavirus disease (COVID-19). J Community Health. (2020) 45:881–90. doi: 10.1007/s10900-020-00827-7

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Almotairy AM, Sheikh WA, Joraid AAA, Bajwi AA, Alharbi MSF, Al-Dubai SAR. Association between knowledge of influenza vaccine and vaccination status among general population attending primary health care centers in Al-Madinah, Saudi Arabia. J Family Med Prim Care. (2019) 8:2971–4. doi: 10.4103/jfmpc.jfmpc_547_19

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Bish A, Yardley L, Nicoll A, Michie S. Factors associated with uptake of vaccination against pandemic influenza: a systematic review. Vaccine. (2011) 29:6472–84. doi: 10.1016/j.vaccine.2011.06.107

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Solís Arce JS, Warren SS, Meriggi NF, Scacco A, McMurry N, Voors M, et al. COVID-19 vaccine acceptance and hesitancy in low- and middle-income countries. Nat Med. (2021) 27: 1385–94. doi: 10.1101/2021.03.11.21253419

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Razai, Chaudhry UAR, Doerholt K, Bauld L, Majeed A. Covid-19 vaccination hesitancy. BMJ. (2021) 373: n1138. doi: 10.1136/bmj.n1138

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Kreps S, Prasad S, Brownstein JS, Hswen Y, Garibaldi BT, Zhang B, et al. Factors associated with US Adults' Likelihood of accepting COVID-19 vaccination. JAMA Netw Open. (2020) 3: e2025594. doi: 10.1001/jamanetworkopen.2020.25594

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Folcarelli, L. Miraglia Del Giudice G, Corea F, Angelillo IF. Intention to Receive the COVID-19 Vaccine Booster Dose in a University Community in Italy. Vaccines. (2022) 10:146. doi: 10.3390/vaccines10020146

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Johnson NF, Velásquez N, Restrepo NJ, Leahy R, Gabriel N, El Oud S, et al. The online competition between pro- and anti-vaccination views. Nature. (2020) 582:230–3. doi: 10.1038/s41586-020-2281-1

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Betsch C, Sachse K. Debunking vaccination myths: strong risk negations can increase perceived vaccination risks. Health Psychol. (2013) 32:146–55. doi: 10.1037/a0027387

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Bendau A, Petzold MB, Pyrkosch L, Mascarell Maricic L, Betzler F, Rogoll J. et al.Associations between COVID-19 related media consumption and symptoms of anxiety, depression and COVID-19 related fear in the general population in Germany. Eur Arch Psychiatry Clin Neurosci. (2021) 271:283–91. doi: 10.1007/s00406-020-01171-6

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Van Nguyen D, Nguyen PH. Social media and COVID-19 vaccination hesitancy: mediating role of the COVID-19 vaccine perception. Heliyon. (2022) 8:e10575. doi: 10.1016/j.heliyon.2022.e10575

PubMed Abstract | CrossRef Full Text | Google Scholar

53. Di Mauro A, Di Mauro F, De Nitto S, Rizzo L, Greco C, et al. Social Media Interventions Strengthened COVID-19 Immunization Campaign. Front Pediatr. (2022) 10:869893. doi: 10.3389/fped.2022.869893

PubMed Abstract | CrossRef Full Text | Google Scholar

54. McMurray R, Cheater FM, Weighall A, Nelson C, Schweiger M, Mukherjee S. Managing controversy through consultation: a qualitative study of communication and trust around MMR vaccination decisions. Br J Gen Pract. (2004) 54:520–5.

PubMed Abstract | Google Scholar

55. Malik AA, McFadden SM, Elharake J, Omer SB. Determinants of COVID-19 vaccine acceptance in the US. EClinicalMedicine. (2020) 26:100495. doi: 10.1016/j.eclinm.2020.100495

PubMed Abstract | CrossRef Full Text | Google Scholar

56. Yaqub O, Castle-Clarke S, Sevdalis N, Chataway J. Attitudes to vaccination: a critical review. Soc Sci Med. (2014) 112:1–11. doi: 10.1016/j.socscimed.2014.04.018

PubMed Abstract | CrossRef Full Text | Google Scholar

57. Trucchi C, Costantino C, Restivo V, Bertoncello C, Fortunato F, et al. Immunization Campaigns and Strategies against Human Papillomavirus in Italy: the Results of a Survey to Regional and Local Health Units Representatives. Biomed Res Int. (2019) 2019:6764154. doi: 10.1155/2019/6764154

PubMed Abstract | CrossRef Full Text | Google Scholar