Assessing COVID-19 vaccination strategies in varied demographics using an individual-based model

Background: New variants of SARS-CoV-2 are constantly discovered. Administration of COVID-19 vaccines and booster doses, combined with the application of non-pharmaceutical interventions (NPIs), is often used to prevent outbreaks of emerging variants. Such outbreak dynamics are further complicated by the population's behavior and demographic composition. Hence, realistic simulations are needed to estimate the efficiency of proposed vaccination strategies in conjunction with NPIs.

Methods: We developed an individual-based model of COVID-19 dynamics that considers age-dependent parameters such as contact matrices, probabilities of symptomatic and severe disease, and households' age distribution. As a case study, we simulate outbreak dynamics under the demographic compositions of two Israeli cities with different household sizes and age distributions. We compare two vaccination strategies: vaccinate individuals in a currently prioritized age group, or dynamically prioritize neighborhoods with a high estimated reproductive number. Total infections and hospitalizations are used to compare the efficiency of the vaccination strategies under the two demographic structures, in conjunction with different NPIs.

Results: We demonstrate the effectiveness of vaccination strategies targeting highly infected localities and of NPIs actively detecting asymptomatic infections. We further show that different optimal vaccination strategies exist for each sub-population's demographic composition and that their application is superior to a uniformly applied strategy.

Conclusion: Our study emphasizes the importance of tailoring vaccination strategies to subpopulations' infection rates and to the unique characteristics of their demographics (e.g., household size and age distributions). The presented simulation framework and findings can help better design future responses against the following emerging variants.

Introduction

In December 2019, a new virus, SARS-CoV-2, emerged in Wuhan, China. In March 2020, the World Health Organization (WHO) announced coronavirus disease 19 (COVID-19), a pandemic (1). Countries closed their borders and implemented harsh travel restrictions to slow the virus's spread (2). Nevertheless, the virus led to massive mortality, severe hospital loads, and a worldwide impact on the economy (1). At the beginning of the pandemic, a vaccine was not available. Hence, to relieve the burden on hospitals and limit the number of casualties, governments worldwide applied non-pharmaceutical interventions (NPIs). These NPIs included home isolation of infected or suspected individuals, closure of workplaces, schools, kindergartens, and more (3).

The Food and Drug Administration (FDA) approved the first SARS-CoV-2 vaccine at the end of 2020, and a worldwide vaccination operation started. Since then, 4.6 billion people have been fully vaccinated (received two doses), and 11.8 billion doses have been administered worldwide (4). Countries chose different vaccination strategies, including rollout rates and prioritization of sub-groups (age, comorbidities, or essential workers). For example, Israel was an especially early adopter of the vaccine and prioritized older ages and individuals with health risks (5, 6). Two months after the initiation of the first vaccination campaign, almost 70% of the individuals eligible for vaccination (age > 18 years), and 80% of individuals older than 60 years, were fully vaccinated in Israel (6).

New variants of the SARS-CoV-2 are constantly evolving worldwide (7, 8); some have become the locally or globally predominant variants. For example, in April 2021, the Delta variant emerged from India and quickly became the predominant variant in many parts of the world (9). In Israel, which was among the first countries to reach high vaccination coverage by the end of March 2021, vaccine breakthroughs by the Delta variant were already common during July 2021. Analyses of the breakthrough data pointed to the importance of waning vaccine immunity (10). To mitigate the Delta variant outbreak while minimizing lockdowns, Israel administered a third dose of the COVID-19 vaccine. Statistical and mathematical models have demonstrated that the booster campaign was instrumental in controlling the Delta resurgence (11–13). However, to estimate the efficiency of other, untested vaccination strategies in conjunction with NPIs, realistic simulation tools are needed.

Model-based analyses have been used to help assess the effectiveness of different NPIs and vaccination strategies (14–17), as controlled experiments are impossible and retrospective data analysis is often lacking. Most modeling studies of SARS-CoV-2 employ compartmental models, such as the differential equation-based SEIR (Susceptible - Exposed - Infected - Recovered/ Removed) model. These models categorize all individuals into a few compartments, not considering the variance in their roles in the outbreak.

An alternative complementary modeling strategy is individual-based models (IBMs; also called agent-based models, ABMs) with stochastic, interacting autonomous agents representing individuals in the population. IBMs can simulate complex dynamics, the system's temporal evolution, and generate intricate patterns of behaviors produced by individuals' interactions. Incorporating fundamental population-specific features can provide insight into systems with small susceptible populations, which are highly affected by stochasticity (18). However, incorporating such features comes at the cost of intense computational demands and decreased interpretability of some results, due to the models' complexity. Nonetheless, IBMs have been previously used to simulate the spread of infectious diseases (19–22) and specifically to study the effects of NPIs and vaccination strategies against COVID-19 (23–30). For instance, previous studies have shown that prioritizing vaccination of the elderly could limit future deaths or quality-adjusted life year losses (31). In contrast, other IBM models have suggested that prioritizing vaccination of individuals with a large number of contacts can be useful to mitigate outbreaks (32).

Here, we developed an IBM populated with COVID-19 parameters. The model simulates outbreaks while considering age-dependent parameters, such as contact matrices, symptomatic and severe disease probabilities, and households' age distribution. Demographic characteristics of a population are especially influential in COVID-19 outbreaks (33, 34). Hence, our simulation utilizes the demographic compositions of two Israeli cities with similar population sizes but different household sizes and age distributions as case studies. First, the city of Holon, in which about a quarter of the population is <18 years, and the mean household size is approximately 2.8 individuals. In contrast, the city of Bnei Brak has a much younger and denser population, with approximately half of its population <18 years and an average household size of 4.5 individuals. Using these two disparate demographics, we investigate the effects of different vaccination strategies, in combinations with different NPIs, on infections and hospitalizations.

Methods

IBM architecture

The IBM is based on three components (the explicit simulation framework is found in Supplementary material, sections “SEIR model, parameters, and their distribution”, and “Creating a population”):

1 Demographics: Each individual in the simulated synthetic population is assigned a certain age, neighborhood, and household. Each individual also has weekly routines (a workplace, school, and time spent in their neighborhood), determining contacts with other individuals with overlapping routines. Here, we use the demographics of two cities similar in population size (about 194,000 in Bnei Brak and 193,000 in Holon, as of 2017) but different in their household sizes and age distributions, as case studies. Based on the demographic data of either of the two cities in Israel, a synthetic population comprising household members' joint distribution, ages, employment, and education is generated.

2 Infection dynamics: Daily contacts between individuals in the described social setting are simulated (35). Age-heterogeneous contact matrices were used as the risk of contacting an infected person varies between age groups and locations, representing the probability of contacting persons from another age group. This simulation uses the POLYMOD contact matrix (35). POLYMOD was generated in 2017 and includes contact matrices based on data from 152 countries, including Israel.

3 Disease states (Figure 1): At any given time point, each individual is classified into one of the disease states: susceptible, infected at a latent state, subclinical (also termed asymptomatic), presymptomatic (incubation), clinical (symptomatic), immune, critical, and deceased. Each susceptible individual may change to a latently infected state when contacting an infectious individual. The contact probability, relative infectiousness, and duration distribution in each state are based on previous studies (26). Other parameters, such as susceptibility to infection on contact, relative infectiousness of subclinical cases, and the delay from disease onset to hospitalization, are based on empirical data and plausible ranges from relevant literature (see Supplementary material).

FIGURE 1

Figure 1. Illustration of the IBM framework. (A) A synthetic population was generated based on Israeli demographics. The population is divided into a hierarchy of social circles, including households and neighborhoods. Contacts between different age groups and settings, such as their household, school, or workplace, are drawn from data-informed contact matrices (35). (B) Disease progression and immunity states are imposed on each individual in the population. Orange rectangles represent exposed and infectious states, and purple rectangles represent removed states. Transition rates between the states are denoted with their parameters, which are given in Table 1.

The simulations and their analyses were conducted in Python 3.7, using the following packages: Numpy 1.22.3, Pandas 1.4.2, Matplotlib 3.5.1, Scipy 1.8.0 and Pyfunctional 0.7.0.

Simulation parameters

The main simulation parameters are shown in Table 1. Additional model parameters and specifications are given in Supplementary Table 1.

TABLE 1

Table 1. Relevant parameters and estimated distribution.

Vaccination strategies

Israel's country-wide daily vaccination rate, from the initial vaccination campaign, was used to define the daily vaccination rollout for the two evaluated cities, scaled to the size of the cities modeled (4). This scaling led to a daily rollout of 700 vaccines in each city. We deliberately did not use the actual rate of vaccination of each city during the epidemic, so we can directly estimate the effects of demographics, unconfounded by other factors. Individuals who were susceptible, asymptomatic, latent, or incubating post-latent at the time of vaccination were eligible for vaccination. However, individuals that recovered from an asymptomatic infection were not vaccinated. Although this might not reflect reality under policies not testing for antibodies before vaccination, the effect should not be substantial: asymptomatic infections are more common in individuals below the age of 18 [(47); see Supplementary Table 1 for details], who were not in the age group eligible for vaccination in this simulation. Furthermore, in this simulation framework, individuals were immediately immunized after receiving the vaccine, not reflecting the actual time until immunity, approximately 1–2 weeks (48, 49). Thus, a correction for the delayed immunity was implemented by shifting the vaccination timeline by a week (i.e., individuals scheduled to be vaccinated at time t were actually vaccinated at time t + 7).

Four vaccination strategies were evaluated, keeping the same number of vaccinated individuals per day. Vaccine efficacy (VE) of 90% in preventing infections and hospitalizations was used, as was suggested during the first vaccination campaign (42–44). VE was modeled by limiting the vaccinated proportion by the value of VE. For example, if VE was set to 90%, then 90% of the individuals originally chosen for vaccination were instead vaccinated daily. The following vaccination strategies were the focus of this study:

1 General strategy—vaccinate individuals older than the minimum vaccination age (set at 18) and within the currently vaccinated age group (with random vaccine allocation within the age groups). Meaning that each day, N randomly chosen individuals who meet the criteria for vaccination are being vaccinated.

2 Neighborhood strategy—each day, the neighborhood (defined as housing approximately 3,000 individuals) in which the estimated instantaneous reproductive number R_t (which estimates the average number of secondary cases caused by an infected individual at time t; see Equation 1 in Supplementary material) is the highest is vaccinated first. Since R_t can only be calculated after the first infections are recovered and secondary infections are present, during the first vaccination period, the Neighborhood strategy vaccinates from the oldest age group and descends until enough data to calculate the R_t is available.

Each strategy was tested with three types of age prioritization: a descending order, where the oldest age group is vaccinated first; an ascending order, where the youngest age group is vaccinated first; and no age prioritization (i.e., vaccination was done randomly in proportion to the age distribution). Results for the latter prioritization strategy were left for the Supplementary Figure 4. Other, less successful strategies were also implemented and examined, and their details and results are presented in Supplementary material under “vaccination strategies”. These vaccination strategies included: a Household strategy, wherein individuals in a prioritized age group are vaccinated by clusters of households; an All At Once strategy, in which an entire household is vaccinated if it contains at least one individual in the currently prioritized age group.

NPI modeling

The NPIs in this simulation were modeled by lowering the contact rates between the relevant groups. For example, school closure was modeled by limiting contacts between school-aged individuals, and house quarantine was modeled by reducing contacts with the general public while increasing contacts within the household. Compliance with the NPIs was modeled by introducing a parameter governing the proportion of individuals complying with each NPI.

Several NPIs have been investigated. The two NPIs yielding reduction in infections and hospitalizations without forcing a complete lockdown were chosen to be investigated in combination with the vaccination strategies. In both NPIs, social distancing was implemented in the population. Other interventions were implemented only for specific individuals:

1 Household isolation includes the isolation of symptomatic cases and their household members.

2 Asymptomatic Detection includes isolation of symptomatic cases and their household members and test and isolation of asymptomatic cases below the age of 18. To consider compliance and the tests' imperfect sensitivity, only 70% of asymptomatic cases in individuals <18 were detected and isolated [in accordance with estimates of PCR tests' sensitivity in detecting infections (45)].

Simulation configurations

The main simulation configurations examined and presented in the main text were:

- City demographics (Holon, Bnei Brak)

- Vaccination Strategies

- Strategy (General, Neighborhood)

- Age prioritization (Ascending, Descending, No Age priritization)

- NPI (Household Isolation, Asymptomatic Detection)

Evaluation metrics

Two metrics were used for each scenario to compare the performance of the different vaccination strategies: the total number of infections and the total number of hospitalized cases per 100 k individuals, averaged over 500 simulations. These metrics were chosen to reflect disease spread and morbidity. All simulations were run for 150 days, as this period sufficed for the outbreaks to fade and to emulate simplified SARS-CoV-2 variant outbreak dynamics without waning immunity (10). Additionally, this period corresponds to the eligibility criterion of the minimal time elapsed from vaccination to a booster shot, as was set in Israel (10).

The mean differences in the total infections and hospitalizations across simulation repetitions were compared for pairs of vaccination strategies. The central limit theorem was used to derive 95% confidence intervals (CI) for these comparisons, assuming unequal variances.

Results

First, typical outbreak dynamics under the Asymptomatic Detection intervention are shown in Figure 2. The results are presented for both demographics, and include the number of infected (Figures 2A,B) and hospitalized (Figures 2C,D) individuals, and the instantaneous reproductive number R_t (Figures 2E,F), over time. An analogous figure for the Household Isolation intervention can be found in Supplementary Figure 8.

FIGURE 2

Figure 2. Typical outbreak dynamics under different vaccination strategies and demographic structures. The cumulative number of new infections per 100 k (A,B), hospitalizations per 100k (C,D), and R_t (E,F), are shown. The left and right-hand columns present the results under the demography of Bnei Brak and Holon, respectively. Each panel presents the daily mean of 500 simulations, and the shaded regions around the curves represent the standard error of the mean.

The two Neighborhood vaccination strategies, prioritizing neighborhoods with high transmission rates, yielded low infection rates throughout the simulations. Under the demographics of Bnei Brak, the Neighborhood Descending strategy allowed for low hospitalization rates, while keeping the total infections lower than the General strategies and similar to the Neighborhood Ascending strategy. Furthermore, while the General Ascending strategy resulted in only a few more infections than both Neighborhood strategies, it resulted in almost twice as many hospitalizations.

Simulating Holon's demographics, the Neighborhood Descending strategy resulted in the best trade-off between both criteria (infections and hospitalizations) compared to the other vaccination strategies (Figures 2B,D). While this strategy did not result in the lowest number of infections, it yielded relatively low numbers in both criteria. On the other hand, the General Descending strategy resulted in almost twice as many infections as the two ascending strategies. Moreover, this strategy resulted in about 1.6-fold infections as its analogous Neighborhood strategy. When simulating the city demographics of Bnei Brak, the Neighborhood Descending strategy again resulted in an advantage over its analogous General strategy, but with a less substantial difference than in the demographics Holon. Thus, the Neighborhood Descending strategy yielded a reasonable trade-off between the two metrics (total infections and total hospitalizations), for both cities.

To complete the picture, we also examined the daily estimated instantaneous reproductive number (compare Figures 2E,F). Implementing Asymptomatic Detection with any vaccination strategy resulted in a decline in the reproductive number in both cities, with a faster decline in Holon, eventually dropping below 1 (i.e., the number of infections is reducing). In contrast, the R_t values under the Household Isolation intervention did not drop below 1 until the completion of the simulation (Supplementary Figure 8). Lastly, while comparing the overall performance of the different strategies, it appears that there is larger variability between the strategies in the total infections and the R_t-values in Holon (Figures 2A,B,E,F).

In Figure 3, we present a comparison of the distributions of the total infections and hospitalizations when applying the Asymptomatic Detection and the Household Isolation NPIs, and for different vaccination strategies. All comparisons were performed using the number of cases per 100,000 individuals. Compared to Household Isolation, Asymptomatic Detection limited both epidemiological metrics assessed: it resulted in approximately 1/8 of the total infections in Bnei Brak's demographics, less than half the infections in Holon's demographics, and <1/2 of the hospitalized cases in both demographics.

FIGURE 3

Figure 3. Examining the efficiency of the two main NPIs. Each panel shows violin plots of the number of infected (A,B) or hospitalized (C,D) per 100 k individuals at the end of 500 simulations. The plots are further stratified by the application of the Asymptomatic Detection (A,C) or Household Isolation (B,D) interventions.

Simulating Bnei Brak's demographics, under the Asymptomatic Detection intervention, both Neighborhood strategies led to fewer infections per 100,000 individuals compared to the General Ascending strategy [- 48.1, 95% CI (- 74.3, - 21.9) for Neighborhood Ascending, and - 37.6, 95% CI (- 64.5, - 10.7) for Neighborhood Descending]. Furthermore, the Neighborhood Descending strategy also resulted in fewer infections compared to the General Descending strategy [- 191.8, 95% CI (- 221.3, - 162.3)]. The number of infections under Neighborhood Ascending and Neighborhood Descending were very similar and not statistically different [- 10.5, 95% CI (- 35.6, 14.6)]. Comparing the total hospitalizations, rates were higher for the Neighborhood Descending strategy than for the General Descending strategy [9.8, 95% CI (5.3, 14.3)]. Still, the Neighborhood Descending strategy performed better than the Neighborhood Ascending [- 8.8, 95% CI (- 13.6, - 4.0)] and the General Ascending strategies [- 72.6, 95% CI (- 78.8, - 66.5)]. Moreover, the Neighborhood Ascending strategy led to fewer hospitalizations than the General Ascending strategy [- 63.9, 95% CI (- 70.5, - 57.5)].

We observed a similar trend when applying the Household Isolation intervention: Neighborhood Ascending reduced infections better than the General Ascending [- 334.0, 95% CI (- 508.8, - 167.1)] and General Descending [- 800.5, 95% CI (- 975.3, - 625.6)] strategies. Similar to the results of applying Asymptomatic Detection, the Neighborhood Descending strategy reduced more infections than both the General Ascending [- 328.6, 95% CI (- 498.4, - 158.8)] and General Descending [- 791.1, 95% CI (- 964.9, - 617.4)] strategies. The difference between the Neighborhood Ascending and the Neighborhood Descending strategies was again not significant [- 9.3, 95% CI (- 185.6, 167.0)]. When examining the ability to limit total hospitalizations, both Descending strategies were better than their parallel Ascending strategies [- 93.4, 95% CI (- 109.5, - 77.3) for the difference between the Neighborhood Descending and Neighborhood Ascending strategies and - 619.3, 95% CI (- 641.1, - 597.4) for the difference between the General strategies]. While comparing both Descending strategies, the Neighborhood Descending strategy resulted in more hospitalization than the General Descending strategy [73.6, 95% CI (64.5, 82.8)].

When simulating the city demographics of Holon, there were no significant differences in infection rates between the Neighborhood Ascending and the General Ascending strategies [- 13.3, 95% CI (- 31.3, 4.6)]. Still, both Neighborhood strategies performed better than the commonly used General Descending strategy [- 333.9, 95% CI (- 363.2, - 304.5) for Neighborhood Ascending vs. General Descending; and - 238.9, 95% CI (- 270.6, - 207.2) for Neighborhood Descending vs. General Descending]. Neighborhood Descending performed better than all other strategies in reducing hospitalizations [- 56.0, 95% CI (- 66.2, - 45.8) for Neighborhood Descending vs. General Ascending;- 29.2, 95% CI (- 38.6, - 19.7) for Neighborhood Descending vs. General Descending; and - 14.7, 95% CI (- 24.4, - 5.0) for Neighborhood Descending vs. Neighborhood Ascending].

While applying Household Isolation to the city demographics of Holon, the Neighborhood Ascending strategy resulted in fewer infections than its analogous General Ascending strategy [- 154.7, 95% CI (- 207.2, - 102.3)] and an even more substantial reduction compared to the General Descending strategy [- 1308.5, 95% CI (- 1388.0, - 1229.1)]. Comparing the total hospitalizations, the Neighborhood Descending strategy had an advantage compared to all other strategies [- 234.8, 95% CI (- 256.3, - 213.2) for Neighborhood Descending vs. General Ascending; - 41.1, 95% CI [- 56.5, - 25.7] for Neighborhood Descending vs. General Descending; and - 52.8, 95% CI (- 71.9, - 33.7) for Neighborhood Descending vs. Neighborhood Ascending].

We further sought to examine whether vaccination strategies should be uniformly or differentially applied to locations with differing demographics. To this end, we independently simulated outbreaks under both demographic compositions (i.e., no spillover between the two populations) and both NPIs and summed the infections and hospitalizations per 100,000 people. These results are presented in Figure 4, under either uniform vaccination strategies, where the same strategy is applied in both cities, or under a different strategy for each demographic composition. When applying Household Isolation, the optimal strategy in terms of the combined number of hospitalizations was General Descending under the Bnei Brak demography and Neighborhood Descending under the Holon demography. Similarly, when applying the Asymptomatic Detection intervention, using the same combination of strategies resulted in the lowest combined number of hospitalizations.

FIGURE 4

Figure 4. Applying vaccination strategies uniformly and differentially between demographics. Colors represent the mean hospitalizations (top) and infections (bottom) per 100 k individuals, combined for both demographic compositions. Results are shown using either the same vaccination strategy in both demographic compositions or the best strategy for each demographic composition, under Household Isolation (left) or Asymptomatic Detection (right). The numbers represent the mean of 500 simulations per scenario. The vaccination strategies are sorted according to the total hospitalizations. The General Descending strategy is highlighted as it is the most commonly used strategy in most countries. ND, Neighborhood Descending; NA, Neighborhood Ascending, and GD, General Descending.

Discussion

This study employs a realistic individual-based model of COVID-19 dynamics to investigate epidemic outcomes under different vaccination strategies. The unique structure of our IBM enabled us to compare complex vaccination strategies that target subpopulations and their interplay with applications of different NPIs. These investigations would have been extremely difficult to implement using classic SEIR-based compartmental models. Moreover, we explicitly considered differing demographic structures, incorporating population structures from two demographically distinct Israeli cities.

Our results are consistent with the widely accepted notion that prioritizing the elderly for vaccination reduces hospitalizations under a broad range of conditions. Indeed, this has been demonstrated in recent theoretical studies (15, 31) and was the strategy of choice in many countries (50), following the finding that older individuals have higher chances of developing severe COVID-19 given infection. On the other hand, we showed that prioritizing vaccination for younger individuals reduced infections. This is due to the high rate of social interactions of younger individuals, coupled with their higher rates of asymptomatic infections (47, 51), leading to lower probabilities of case isolations. Similar results have been demonstrated in recent theoretical studies that evaluated vaccination strategies focusing on individuals with a large number of contacts (32).

We further examined the effects of NPIs and vaccination strategies on the two distinct demographic compositions. We found that the same interventions and vaccination strategies produced different results in the selected population structures. Under the ‘older’ demography of Holon, the different vaccination strategies mainly affected the number of infections and, correspondingly, the R_t values. With respect to NPIs, actively detecting asymptomatic infections (Asymptomatic Detection) led to a significantly lower number of infections and hospitalizations than isolating household members of symptomatic cases (Household Isolation).

Several reasons could explain the success of the Asymptomatic Detection intervention. First, younger individuals tend to be more mobile and have more contacts per day (52), leading to higher chances of secondary infections for each asymptomatic case. Second, since younger individuals are less prone to present symptoms (33, 53), their abundance may reduce the effectiveness of NPIs that mainly target symptomatic cases (54). Altogether, actively testing to detect asymptomatic infections can be valuable in demographic settings with a younger population.

Nevertheless, applying the Asymptomatic Detection intervention is not trivial. First, it requires compliance from parents to test their children frequently. Second, it is expensive to implement due to the high use of testing kits. On the other hand, the Asymptomatic Detection intervention successfully reduced both the number of infections and hospitalizations compared to Household Isolation. For example, under the demographics of Bnei Brak, the Household Isolation intervention resulted in an eight-fold increase in infections and almost a two-fold increase in hospitalizations relative to applying Asymptomatic Detection. Accordingly, when aiming to limit the number of hospitalizations and reduce the burden on intensive care units, Asymptomatic Detection should be investigated as a useful NPI, especially in settings with a young population structure.

Under both demographic settings, the Neighborhood strategies were less affected by the order of the vaccination than the General strategies. A possible explanation is that the Neighborhood strategies focused all of the vaccines on a single neighborhood at a time. Thus, it allowed for a more rapid shift from the prioritized age group to other age groups within the neighborhood. Moreover, for the demographics of Holon, the Neighborhood Descending strategy performed better than any other strategy, under both NPIs, to reduce hospitalizations and better than the General Descending strategy in reducing infections. Under the demographics of Bnei Brak, the Neighborhood Descending strategy performed best in reducing infections and resulted in the second-lowest total hospitalizations. Notably, the General Descending strategy is the most used vaccination strategy worldwide (50). However, while reducing hospitalizations, the General Descending strategy performed the worst at reducing infections compared to the other vaccination strategies simulated. We hence propose that using the Neighborhood Descending strategy as a potential alternative can provide a good trade-off between total hospitalizations and infections.

The proposed Neighborhood strategies can be considered as previously studied “spatial vaccination” strategies, which prioritize individuals for vaccination based on their geographic location. For example, a recent study investigated the use of a spatial vaccination strategy in which regions at higher risk of importing the pathogen are prioritized, compared to a vaccination strategy in which highly connected regions are prioritized for vaccination (55). The Neighborhood strategies studied here are also analogous to “targeted geographic vaccination”. This previously used vaccination strategy focuses on areas, neighborhoods, or villages with higher infection rates (56) and was successfully used in the Ebola outbreak in Chow in South Kivu (57). Moreover, it was shown that the targeted geographic vaccination was more effective than both mass and ring vaccination (58). Another similar vaccination strategy, used to some extent in the UK against the B.1.617.2 SARS-CoV-2 variant, is “surge vaccination” (59). Under this vaccination strategy, younger age groups and previously unvaccinated individuals in areas with high prevalence or rapidly growing outbreaks are prioritized for vaccination. Furthermore, the UK government has suggested using the surge vaccination strategy in tandem with NPIs, to mitigate outbreaks while vaccine induced immunity develops (60). Hence, focusing on highly infected population units, such as a neighborhood, has empirical support for potential efficiency against both COVID-19 and other infectious diseases- aligned with our results.

The disadvantages of the Neighborhood vaccination strategies are their logistical demands and public perception. The selected neighborhood and prioritization within the neighborhood can change rapidly, requiring efficient information flow and high compliance from the neighborhoods' residents. To overcome the logistical difficulties, small neighborhoods can be combined into larger clusters based on their R_t. Furthermore, the selection of the next cluster to vaccinate can be made less frequently (e.g., every week). Future studies could examine how the frequency of prioritizing the vaccination locations affects optimal strategies, using explicit simulation frameworks, such as ours. Finally, targeting specific subpopulations before others can be perceived negatively by the public. Subpopulations offered the vaccine early on might hesitate to comply for fear of side effects (61); alternatively, sup-populations vaccinated later on may feel discriminated against (62).

We also sought to understand whether applying mixed strategies to different populations could be beneficial. While a uniform vaccination strategy is usually chosen across entire countries, we showed a potential advantage in combining different strategies for different sub-populations. For example, while applying the Household Isolation intervention, using a combined strategy resulted in 7% fewer hospitalizations compared to the best uniform strategy, which in this case was the commonly used General Descending strategy. This result demonstrates the potential benefit of selecting the optimal strategy per city, county, or other lower-level localities, as countries are not uniform in their demographics. However, similar difficulties to those elaborated above regarding the Neighborhood strategies apply when implementing combinations of vaccination strategies in different localities.

We also note that our assumed form of VE imposes potential limitations on the generalizability of our study. Since we model vaccination by moving individuals directly to the recovered compartment, our model refers to vaccination as equally efficient against infections and developing severe disease given infection. Consequently, our study is mainly relevant for scenarios where VE is similar with respect to both of these outcomes. Indeed, this assumption held in the initial vaccination campaigns against the first SARS-CoV-2 variants (42–44). Future work should focus on expanding this model and decomposing VE so it can assume different values against infection, transmission, and severe disease.

To conclude, we have shown that subpopulations with different demographic compositions may require different vaccination strategies. Our study emphasizes the importance of tailoring a strategy to the unique characteristics of a subpopulation's demographics (e.g., household size distribution and age distribution) instead of following a uniformly applied strategy. We also demonstrated the effectiveness of vaccination strategies targeting highly infected localities and of NPIs that actively seek asymptomatic infections. The presented simulation framework and our findings can help better design future responses against emerging SARS-CoV-2 variants and other pathogens.

Data availability statement

The datasets presented in this study can be found in online repositories. The names of the repository/repositories and accession number(s) can be found in the article/Supplementary material.

Author contributions

NB-Z, DN, AH, and UO conceived the model and the study. NB-Z, YD, DB-A, and AS wrote the code and tested the model. NB-Z performed the simulations. NB-Z and UO interpreted the results and wrote the first draft of the paper. All authors contributed to the article and approved the submitted version.

Funding

This work was supported by a grant from Tel Aviv University Center for AI and Data Science (TAD) in collaboration with Google, as part of the initiative of AI and DS for social good.

Acknowledgments

We thank Ronen Olinky, Tom Kalavari, and the rest of the scenarios team recruited for the early COVID-19 response by the Gertner Institute for Epidemiology and Health Policy Research, for helping with the design and coding of an early version of the model.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The reviewer RT declared a past co-authorship with the author UO to the handling editor.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpubh.2022.966756/full#supplementary-material

References

1. World Health Organization. Origin of SARS-CoV-2, 26 march 2020 (No. WHO/2019-nCoV/FAQ/Virus_origin/2020.1). World Health Organization (2020).

Google Scholar

2. Daon Y, Thompson RN, Obolski U. Estimating COVID-19 outbreak risk through air travel. J Travel Med. (2020) 27:taaa093. doi: 10.1093/jtm/taaa093

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Perra N. Non-pharmaceutical interventions during the COVID-19 pandemic: a review. Phys Rep. (2021) 913:1–52. doi: 10.1016/j.physrep.2021.02.001

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Ritchie H, Mathieu E, Rodés-Guirao L, Appel C, Giattino C, Ortiz-Ospina E, et al. Coronavirus pandemic (COVID-19). Our World Data. (2020). Available online at: https://ourworldindata.org/coronavirus (accessed June 11, 2022).

Google Scholar

5. Rosen B, Waitzberg R, Israeli A. Israel's rapid rollout of vaccinations for COVID-19. Isr J Health Policy Res. (2021) 10:6. doi: 10.1186/s13584-021-00440-6

PubMed Abstract | CrossRef Full Text

6. Rossman H, Shilo S, Meir T, Gorfine M, Shalit U, Segal E. COVID-19 dynamics after a national immunization program in Israel. Nat Med. (2021) 27:1055–61. doi: 10.1038/s41591-021-01337-2

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Tao K, Tzou PL, Nouhin J, Gupta RK, de Oliveira T, Kosakovsky Pond SL, et al. The biological and clinical significance of emerging SARS-CoV-2 variants. Nat Rev Genet. (2021) 22:757–73. doi: 10.1038/s41576-021-00408-x

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Harvey WT, Carabelli AM, Jackson B, Gupta RK, Thomson EC, Harrison EM, et al. SARS-CoV-2 variants, spike mutations and immune escape. Nat Rev Microbiol. (2021) 19:409–24. doi: 10.1038/s41579-021-00573-0

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Dhar MS, Marwal R, Vs R, Ponnusamy K, Jolly B, Bhoyar RC, et al. Genomic characterization and epidemiology of an emerging SARS-CoV-2 variant in Delhi, India. Science. (2021) 374:995–9. doi: 10.1101/2021.06.02.21258076

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Goldberg Y, Mandel M, Bar-On Bar-On YM, Bodenheimer O, Freedman L, Haas EJ, et al. Waning Immunity after the BNT162b2 Vaccine in Israel. N Engl J Med. (2021) 385:e85. doi: 10.1056/NEJMoa2114228

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Gavish N, Yaari R, Huppert A, Katriel G. Population-level implications of the Israeli booster campaign to curtail COVID-19 resurgence. Sci Transl Med. (2022) 14:eabn9836. doi: 10.1126/scitranslmed.abn9836

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Barda N, Dagan N, Cohen C, Hernán MA, Lipsitch M, Kohane IS, et al. Effectiveness of a third dose of the BNT162b2 mRNA COVID-19 vaccine for preventing severe outcomes in Israel: an observational study. Lancet. (2021) 398:2093–100. doi: 10.1016/S0140-6736(21)02249-2

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Feng A, Obolski U, Stone L, He D. Modelling COVID-19 vaccine breakthrough infections in highly vaccinated Israel—the effects of waning immunity and third vaccination dose. medRxiv (2022). doi: 10.1101/2022.01.08.22268950

CrossRef Full Text | Google Scholar

14. Moore S, Hill EM, Tildesley MJ, Dyson L, Keeling MJ. Vaccination and non-pharmaceutical interventions for COVID-19: a mathematical modelling study. Lancet Infect Dis. (2021) 21:793–802. doi: 10.1016/S1473-3099(21)00143-2

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Bubar KM, Reinholt K, Kissler SM, Lipsitch M, Cobey S, Grad YH, et al. Model-informed COVID-19 vaccine prioritization strategies by age and serostatus. Science. (2021) 371:916–21. doi: 10.1126/science.abe6959

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Thompson RN. Epidemiological models are important tools for guiding COVID-19 interventions. BMC Med. (2020) 18:152. doi: 10.1186/s12916-020-01628-4

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Zhu Z, Chen B, Chen H, Qiu S, Fan C, Zhao Y, et al. Strategy evaluation and optimization with an artificial society toward a Pareto optimum. The Innovation. (2022) 3:100274. doi: 10.1016/j.xinn.2022.100274

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Hu B, Guo H, Zhou P, Shi ZL. Characteristics of SARS-CoV-2 and COVID-19. Nat Rev Microbiol. (2021) 19:141–54. doi: 10.1038/s41579-020-00459-7

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Cliff OM, Harding N, Piraveenan M, Erten EY, Gambhir M, Prokopenko M. Investigating spatiotemporal dynamics and synchrony of influenza epidemics in Australia: an agent-based modelling approach. Simul Model Pract Theory. (2018) 87:412–31. doi: 10.1016/j.simpat.2018.07.005

CrossRef Full Text | Google Scholar

20. Germann TC, Kadau K, Longini IM, Macken CA. Mitigation strategies for pandemic influenza in the United States. Proc Natl Acad Sci. (2006) 103:5935–40. doi: 10.1073/pnas.0601266103

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Marini M, Brunner C, Chokani N, Abhari RS. Enhancing response preparedness to influenza epidemics: Agent-based study of 2050 influenza season in Switzerland. Simul Model Pract Theory. (2020) 103:102091. doi: 10.1016/j.simpat.2020.102091

CrossRef Full Text | Google Scholar

22. Lourenço J, Daon Y, Gori A, Obolski U. Pneumococcal competition modulates antibiotic resistance in the pre-vaccination era: a modelling study. Vaccines. (2021) 9:265. doi: 10.3390/vaccines9030265

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Cuevas E. An agent-based model to evaluate the COVID-19 transmission risks in facilities. Comput Biol Med. (2020) 121:103827. doi: 10.1016/j.compbiomed.2020.103827

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Silva PCL, Batista PVC, Lima HS, Alves MA, Guimarães FG, Silva RCP, et al. An agent-based model of COVID-19 epidemic to simulate health and economic effects of social distancing interventions. Chaos Solitons Fractals. (2020) 139:110088. doi: 10.1016/j.chaos.2020.110088

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Panovska-Griffiths J, Kerr CC, Stuart RM, Mistry D, Klein DJ, Viner RM, et al. Determining the optimal strategy for reopening schools, the impact of test and trace interventions, and the risk of occurrence of a second COVID-19 epidemic wave in the UK: a modelling study. Lancet Child Adolesc Health. (2020) 4:817–27. doi: 10.1016/S2352-4642(20)30250-9

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Davies NG, Kucharski AJ, Eggo RM, Gimma A, Edmunds WJ, Jombart T, et al. Effects of non-pharmaceutical interventions on COVID-19 cases, deaths, and demand for hospital services in the UK: a modelling study. Lancet Public Health. (2020) 5:e375–85. doi: 10.1101/2020.04.01.20049908

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Aleta A, Martín-Corral D. Pastore y Piontti A, Ajelli M, Litvinova M, Chinazzi M, et al. Modelling the impact of testing, contact tracing and household quarantine on second waves of COVID-19. Nat Hum Behav. (2020) 4:964–71. doi: 10.1038/s41562-020-0931-9

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Català M, Li X, Prats C, Prieto-Alhambra D. The impact of prioritisation and dosing intervals on the effects of COVID-19 vaccination in Europe: an agent-based cohort model. Sci Rep. (2021) 11:18812. doi: 10.1038/s41598-021-98216-0

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Chang SL, Harding N, Zachreson C, Cliff OM, Prokopenko M. Modelling transmission and control of the COVID-19 pandemic in Australia. Nat Commun. (2020) 11:5710. doi: 10.1038/s41467-020-19393-6

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Dunuwila PM, Rajapakse RACP. Evaluating Optimal Lockdown and Testing Strategies for COVID-19 using Multi-Agent Social Simulation. In: 2020 2nd International Conference on Advancements in Computing (ICAC). (2020). p. 240–5. doi: 10.1109/ICAC51239.2020.9357132

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Moore S, Hill EM, Dyson L, Tildesley MJ, Keeling MJ. Modelling optimal vaccination strategy for SARS-CoV-2 in the UK. PLOS Comput Biol. (2021) 17:e1008849. doi: 10.1371/journal.pcbi.1008849

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Gog JR, Hill EM, Danon L, Thompson RN. Vaccine escape in a heterogeneous population: insights for SARS-CoV-2 from a simple model. R Soc Open Sci. 8:210530. doi: 10.1098/rsos.210530

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Davies NG, Klepac P, Liu Y, Prem K, Jit M. CMMID COVID-19 working group, et al. Age-dependent effects in the transmission and control of COVID-19 epidemics. Nat Med. (2020) 26:1205–11. doi: 10.1038/s41591-020-0962-9

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Dowd JB, Andriano L, Brazel DM, Rotondi V, Block P, Ding X, et al. Demographic science aids in understanding the spread and fatality rates of COVID-19. Proc Natl Acad Sci. (2020) 117:9696–8. doi: 10.1073/pnas.2004911117

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Prem K, Cook AR, Jit M. Projecting social contact matrices in 152 countries using contact surveys and demographic data. PLOS Comput Biol. (2017) 13:e1005697. doi: 10.1371/journal.pcbi.1005697

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Nishiura H, Linton NM, Akhmetzhanov AR. Serial interval of novel coronavirus (COVID-19) infections. Int J Infect Dis. (2020) 93:284–6. doi: 10.1016/j.ijid.2020.02.060

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Li Q, Guan X, Wu P, Wang X, Zhou L, Tong Y, et al. Early transmission dynamics in Wuhan, China, of novel coronavirus–infected pneumonia. N Engl J Med. (2020) 382:1199–207. doi: 10.1056/NEJMoa2001316

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Bi Q, Wu Y, Mei S, Ye C, Zou X, Zhang Z, et al. Epidemiology and transmission of COVID-19 in 391 cases and 1,286 of their close contacts in Shenzhen, China: a retrospective cohort study. Lancet Infect Dis. (2020) 20:911–9. doi: 10.1101/2020.03.03.20028423

CrossRef Full Text | Google Scholar

39. Russell TW, Hellewell J, Jarvis CI, van Zandvoort K, Abbott S, Ratnayake R, et al. Estimating the infection and case fatality ratio for coronavirus disease (COVID-19) using age-adjusted data from the outbreak on the Diamond Princess cruise ship, February 2020. Eurosurveillance. (2020) 25:2000256. doi: 10.2807/1560-7917.ES.2020.25.12.2000256

PubMed Abstract | CrossRef Full Text | Google Scholar

40. D'Arienzo M, Coniglio A. Assessment of the SARS-CoV-2 basic reproduction number, R0, based on the early phase of COVID-19 outbreak in Italy. Biosaf Health. (2020) 2:57–9. doi: 10.1016/j.bsheal.2020.03.004

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Du Z, Liu C, Wang C, Xu L, Xu M, Wang L, et al. Reproduction numbers of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) variants: a systematic review and meta-analysis. Clin Infect Dis. (2022) 75:e293–5. doi: 10.1093/cid/ciac137

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Nasreen S, Chung H, He S, Brown KA, Gubbay JB, Buchan SA, et al. Effectiveness of COVID-19 vaccines against symptomatic SARS-CoV-2 infection and severe outcomes with variants of concern in Ontario. Nat Microbiol. (2022) 7:379–85. doi: 10.1038/s41564-021-01053-0

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Katz MA, Harlev EB, Chazan B, Chowers M, Greenberg D, Peretz A, et al. Early effectiveness of BNT162b2 Covid-19 vaccine in preventing SARS-CoV-2 infection in healthcare personnel in six Israeli hospitals (CoVEHPI). Vaccine. (2022) 40:512–20. doi: 10.1016/j.vaccine.2021.11.092

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Dagan N, Barda N, Kepten E, Miron O, Perchik S, Katz MA, et al. BNT162b2 mRNA covid-19 vaccine in a nationwide mass vaccination setting. N Engl J Med. (2021) 384:1412–23. doi: 10.1056/NEJMoa2101765

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Kucirka LM, Lauer SA, Laeyendecker O, Boon D, Lessler J. Variation in false-negative rate of reverse transcriptase polymerase chain reaction–based SARS-CoV-2 tests by time since exposure. Ann Int Med. (2020) 173:262–7. doi: 10.7326/M20-1495

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Ferguson N, Laydon D, Nedjati Gilani G, Imai N, Ainslie K, Baguelin M, et al. Report 9: Impact of Non-Pharmaceutical Interventions (Npis) to Reduce Covid19 Mortality and Healthcare Demand. Imperial College London (2020).

Google Scholar

47. Ma Q, Liu J, Liu Q, Kang L, Liu R, Jing W, et al. Global percentage of asymptomatic SARS-CoV-2 infections among the tested population and individuals with confirmed COVID-19 diagnosis: a systematic review and meta-analysis. JAMA Netw Open. (2021) 4:e2137257. doi: 10.1001/jamanetworkopen.2021.37257

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Bar-On Bar-On YM, Goldberg Y, Mandel M, Bodenheimer O, Freedman L, Kalkstein N, et al. Protection of BNT162b2 vaccine booster against covid-19 in Israel. N Engl J Med. (2021) 385:1393–400. doi: 10.1056/NEJMoa2114255

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Polack FP, Thomas SJ, Kitchin N, Absalon J, Gurtman A, Lockhart S, et al. Safety and efficacy of the BNT162b2 mRNA covid-19 vaccine. N Engl J Med. (2020) 383:2603–15. doi: 10.1056/NEJMoa2034577

PubMed Abstract | CrossRef Full Text | Google Scholar

50. European Centre for Disease Prevention and Control. Overview of the implementation of COVID-19 vaccination strategies and deployment plans in the EU/EEA. Eur Cent Dis Prev Control. (2022)

51. Kronbichler A, Kresse D, Yoon S, Lee KH, Effenberger M, Shin JI. Asymptomatic patients as a source of COVID-19 infections: a systematic review and meta-analysis. Int J Infect Dis. (2020) 98:180–6. doi: 10.1016/j.ijid.2020.06.052

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Mossong J, Hens N, Jit M, Beutels P, Auranen K, Mikolajczyk R, et al. Social contacts and mixing patterns relevant to the spread of infectious diseases. PLoS Med. (2008) 5:e74. doi: 10.1371/journal.pmed.0050074

PubMed Abstract | CrossRef Full Text | Google Scholar

53. Poletti P, Tirani M, Cereda D, Trentini F, Guzzetta G, Sabatino G, et al. Probability of symptoms and critical disease after SARS-CoV-2 infection. JAMA Netw Open. (2021) 4:e211085. doi: 10.1001/jamanetworkopen.2021.1085

PubMed Abstract | CrossRef Full Text | Google Scholar

54. Thompson RN, Lovell-Read FA, Obolski U. Time from symptom onset to hospitalisation of coronavirus disease 2019 (COVID-19) cases: implications for the proportion of transmissions from infectors with few symptoms. J Clin Med. (2020) 9:1297. doi: 10.3390/jcm9051297

PubMed Abstract | CrossRef Full Text | Google Scholar

55. Singer BJ, Thompson RN, Bonsall MB. Evaluating strategies for spatial allocation of vaccines based on risk and centrality. J R Soc Interface. (2020) 19:20210709. doi: 10.1101/2021.09.07.21263209

PubMed Abstract | CrossRef Full Text | Google Scholar

56. European Centre for Disease Prevention and Control. European Centre for Disease Prevention and Control. Overview of the implementation of COVID-19 vaccination strategies and deployment plans in the EU/EEA. 21 April 2022. Stockholm: ECDC; 2022. Vaccine. (2019) 37:7190–200.

57. World Health Organization. Second Ebola Vaccine to Complement “Ring Vaccination” given Green Light in DRC. (2020). Available online at: https://www.who.int/news/item/23-09-2019-second-ebola-vaccine-to-complement-ring-vaccination-given-green-light-in-drc (accessed June 11, 2022).

58. Merler S, Ajelli M, Fumanelli L, Parlamento S, Piontti AP, Dean NE, et al. Containing Ebola at the source with ring vaccination. PLoS Negl Trop Dis. (2016) 10:e0005093. doi: 10.1371/journal.pntd.0005093

PubMed Abstract | CrossRef Full Text | Google Scholar

59. Iacobucci G. Covid-19: local councils initiate surge vaccination to tackle B.1.617.2 variant. BMJ. (2021) 373:n1361. doi: 10.1136/bmj.n1361

PubMed Abstract | CrossRef Full Text | Google Scholar

60. SPI-M-O: Consensus statement on COVID-19. GOVUK. (2021). Available online at: https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/986709/S1237_SPI-M-O_Consensus_Statement.pdf (accessed June 11, 2022).

61. Bullock J, Lane JE, Shults FL. What causes COVID-19 vaccine hesitancy? Ignorance and the lack of bliss in the United Kingdom Humanit. Soc Sci Commun. (2022) 9:87. doi: 10.1057/s41599-022-01092-w

CrossRef Full Text | Google Scholar

62. Willis DE, Andersen JA, Bryant-Moore K, Selig JP, Long CR, Felix HC, et al. COVID-19 vaccine hesitancy: race/ethnicity, trust, and fear. Clin Transl Sci. (2021) 14:2200–7. doi: 10.1111/cts.13077

PubMed Abstract | CrossRef Full Text | Google Scholar