Enhancing mass vaccination programs with queueing theory and spatial optimization

Background: Mass vaccination is a cornerstone of public health emergency preparedness and response. However, injudicious placement of vaccination sites can lead to the formation of long waiting lines or queues, which discourages individuals from waiting to be vaccinated and may thus jeopardize the achievement of public health targets. Queueing theory offers a framework for modeling queue formation at vaccination sites and its effect on vaccine uptake.

Methods: We developed an algorithm that integrates queueing theory within a spatial optimization framework to optimize the placement of mass vaccination sites. The algorithm was built and tested using data from a mass dog rabies vaccination campaign in Arequipa, Peru. We compared expected vaccination coverage and losses from queueing (i.e., attrition) for sites optimized with our queue-conscious algorithm to those used in a previous vaccination campaign, as well as to sites obtained from a queue-naïve version of the same algorithm.

Results: Sites placed by the queue-conscious algorithm resulted in 9–32% less attrition and 11–12% higher vaccination coverage compared to previously used sites and 9–19% less attrition and 1–2% higher vaccination coverage compared to sites placed by the queue-naïve algorithm. Compared to the queue-naïve algorithm, the queue-conscious algorithm placed more sites in densely populated areas to offset high arrival volumes, thereby reducing losses due to excessive queueing. These results were not sensitive to misspecification of queueing parameters or relaxation of the constant arrival rate assumption.

Conclusion: One should consider losses from queueing to optimally place mass vaccination sites, even when empirically derived queueing parameters are not available. Due to the negative impacts of excessive wait times on participant satisfaction, reducing queueing attrition is also expected to yield downstream benefits and improve vaccination coverage in subsequent mass vaccination campaigns.

1 Introduction

The expeditious and equitable distribution of vaccinations and other health services is a cornerstone of public health emergency preparedness. Queues, or waiting lines, result from scarce or misallocated resources and volatility in traffic and service patterns; they can hinder the delivery of critical services and thereby jeopardize the achievement of public health targets. Not only can long queues deter people from waiting to receive essential health services, they can erode individuals’ trust in health systems in certain contexts (1, 2) and can thus discourage participation in future programs. Long wait times was a major structural barrier to testing for COVID-19 during the early phase of the pandemic (3), and poor planning in some jurisdictions resulted in people waiting hours at some mass COVID-19 vaccination sites (4–6). Moreover, excessive queueing during pandemic emergencies also poses health risks, as long wait times may increase exposure to infectious pathogens (7), underscoring the need for safe and efficiently managed healthcare settings (8, 9).

Queueing theory is a branch of applied mathematics that offers a valuable framework for studying the behaviors and effects of waiting lines or queues (10). In brief, queueing models aim to capture how a customer population moves through a queueing system via a series of processes dictated by probabilistic rates: arriving at a service site, receiving service, waiting in a queue if the server is busy, or leaving the queue before service is rendered when waiting times exceed a customer’s willingness to wait. Queueing theory is foundational to operations research and has been applied to many facets of healthcare operations, including the triage process in emergency care departments (11), staffing needs in operating rooms (12), hospital bed management (13, 14), and outpatient scheduling (15). Additionally, it has been applied to COVID-19 vaccine distribution and capacity planning (7, 16–20), as well as the containment of disease outbreaks, bioterrorist attacks, and other public health emergencies (21–24).

Mass dog vaccination campaigns (MDVCs) are held annually in Arequipa, Peru to address the re-emergence of dog rabies in the region (25); they have important parallels with early pandemic vaccination and testing programs in that success depends, in part, on strategically placing and optimally allocating resources across a discrete number of fixed-location facility sites (26). While the World Health Organization (WHO) and Pan American Health Organization (PAHO) recommend a minimum vaccination coverage of 70–80% sustained over multiple years to achieve control and eventual elimination of rabies, the MDVCs in Arequipa, which have relied on convenient or ad hoc placement of fixed-location vaccination sites, have continually fallen short of this goal (27, 28).

We have previously developed a data-driven strategy to optimize the placement of fixed-location MDVC sites and found that spatially optimized vaccination sites improves both overall vaccination coverage and spatial evenness of coverage (28). However, optimization that addresses spatial accessibility without considering queueing is likely to result in an uneven volume of arrivals across facility sites, which may result in long waiting lines (28). Here, we incorporate queueing theory into our existing spatial optimization framework to improve dog rabies vaccine uptake by accounting for both the spatial accessibility of MDVC sites and losses resulting from dog owners who refuse to wait for service in the face of excessive queue lengths (i.e., queueing attrition). We compare the performance of our queue-conscious algorithm to the queue-naïve algorithm in terms of expected vaccination coverage and queueing attrition and evaluate the sensitivity of our results to misspecification of queueing parameters and the assumption of a constant arrival rate within our queueing model.

2 Materials and methods

2.1 A queueing model for MDVCs

We modeled queueing, vaccination, and attrition at each MDVC vaccination site according to an M/M/1 system with first-in-first-out (FIFO) service (Figure 1). The M/M/1 system is a widely used queueing model for single server systems and assumes that customer arrivals occur according to a Poisson process, and job service times are independent and identically distributed (iid) exponential random variables that are independent of the arrival process and queue length. Applied to MDVCs, the M/M/1 queueing model assumes that dogs arrive with their owners to a vaccination site according to a Poisson process with arrival rate λ, meaning that the interarrival times are iid and follow an exponential distribution with parameter λ. The service times (i.e., the time it takes for a dog to get vaccinated) are iid exponential with parameter μ, such that the average service time is equal to 1/μ. The system is assumed to be FIFO, meaning that dogs are vaccinated in the order that their owners join the queue. Only one dog can get vaccinated at a time, as there is only one vaccinator per site, and dogs are assumed to leave the system as soon as they get vaccinated.

Figure 1

Figure 1. An M/M/1 first-in-first-out queueing model for an MDVC vaccination site. (A) Illustrates the processes captured by the queueing model, with the forms of queuing attrition highlighted by the red boxes. (B) Shows the transition-state diagram for the queueing model, where states, depicted by circles, are defined by the number of dogs in the system, and transitions between states, depicted with curved arrows, are labeled by their corresponding transition rates.

The service rate μ was assumed to equal 30 h⁻¹ in accordance with the empirical observation that it takes 2 min on average to vaccinate a dog. The arrival rates were assumed to vary across MDVC sites and were determined as follows. First, the MDVC participation probability function described above was applied to all households falling within an MDVC site’s catchment (i.e., all houses closest to the given MDVC site in terms of travel distance) to determine the probability that each household would participate in the MDVC if the house were inhabited and owned dogs. To obtain the total number of dogs arriving at MDVC site s, these participation probabilities were summed and scaled by the habitability rate, household-dog-ownership rate, and average number of dogs per dog-owning household (57, 40%, and 1.86, respectively); these estimates were derived from household surveys administered following previous MDVCs, and the survey methodology has been described previously (25). The total number of arrivals was then divided by the total operation time for the MDVC site to obtain λ_s, the arrival rate for site s.

A dog enters the queueing system at site s after it arrives at the site and its owner elects to join the vaccination queue. However, some owners may decline to join the queue if they judge the queue to be too long. This first form of attrition is known as balking and was modeled by modifying the arrival rate λ_s so that it decreases by a discouragement factor e^-αn/μ < 1 (10). The modified arrival rate λ_s,n captures the rate that owners join the queue after accounting for those that balk and is given by:

$\begin{array}{ll}{\lambda }_{s,n}=e^{-\alpha n/\mu }·{\lambda }_s& (1)\end{array}$

Where n is the number of dogs that are currently in the system (waiting in queue or being vaccinated), μ is equal to the service rate, and α is a parameter that scales with balking propensity (10).

The other form of attrition, known as reneging, occurs when an owner who has already joined the queue loses patience and exits the queue before their dogs are vaccinated. We modeled reneging by modifying the service rate μ to capture all those leaving the system - both those leaving after vaccination and those who renege. This exit rate μ_n is equal to:

$\begin{array}{ll}{\mu }_n=\mu +\left(n-1\right)·\beta & (2)\end{array}$

Where the second term captures the rate that each of the present n – 1 dogs in queue are reneging, and β scales with reneging propensity (29). Note that in Equations 1, 2, above, the rates of attrition (both balking and reneging) increase with the queue length n – 1, as expected.

In order to calculate the expected number of dogs vaccinated during an MDVC, we need to find a closed-form expression for the vaccination rate at a given vaccination site that accounts for losses due to attrition. The derivation of these closed-form equations can be found in Supplementary Text A, and are based on the stationary distribution of the queueing model, i.e., on $p_{s,n}$ , the probability of finding n dogs in the queueing system at MDVC site s with arrival rate λ_s:

$\begin{array}{ll}{p}_{s,n}=\frac{e^{\frac{-\alpha n\left(n-1\right)}{2\mu }}{\lambda }_s^n\Gamma \left(\frac{\mu }{\beta }\right)}{{\beta }^n\Gamma \left(n+\frac{\mu }{\beta }\right)}{p}_{s,0}& (3)\end{array}$

Where $\Gamma \left(z\right)$ denotes the gamma function, i.e., $\Gamma \left(n\right)=\left(n-1\right)!$ for any integer $n>0$ and $\Gamma \left(z\right)=\int t^{z-1}{e}^{-t}dt$ interpolates the factorial function to non-integer values, and p_s,0 is a normalizing constant given by:

$\begin{array}{ll}{p}_{s,0}={\left[1+{\displaystyle \sum }_{n=1}^{\infty }\frac{{\lambda }_s^n{e}^{\frac{-\alpha n\left(n-1\right)}{2\mu }\Gamma \left(\frac{\mu }{\beta }\right)}}{{\beta }^n\Gamma \left(n+\frac{\mu }{\beta }\right)}\right]}^{-1}& (4)\end{array}$

The expected rate that dogs are vaccinated at site s is then equal to:

$\begin{array}{ll}{v}_s={\displaystyle \sum }_{n=0}^{\infty }{p}_{s,n}{\lambda }_{s,n}-{\displaystyle \sum }_{n=1}^{\infty }{p}_{s,n}\left(n-1\right)\beta & (5)\end{array}$

Where the first term is equal to the rate that dog owners join the queue after accounting for balking, and the second term is equal to the rate that dog owners renege and thus leave the queue before their dogs are vaccinated. The expected number of dogs vaccinated during an MDVC is thus equal to:

$\begin{array}{ll}V={\displaystyle \sum }_{s\in S}{v}_{s}t& (6)\end{array}$

Where S is the set of all selected vaccination sites and t is equal to the total operation time, which is assumed to be the same for all MDVC sites.

In addition to the closed-form equations for the expected behavior of the MDVC queueing system, which were derived assuming the system had reached steady state (Equations 3–6), we also conducted stochastic simulations to study the behavior of the system in the absence of such assumptions. Simulations were conducted for low- and high-attrition parameter regimes (low: α = 0.01 and β = 0.02; high: α = 0.1 and β = 0.1) and for a range of arrival rates (0.5–37.5 dogs/h in increments of 0.5 dogs/h). Low- and high-attrition parameter regimes were chosen to represent the high and low range of feasible values based on our observations of balking and reneging at MDVC sites. An MDVC site operates for four weekend days (over two weekends) for about 4 h per day (t = 16 total hours). To mimic these conditions, a single simulation consisted of four independent four-hour-long trials (days), each initialized with no dogs in the queue at time zero; the number of dogs vaccinated each day was summed across the 4 days to obtain the total dogs vaccinated at an MDVC site. The simulation was run for 1,000 iterations per set of parameter values, and the simulation results were compared to the expected number of dogs vaccinated as determined via the closed-form equations to see how well the two approximated each other.

2.2 Optimizing the location of vaccination sites

We optimized the placement of MDVC sites for the Alto Selva Alegre district of Arequipa; no more than 20 sites can operate in this region during a campaign due to resource constraints, and 70 locations have been approved by the Ministry of Health for use as feasible MDVC sites (Figure 2) (28). We determined the optimal placement of k = 20 sites among these 70 candidate sites by maximizing the expected number of households participating in the MDVC (and hence the total dogs vaccinated). To determine the number of participating households, we first used a fixed-effects Poisson distance-decay function that links a household’s travel distance to their nearest vaccination site and their probability of participating in an MDVC (henceforth referred to as the “MDVC participation probability function”); this function was fit previously using survey data (28). We assumed that participating households travel to their closest MDVC site, and we used the MDVC participation probability function to estimate the number of households that are expected to arrive at each site. We divided the number of arrivals by the total operation time for an MDVC site (i.e., 16 h) to calculate the arrival rate λ_s at each site s. Then, for queue-conscious optimization, we estimated the number of dogs vaccinated at each site using Equation 5, which accounts for attrition resulting from queue formation due to high arrival rates. Queue-naïve optimization, in contrast, assumes that all arriving dogs get vaccinated and thus does not account for queueing-related losses. The objective function (total vaccinated dogs) was then calculated by summing the number of dogs vaccinated at each site.

Figure 2

Figure 2. Potential vaccination site locations in Alto Selva Alegre. The boundaries of Alto Selva Alegre are depicted by the solid, black line. Candidate MDVC sites (N = 70) are indicated by red diamonds, and the locations of houses are shaded brown.

We performed queue-conscious and queue-naïve optimization by implementing a hybrid recursive interchange-genetic algorithm (Supplementary Text B and Supplementary Figures S1, S2). The recursive interchange portion of our algorithm is similar to Teitz and Bart’s (30) solution to the p-median problem that solves the facility location problem by minimizing the average distance traveled by all households to their nearest site, but instead of minimizing average travel distance, our algorithm aims to maximize the expected number of households participating in the MDVC, which allowed our queue-conscious optimization algorithm to simultaneously account for travel distance and queue-length-dependent attrition rates.

The general steps of the recursive interchange algorithm are as follows:

1. Select a random subset of 20 vaccination sites and use the MDVC participation probability function to determine the expected arrival rate λ at each site.

2. Calculate the expected number of dogs vaccinated at each site and sum across all sites to calculate the total number of dogs vaccinated.

3. Exchange one selected site with all non-selected candidate locations and keep the one that maximizes the number of dogs vaccinated.

4. Repeat step 3 with remaining sites to obtain a locally optimized set of sites.

5. Perform steps 1–4 over 1,000 iterations, initializing each iteration with a different random subset of sites.

An animation showing a single iteration of the recursive interchange algorithm can be viewed in the Supplementary materials. The recursive interchange algorithm was repeated over 1,000 iterations to increase performance, as the algorithm does not guarantee a globally optimal solution. Performance was further enhanced by combining the recursive interchange algorithm with a genetic algorithm that “mates” parental sets output by the recursive interchange algorithm, mimicking natural selection by introducing crossover and mutation and ultimately producing new starting sets on which to repeat the recursive interchange algorithm. The cycling between the recursive interchange and genetic algorithms was repeated until the expected number of dogs vaccinated did not increase over two subsequent rounds of optimization (stopping condition). A full description of the hybrid algorithm can be found in the Supplementary Text B.

MDVC sites were optimized under three scenarios: no attrition (α = β = 0), low attrition (α = 0.01, β = 0.02), and high attrition (α = 0.1, β = 0.1). Note the no-attrition scenario is the least realistic, as some degree of balking and reneging is expected to occur in the real world. The low- and high-attrition queue-conscious solutions were compared to the queue-naïve solution obtained under the assumption of no attrition (i.e., all dogs that arrive get vaccinated) to determine how the incorporation of queueing behaviors impacted the amount of dogs lost to attrition and the total vaccination coverage, which was calculated as the proportion of dog-owning households that are expected to participate in the MDVC. Additionally, the queue-conscious solutions were compared to the locations of actual sites used in the 2016 MDVC to evaluate how the performance of sites placed by the queue-conscious algorithm compared to a real-world baseline (28). Note that although the queue-naïve solution to the location problem was obtained assuming no attrition, its performance was assessed under the assumption of a low- or high-attrition parameter regime. Additionally, the optimized sites were mapped along with their catchments to compare how site placement varied between the queue-conscious and queue-naïve solutions.

2.3 Sensitivity analyses

To determine how our results may have been impacted by misspecification of α and β, we considered four possible scenarios for true balking and reneging propensities. In addition to the low- and high-attrition scenarios discussed previously (α = 0.01/β = 0.02 and α = 0.1/β = 0.1, respectively), we considered two additional scenarios for true balking and reneging propensities: (1) low balking and high reneging (α = 0.01, β = 0.1) and (2) high balking and low reneging (α = 0.1, β = 0.02). We applied the low- and high-attrition solutions to these four scenarios to evaluate performance (in terms of number of vaccinations and losses to attrition) for situations in which α and β are correctly and incorrectly specified. For each scenario and queue-conscious solution applied, performance was evaluated using the number vaccinated and losses to attrition achieved by the queue-naïve solution as a benchmark.

The optimization methods detailed above rely on the use of the closed-form equations for the queueing system, which assumes a constant arrival rate λ. We considered how this assumption impacted our results by allowing λ to vary in a step-wise manner to approximate time-varying arrival rates that have been observed in the field (Supplementary Figure S3). Four time-varying arrival densities were considered: (a) a steep unimodal peak density, (b) a wide unimodal density that is skewed right, (c) a wide unimodal density that is skewed left, and (d) a bimodal density distribution (Supplementary Figure S4). Eight total scenarios were considered, representing all combinations of the four time-varying arrival densities and low- and high-attrition parameter regimes. Queueing simulations were performed for each scenario, and natural splines were used to summarize the behavior of the system over a range of arrival rates (Supplementary Text C and Supplementary Figure S5). Once again, the performance of the low- and high-attrition solutions were assessed for each scenario, using performance under the queue-naïve solution as a benchmark. Additionally, the different non-constant arrival rate densities were compared to the baseline assumption of a constant arrival rate to determine how this assumption impacted estimations of the number of vaccinations and losses to attrition.

3 Results

3.1 Queue-conscious optimization for MDVCs

As expected, the amount of balking and reneging was greater for higher arrival rates and for higher α and β values, representing greater attrition propensity (Figure 3). Although the closed-form expression for the expected number of vaccinations (Equation 6) was derived under steady-state assumptions, the results of the stochastic simulations closely approximated results obtained using Equation 6 across a range of arrival rates for both high- and low-attrition parameter regimes (root mean square percentage error < 2% for both regimes; Supplementary Figure S6). Thus, Equation 6 was used as the objective function in the hybrid algorithm that was used to optimize MDVC site placement.

Figure 3

Figure 3. Realized trials of the stochastic queueing model. Each trial of the stochastic queueing simulation represents a single four-hour day at an MDVC site. The gray-shaded portion of each plot tracks the queue length over the four-hour period, and the colored shapes in the white portion of each plot tracks the occurrences of balking (red triangles), reneging (red diamonds) and vaccination (blue circles). The number of balking events (B), reneging events (R), and vaccinations (V) are reported for each trial. Trials are shown for two different α/β parameter regimes (low: α = 0.01, β = 0.02 and high: α = 0.1, β = 0.1) and two different arrival rates (15 and 30 dogs per hour).

Compared to the queue-naïve algorithm, the queue-conscious algorithm favored a more even distribution in the number of arrivals across all selected sites (Figure 4). The queue-conscious algorithm “flattens” the distribution of arrivals by placing more sites in densely populated areas to divide the higher vaccination workload across more vaccinators and placing fewer sites in less populous areas (Figure 5). This difference in site distribution is expected, because too many arrivals at a site result in the formation of long queues and more losses from balking and reneging; these losses are accounted for (penalized) by the queue-conscious algorithm but not by the queue-naïve algorithm, which assumes that all arrivals get vaccinated. This difference between the queue-naïve and queue-conscious algorithms also results in the queue-naïve algorithm yielding more arrivals, as it maximizes the number of participating households simply by maximizing the number of arrivals.

Figure 4

Figure 4. Arrivals histograms for sites selected by queue-naïve and queue-conscious optimization compared to actual sites used in the 2016 MDVC assuming low- and high-attrition parameter values. The height of each stacked bar represents the expected number of dogs that arrive at a selected vaccination site. Bars are subdivided by color according to whether dogs ultimately get vaccinated (blue) or are lost to attrition, either through balking (dark red) or reneging (light red). The text above the bars gives the total number of arrivals, total losses to attrition, and overall vaccination coverage achieved for each set of sites. Top row shows results assuming a low-attrition parameter regime, and bottom row shows results for a high-attrition parameter regime. The number of dogs vaccinated and the number of dogs lost to attrition for all situations were determined using Equation 6 and the equations outlined in the electronic Supplementary Text A.

Figure 5

Figure 5. Locations of MDVC sites selected by the queue-naïve vs. queue-conscious algorithm for the low- and high-attrition systems. The locations of selected vaccination sites are indicated by white circles that are labeled and scaled according to the expected number of arriving dogs, which were calculated using Equation 6. Top row shows results for the low-attrition system, and bottom row shows results for the high-attrition system. Houses in the study area are small dots colored according to their catchment, representing the area in which a MDVC site is the closest site for houses in terms of travel distance. Areas in which the queue-conscious algorithm placed a higher density of vaccination sites compared to the queue-naïve algorithm are indicated by ellipses with solid lines, and areas in which the queue-conscious algorithm placed one fewer site are indicated by ellipses with dotted lines.

Within the low-attrition system (α = 0.01, β = 0.02), vaccination sites that were placed using the queue-conscious algorithm achieved an expected vaccination coverage of 57.2% compared to 56.4% achieved by the queue-naïve algorithm (Table 1 and Figure 4). The amount of queueing attrition (i.e., the expected number of dog owners balked or reneged) was also lower for sites placed using the queue-conscious algorithm: 596 vs. 733 for the queue-naïve algorithm, representing a 19% reduction. Trends were similar for the high-attrition system (α = 0.1, β = 0.1), in which the queue-conscious algorithm improved the expected vaccination coverage from 47.2 to 48% and reduced queueing attrition by 9% from 1,727 to 1,566. Queue-conscious optimization resulted in markedly superior performance when compared to that of historic MDVC sites, increasing expected vaccination coverage from 50.9 to 57.2% in the low-attrition regime and from 43.2 to 48% in the high-attrition regime; it also decreased queueing attrition by 32% (from 882 to 596) and 9% (from 1,721 to 1,566) in the low- and high-attrition regimes, respectively (Table 1 and Supplementary Figure S7).

Table 1

Table 1. Performance of vaccination sites placed by queue-naïve and queue-conscious optimization compared to actual sites used in 2016 for low- and high-attrition parameter regimes.

3.2 Sensitivity analyses

These results were robust to misspecification of α and β, and the performance varied only slightly between the high- and low-attrition solutions for all combinations of α and β considered (Figure 6). When the true values of α and β are low (α = 0.01 and β = 0.02), overestimating these parameters in the optimization did not result in a substantial loss in the number of dogs vaccinated compared to the correctly optimized solution (82 vs. 84 more dogs vaccinated beyond the queue-naïve solution). Similarly, when the true values of α and β are high (α = β = 0.1), underestimating these parameters in the optimization did not markedly impact the number of dogs vaccinated compared to the correctly optimized solution (83 vs. 85 more dogs vaccinated beyond the queue-naïve solution). Moreover, applying the low- and high-attrition solutions resulted in a similar number of dogs vaccinated when the true value of α is low and the true value of β is high and vice-versa (Figure 6A). The high-attrition solution resulted in a greater reduction in queueing attrition than the low-attrition solution for all four attrition scenarios, though both solutions resulted in substantially fewer losses compared to the queue-naïve solution (Figure 6B). Taken together, these results demonstrate that the queue-conscious algorithm outperforms the queue-naïve algorithm even in the presence of mis-specified queueing parameters.

Figure 6

Figure 6. Sensitivity of results to misspecification of balking and reneging parameters. Panels a-b illustrate how misspecification of α and β impacts the expected number of dogs vaccinated (A) and the number of dogs lost to attrition (B). The performance of the low- and high-attrition solutions are provided with the queue-naïve solution acting as a benchmark; thus (A) shows the additional number of dogs vaccinated beyond the expected number achieved with the queue-naïve solution, and (B) shows the reduction in attrition compared to the queue-naïve solution. Bars outlined in bold represent scenarios in which the balking and reneging parameters are correctly estimated in the optimization. (C) Provides a legend with the values of α and β for the four balking/reneging scenarios considered.

The superior performance of the queue-conscious algorithm compared to the queue-naïve algorithm was also robust to relaxation of the constant arrival rate assumption. For all four time-varying arrival densities and attrition regimes, both low- and high-attrition solutions substantially outperformed the queue-naïve solution in terms of the numbers vaccinated and lost to attrition (Supplementary Figure S7). Interestingly, with the exception of arrival density D under a low-attrition regime, for which the low- and high-attrition solutions yielded roughly equal numbers of vaccinations, the high-attrition solution outperformed the low-attrition solution in terms of the numbers vaccinated. The high-attrition solution also resulted in less queueing attrition than the low-attrition solution for all scenarios considered. In addition, non-constant arrival rates resulted in more queueing attrition and fewer dogs vaccinated compared to an otherwise equivalent scenario where the constant arrival rate assumption is met (Supplementary Figure S8).

4 Discussion

We developed an optimization algorithm that integrates queueing theory into a spatial optimization framework to improve the placement of mass vaccination sites. We applied our algorithm to the MDVC in Arequipa, Peru by simultaneously minimizing travel distance to MDVC sites and queueing attrition resulting from large arrival volumes at some sites. Our queue-conscious algorithm decreased queueing attrition by 9–32% and increased expected vaccination coverage by 11–12% compared to actual sites used in a previous MDVC and decreased queueing attrition by 9–19% and increased expected vaccination coverage by 1–2% compared to a queue-naïve version of the same algorithm. MDVC site optimization that accounted for queueing placed more vaccination sites in densely populated areas to even out the number of expected arrivals across sites, and sensitivity analyses revealed that accounting for queueing resulted in improved MDVC performance, even in the absence of accurate parameter estimates. Moreover, the expected gains in vaccination coverage do not capture the indirect gains from reduced queueing and increased MDVC participant satisfaction, which is likely to improve turnout in subsequent campaigns.

Longer wait times have been negatively associated with patient satisfaction in a variety of healthcare contexts, and patients report being less likely to repeatedly patronize a medical practice with long wait times compared to one with shorter wait times (1, 31, 32). For dog rabies vaccination, individuals who must wait a long time before receiving vaccinations for their dogs may be far less likely to participate in subsequent vaccination campaigns. Furthermore, considering the evidence of social contagion around vaccines (33–37), dog owners could share their negative experiences waiting at an MDVC site with friends and neighbors, discouraging them from participating. The reduction of attrition resulting from well-placed vaccination sites may pay dividends in improving turnout and vaccination coverage in subsequent MDVCs; this is particularly important for dog rabies elimination, which requires sustained high levels of vaccination year after year (38–40).

We assumed that owners arrived with their dogs to MDVC sites at time-invariant rates. The rationale behind this assumption was twofold: (1) it ensured tractability of the queueing equations, and (2) it was unclear how to specify a non-constant arrival rate in the face of heterogeneity in the trajectory of rates observed at MDVC sites (Supplementary Figure S3). Our sensitivity analysis indicated that the queue-conscious solutions outperformed the queue-naïve solutions even when arrival rates varied over time (Supplementary Figure S7). We also found that non-constant arrival rates resulted in more queueing attrition and fewer dogs vaccinated than the baseline assumption of a constant arrival rate (Supplementary Figure S8). This result can be explained by the fact that a time-varying arrival density leads to swells of arrivals during peak intervals, when queue lengths would escalate and cause attrition to spike.

Surprisingly, the high-attrition solution performed as well as or better than the low-attrition solution for all time-varying arrival scenarios, even those in which the true attrition rates were low (Supplementary Figure S7). This result can be explained by the spikes in attrition that accompany time-varying arrival rates but are not captured by the low-attrition solution, which are obtained under the assumption of a constant arrival rate. As a result, even when α and β are low, the expected vaccination rate is higher with the high-attrition solution, as it favors a more even distribution in the number of arrivals across vaccination sites (compare top vs. bottom rows of Figures 4, 5). These results suggest that applying MDVC optimization in the real world is as much an art as it is a precise science. Even if the “true” balking and reneging rates could be determined, it may be beneficial to slightly overestimate these parameters to offset the reality of non-constant arrival rates.

The queue-conscious algorithm we employed decreases queue lengths across the study area, but some queueing is inevitable. Attrition can be minimized further by improving the waiting experience for queueing dog owners (41, 42). In the context of MDVCs, accommodations should be made for aggressive dogs, whose presence in a queue can cause other owners to balk or renege. Some vaccinators may choose to deviate from FIFO principles and vaccinate aggressive dogs first regardless of when they arrive to remove them from the queue more quickly. This strategy should be explained clearly to the owners present as violations of FIFO are generally perceived as unfair (42, 43). MDVC participant satisfaction should be prioritized wherever possible, as it impacts whether individuals will continue to participate in future MDVCs. Other behavioral interventions that can minimize queueing attrition are messaging and incentives to flatten out the arrival rate. Field observations show arrival peaks, longer queue lengths, and greater attrition at midday (Supplementary Figure S3). Attrition during these peaks can be mitigated by communicating about shorter wait times early in the morning or incentivizing early arrivals by rewarding a limited quantity of “doorbuster” prizes (e.g., dog food or dewormer medication).

The expected vaccination coverage achieved by our optimization of fixed-location vaccination sites (57 and 48% for the low- and high-attrition scenarios, respectively) falls short of the 70–80% threshold recommended by World Health Organization (38) and Pan American Health Organization (44). This gap can be met, in part, by combining fixed-location vaccination sites with door-to-door vaccination in areas with low penetration by the fixed-location campaign. This two-pronged approach has been leveraged successfully in other MDVCs (45, 46) as well as pandemic-era COVID-19 vaccination programs (47, 48). A benefit of combining door-to-door vaccination with fixed-point vaccination is the ability to target high-risk or underserved areas, which not only increases total vaccine uptake but also promotes vaccine equity. We have previously found that the queue-naïve algorithm increases the spatial evenness of vaccine coverage, a dimension of vaccine equity, even though it does not explicitly optimize for spatial equity (28). By placing more vaccination sites in more populous areas and limiting the placement of sites in less populous ones, the queue-conscious algorithm inadvertently decreases the spatial equity of fixed-point vaccinations compared to the queue-naïve algorithm, which is a limitation of the queue-conscious approach. In many Latin American cities, including Arequipa, the less populous peri-urban areas also coincide with areas of greater socioeconomic disadvantage (25, 26); thus, it is crucial for peri-urban areas to be prioritized by door-to-door campaigns following the deployment of fixed-point vaccination sites to ensure vaccine equity. Disadvantaged groups face the greatest barriers in accessing health services and are thus least able to travel to vaccination sites and wait for service (49–51). They might benefit the most from this combined approach.

There are other limitations of our study. The balking and reneging parameters α and β were not estimated from data but selected to model two hypothetical parameter regimes that fell within the upper and lower bounds of values that could feasibly capture real-world dynamics. While this lack of empirical estimation is a study limitation, our sensitivity analyses also indicated that the performance of our optimization algorithm was robust to misspecification of these parameters. In addition, the MDVC participation probability function that was used to optimize vaccination site locations included distance to the nearest site as a sole predictor and did not consider other household-level factors such as socioeconomic status (SES) or local environment factors such as urban/peri-urban status. Future studies can investigate how travel distance to MDVC sites affect MDVC participation among different household SES levels and across urban and peri-urban areas to derive a more nuanced MDVC participation function. Doing so can also be a means of promoting vaccine equity; for example, if future investigations revealed that marginalized groups are less able to travel long distances to participate in the MDVC, then the algorithm using this “updated” function would favor placing more sites near marginalized populations. Additionally, deviations from a constant arrival rate in the real world may impact the generalizability of our results, though our sensitivity analyses suggested that the superior performance of the queue-conscious algorithm was robust to relaxation of the constant arrival rate assumption. Finally, our algorithm assumed that all MDVC sites were operated by a single vaccinator (i.e., M/M/1). As a result, the algorithm tended to place multiple, adjacent single-vaccinator sites in highly populous areas. There are generally efficiency gains associated with multi-server (i.e., multi-vaccinator) queueing systems (where multiple vaccinators serve a single queue) compared to single-server systems with designated queues (10). However, pooling vaccinators (i.e., placing k vaccinators across fewer than k sites) may also lead to performance loss, as reducing the number of sites could result in longer queues, which may increase perceived waiting times and result in greater attrition (52); reducing the number of sites may also increase travel distances for some dog owners and thus decrease their probability of participation. A possible extension of our work would be to examine the tradeoff between gains from pooling vaccinators and losses due to slightly longer travel distances and potentially longer queue lengths.

In summary, our spatial optimization framework that incorporates expected losses from queueing offers insights for current vaccine-preventable disease programs and for future pandemic preparedness efforts. We maximized the total vaccine uptake by enhancing the spatial accessibility of vaccination sites while mitigating excessive queue lengths to reduce losses due to queueing attrition. We found that explicitly modeling queueing behavior, even with imprecise parameter estimates, led to gains in vaccination coverage and fewer losses to attrition than optimization that ignores the effects of queueing. Combined with door-to-door outreach and targeted media campaigns, rational placement of fixed-point vaccination sites is expected to bring vaccine uptake closer to threshold levels recommended for the control and eventual elimination of dog rabies. Considering the impact of excessive wait times on other vaccination campaigns, including the early rollout of the COVID-19 vaccine, our spatial optimization framework that explicitly considers queueing attrition can be broadly adopted to support other mass vaccination programs.

Data availability statement

The datasets presented in this study can be found in online repositories. The names of the repository/repositories and accession number(s) can be found at: https://github.com/sherriexie/spatialoptimizationqueueing. Note that identifiable data (residential geocodes) have been removed from the data repository.

Ethics statement

The studies involving humans were approved by the Institutional Review Boards of Universidad Peruana Cayetano Heredia (approval number: 65369) and University of Pennsylvania (approval number: 823736). The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.

Author contributions

SX: Conceptualization, Formal analysis, Investigation, Methodology, Project administration, Software, Validation, Visualization, Writing – original draft, Writing – review & editing. MR: Conceptualization, Formal analysis, Methodology, Writing – review & editing. SC: Formal analysis, Writing – review & editing. BB: Methodology, Writing – review & editing. ED: Data curation, Writing – review & editing. ML: Conceptualization, Methodology, Writing – review & editing. RC-N: Conceptualization, Funding acquisition, Methodology, Project administration, Supervision, Writing – review & editing.

Funding

The author(s) declare that financial support was received for the research, authorship, and/or publication of this article. This study was supported by the National Institute of Allergy and Infectious Diseases (grant nos. K01AI139284 and R01AI168291) to RC-N.

Acknowledgments

We gratefully acknowledge the contributions of and the work done by the Gerencia Regional de Salud de Arequipa, the Red de Salud Arequipa Caylloma, the Laboratorio Referencial Regional Arequipa, and the Microredes of the city of Arequipa. We acknowledge the work of the members of the Zoonotic Disease Research Laboratory, One Health Unit, and their contribution collecting part of the data used in this study.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpubh.2024.1440673/full#supplementary-material

References

1. Bleustein, C, Rothschild, DB, Valen, A, Valatis, E, Schweitzer, L, and Jones, R. Wait times, patient satisfaction scores, and the perception of care. Am J Manag Care. (2014) 20:393–400.

PubMed Abstract | Google Scholar

2. Ward, PR, Rokkas, P, Cenko, C, Pulvirenti, M, Dean, N, Carney, AS, et al. ‘Waiting for’ and ‘waiting in’ public and private hospitals: a qualitative study of patient trust in South Australia. BMC Health Serv Res. (2017) 17:333. doi: 10.1186/s12913-017-2281-5

PubMed Abstract | Crossref Full Text | Google Scholar

3. Embrett, M, Sim, SM, Caldwell, HAT, Boulos, L, Yu, Z, Agarwal, G, et al. Barriers to and strategies to address COVID-19 testing hesitancy: a rapid scoping review. BMC Public Health. (2022) 22:750. doi: 10.1186/s12889-022-13127-7

PubMed Abstract | Crossref Full Text | Google Scholar

4. Goralnick, E, Kaufmann, C, and Gawande, AA. Mass-vaccination sites — an essential innovation to curb the Covid-19 pandemic. N Engl J Med. (2021) 384:e67. doi: 10.1056/NEJMp2102535

PubMed Abstract | Crossref Full Text | Google Scholar

5. Rosner, E, Lapin, T, and Garger, K. (2021). Hours-long waits reported at Javits center COVID vaccine site in NYC. New York Post. Available at: https://nypost.com/2021/03/02/hours-long-waits-reported-at-javits-center-covid-vaccine-site-in-nyc/ (Accessed March 2, 2021).

Google Scholar

6. CBS Baltimore. As interest in COVID-19 vaccine rises, Baltimoreans face long lines, wait times. New York, NY, CBS News: (2021).

Google Scholar

7. Di Pumpo, M, Ianni, A, Miccoli, GA, Di Mattia, A, Gualandi, R, Pascucci, D, et al. Queueing theory and COVID-19 prevention: model proposal to maximize safety and performance of vaccination sites. Front Public Health. (2022) 10:8406777. doi: 10.3389/fpubh.2022.840677

PubMed Abstract | Crossref Full Text | Google Scholar

8. Orsi, A, Butera, F, Piazza, MF, Schenone, S, Canepa, P, Caligiuri, P, et al. Analysis of a 3-months measles outbreak in western Liguria, Italy: are hospital safe and healthcare workers reliable? J Infect Public Health. (2020) 13:619–24. doi: 10.1016/j.jiph.2019.08.016

PubMed Abstract | Crossref Full Text | Google Scholar

9. Mo, Y, Eyre, DW, Lumley, SF, Walker, TM, Shaw, RH, O’Donnell, D, et al. Transmission of community- and hospital-acquired SARS-CoV-2 in hospital settings in the UK: a cohort study. PLoS Med. (2021) 18:e1003816. doi: 10.1371/journal.pmed.1003816

PubMed Abstract | Crossref Full Text | Google Scholar

10. Shortle, JF, Thompson, JM, Gross, D, and Harris, CM. Fundamentals of queueing theory. 5th ed. New York, NY: John Wiley & Sons (2018).

Google Scholar

11. Moreno-Carrillo, A, Arenas, LMÁ, Fonseca, JA, Caicedo, CA, Tovar, SV, and Muñoz-Velandia, OM. Application of queuing theory to optimize the triage process in a tertiary emergency care (“ER”) department. J Emerg Trauma Shock. (2019) 12:268–73. doi: 10.4103/JETS.JETS_42_19

PubMed Abstract | Crossref Full Text | Google Scholar

12. Tucker, JB, Barone, JE, Cecere, J, Blabey, RG, and Rha, CK. Using queueing theory to determine operating room staffing needs. J Trauma Acute Care Surg. (1999) 46:71–9. doi: 10.1097/00005373-199901000-00012

PubMed Abstract | Crossref Full Text | Google Scholar

13. Zonderland, ME, Boucherie, RJ, Litvak, N, and Vleggeert-Lankamp, CLAM. Planning and scheduling of semi-urgent surgeries. Health Care Manag Sci. (2010) 13:256–67. doi: 10.1007/s10729-010-9127-6

PubMed Abstract | Crossref Full Text | Google Scholar

14. de Bruin, AM, Bekker, R, van Zanten, L, and Koole, GM. Dimensioning hospital wards using the Erlang loss model. Ann Oper Res. (2010) 178:23–43. doi: 10.1007/s10479-009-0647-8

Crossref Full Text | Google Scholar

15. Cayirli, T, and Veral, E. Outpatient scheduling in health care: a review of literature. Prod Oper Manag. (2003) 12:519–49. doi: 10.1111/j.1937-5956.2003.tb00218.x

Crossref Full Text | Google Scholar

16. Lee, EK, Li, ZL, Liu, YK, and LeDuc, J. Strategies for vaccine prioritization and mass dispensing. Vaccine. (2021) 9:506. doi: 10.3390/vaccines9050506

PubMed Abstract | Crossref Full Text | Google Scholar

17. Kumar, A, Kumar, G, Ramane, TV, and Singh, G. Optimal Covid-19 vaccine stations location and allocation strategies. BIJ. (2022) 30:3328–56. doi: 10.1108/BIJ-02-2022-0089

PubMed Abstract | Crossref Full Text | Google Scholar

18. Hanly, M, Churches, T, Fitzgerald, O, Caterson, I, MacIntyre, CR, and Jorm, L. Modelling vaccination capacity at mass vaccination hubs and general practice clinics: a simulation study. BMC Health Serv Res. (2022) 22:1059. doi: 10.1186/s12913-022-08447-8

PubMed Abstract | Crossref Full Text | Google Scholar

19. Jahani, H, Chaleshtori, AE, Khaksar, SMS, Aghaie, A, and Sheu, JB. COVID-19 vaccine distribution planning using a congested queuing system—a real case from Australia. Transp Res Part E Logistics Transp Rev. (2022) 163:102749. doi: 10.1016/j.tre.2022.102749

PubMed Abstract | Crossref Full Text | Google Scholar

20. Hirbod, F, Eshghali, M, Sheikhasadi, M, Jolai, F, and Aghsami, A. A state-dependent M/M/1 queueing location-allocation model for vaccine distribution using metaheuristic algorithms. J Comput Design Eng. (2023) 10:1507–30. doi: 10.1093/jcde/qwad058

Crossref Full Text | Google Scholar

21. Lee, E. Modeling and optimizing the public-health infrastructure for emergency response. Interfaces. (2009) 39:476–90. doi: 10.1287/inte.1090.0463

PubMed Abstract | Crossref Full Text | Google Scholar

22. Lee, EK, Smalley, HK, Zhang, Y, Pietz, F, and Benecke, B. Facility location and multi-modality mass dispensing strategies and emergency response for biodefence and infectious disease outbreaks. IJRAM. (2009) 12:311. doi: 10.1504/IJRAM.2009.025925

PubMed Abstract | Crossref Full Text | Google Scholar

23. Lee, EK, Pietz, F, Benecke, B, Mason, J, and Burel, G. Advancing public health and medical preparedness with operations research. Interfaces. (2013) 43:79–98. doi: 10.1287/inte.2013.0676

PubMed Abstract | Crossref Full Text | Google Scholar

24. Blank, F. A spatial queuing model for the location decision of emergency medical vehicles for pandemic outbreaks: the case of Za’atari refugee camp. JHLSCM. (2021) 11:296–319. doi: 10.1108/JHLSCM-07-2020-0058

Crossref Full Text | Google Scholar

25. Castillo-Neyra, R, Toledo, AM, Arevalo-Nieto, C, MacDonald, H, De la Puente-León, M, Naquira-Velarde, C, et al. Socio-spatial heterogeneity in participation in mass dog rabies vaccination campaigns, Arequipa, Peru. PLoS Negl Trop Dis. (2019) 13:e0007600

Google Scholar

26. Castillo-Neyra, R, Brown, J, Borrini, K, Arevalo, C, Levy, MZ, Buttenheim, A, et al. Barriers to dog rabies vaccination during an urban rabies outbreak: qualitative findings from Arequipa, Peru. PLoS Negl Trop Dis. (2017) 11:e0005460

Google Scholar

27. Raynor, B, Díaz, EW, Shinnick, J, Zegarra, E, Monroy, Y, Mena, C, et al. The impact of the COVID-19 pandemic on rabies reemergence in Latin America: the case of Arequipa, Peru. PLoS Negl Trop Dis. (2021) 15:e0009414. doi: 10.1371/journal.pntd.0009414

PubMed Abstract | Crossref Full Text | Google Scholar

28. Castillo-Neyra, R, Xie, S, Bellotti, BR, Diaz, EW, Saxena, A, Toledo, AM, et al. Optimizing the location of vaccination sites to stop a zoonotic epidemic. Sci Rep. (2024) 14:15910. doi: 10.1038/s41598-024-66674-x

PubMed Abstract | Crossref Full Text | Google Scholar

29. Kulkarni, VG. Modeling and analysis of stochastic systems. 3rd ed. New York, NY: Chapman and Hall/CRC (2016).

Google Scholar

30. Teitz, MB, and Bart, P. Heuristic methods for estimating the generalized vertex median of a weighted graph. Oper Res. (1968) 16:955–61. doi: 10.1287/opre.16.5.955

PubMed Abstract | Crossref Full Text | Google Scholar

31. Anderson, RT, Camacho, FT, and Balkrishnan, R. Willing to wait?: the influence of patient wait time on satisfaction with primary care. BMC Health Serv Res. (2007) 7:31. doi: 10.1186/1472-6963-7-31

PubMed Abstract | Crossref Full Text | Google Scholar

32. Camacho, F, Anderson, R, Safrit, A, Jones, AS, and Hoffmann, P. The relationship between Patient’s perceived waiting time and office-based practice satisfaction. N C Med J. (2006) 67:409–13. doi: 10.18043/ncm.67.6.409

PubMed Abstract | Crossref Full Text | Google Scholar

33. Konstantinou, P, Georgiou, K, Kumar, N, Kyprianidou, M, Nicolaides, C, Karekla, M, et al. Transmission of vaccination attitudes and uptake based on social contagion theory: a scoping review. Vaccine. (2021) 9:607. doi: 10.3390/vaccines9060607

PubMed Abstract | Crossref Full Text | Google Scholar

34. Karashiali, C, Konstantinou, P, Christodoulou, A, Kyprianidou, M, Nicolaou, C, Karekla, M, et al. A qualitative study exploring the social contagion of attitudes and uptake of COVID-19 vaccinations. Hum Vaccin Immunother. (2023) 19:2260038. doi: 10.1080/21645515.2023.2260038

PubMed Abstract | Crossref Full Text | Google Scholar

35. Fu, F, Christakis, NA, and Fowler, JH. Dueling biological and social contagions. Sci Rep. (2017) 7:43634. doi: 10.1038/srep43634

PubMed Abstract | Crossref Full Text | Google Scholar

36. Alvarez-Zuzek, LG, Zipfel, CM, and Bansal, S. Spatial clustering in vaccination hesitancy: the role of social influence and social selection. PLoS Comput Biol. (2022) 18:e1010437. doi: 10.1371/journal.pcbi.1010437

PubMed Abstract | Crossref Full Text | Google Scholar

37. Buttenheim, AM, Paz-Soldan, V, Barbu, C, Skovira, C, Quintanilla Calderón, J, Mollesaca Riveros, LM, et al. Is participation contagious? Evidence from a household vector control campaign in urban Peru. J Epidemiol Community Health. (2014) 68:103–9. doi: 10.1136/jech-2013-202661

PubMed Abstract | Crossref Full Text | Google Scholar

38. World Health Organization. WHO expert consultation on rabies: Third Report. Geneva: World Health Organization (2018).

Google Scholar

39. Cleaveland, S. A dog rabies vaccination campaign in rural Africa: impact on the incidence of dog rabies and human dog-bite injuries. Vaccine. (2003) 21:1965–73. doi: 10.1016/S0264-410X(02)00778-8

PubMed Abstract | Crossref Full Text | Google Scholar

40. Vigilato, MAN, Clavijo, A, Knobl, T, Silva, HMT, Cosivi, O, Schneider, MC, et al. Progress towards eliminating canine rabies: policies and perspectives from Latin America and the Caribbean. Philos Trans R Soc B. (2013) 368:20120143. doi: 10.1098/rstb.2012.0143

PubMed Abstract | Crossref Full Text | Google Scholar

41. Liang, CC. Queueing management and improving customer experience: empirical evidence regarding enjoyable queues. J Consum Mark. (2016) 33:257–68. doi: 10.1108/JCM-07-2014-1073

Crossref Full Text | Google Scholar

42. Allon, G, and Kremer, M. Behavioral foundations of queueing systems In: K Donohue, E Katok, and S Leider, editors. The handbook of behavioral operations [internet]. Hoboken, NJ, USA: John Wiley & Sons, Inc. (2018). 323–66.

Google Scholar

43. Zhou, R, and Soman, D. Consumers’ waiting in queues: the role of first-order and second-order justice. Psychol Mark. (2008) 25:262–79. doi: 10.1002/mar.20208

Crossref Full Text | Google Scholar

44. Pan American Health Organization. República Dominicana: Elimination of dog-transmitted rabies in Latin America: Situation analysis. Washington, DC: OPS (2004).

Google Scholar

45. Sánchez-Soriano, C, Gibson, AD, Gamble, L, Burdon Bailey, JL, Green, S, Green, M, et al. Development of a high number, high coverage dog rabies vaccination programme in Sri Lanka. BMC Infect Dis. (2019) 19:977. doi: 10.1186/s12879-019-4585-z

PubMed Abstract | Crossref Full Text | Google Scholar

46. Gibson, AD, Handel, IG, Shervell, K, Roux, T, Mayer, D, Muyila, S, et al. The vaccination of 35,000 dogs in 20 working days using combined static point and door-to-door methods in Blantyre, Malawi. PLoS Negl Trop Dis. (2016) 10:e0004824

Google Scholar

47. le-Morawa, N, Kunkel, A, Darragh, J, Reede, D, Chidavaenzi, NZ, Lees, Y, et al. Effectiveness of a COVID-19 vaccine rollout in a highly affected American Indian community, San Carlos apache tribe, December 2020-February 2021. Public Health Rep. (2023) 138:23S–9S. doi: 10.1177/00333549221120238

PubMed Abstract | Crossref Full Text | Google Scholar

48. Sethy, G, Chisema, MN, Sharma, L, Singhal, S, Joshi, K, Nicks, PO, et al. ‘Vaccinate my village’ strategy in Malawi: an effort to boost COVID-19 vaccination. Expert Rev Vaccines. (2023) 22:180–5. doi: 10.1080/14760584.2023.2171398

PubMed Abstract | Crossref Full Text | Google Scholar

49. Whitehead, J, Carr, P, Scott, N, and Lawrenson, R. Structural disadvantage for priority populations: the spatial inequity of COVID-19 vaccination services in Aotearoa. N Z Med J. (2022) 135:54–67.

PubMed Abstract | Google Scholar

50. Hawkins, D. Disparities in access to paid sick leave during the first year of the COVID-19 pandemic. J Occup Environ Med. (2023) 65:370–7. doi: 10.1097/JOM.0000000000002784

PubMed Abstract | Crossref Full Text | Google Scholar

51. Schnake-Mahl, AS, O’Leary, G, Mullachery, PH, Skinner, A, Kolker, J, Diez Roux, AV, et al. Higher COVID-19 vaccination and narrower disparities in US cities with paid sick leave compared to those without. Health Aff. (2022) 41:1565–74. doi: 10.1377/hlthaff.2022.00779

PubMed Abstract | Crossref Full Text | Google Scholar

52. Sunar, N, Tu, Y, Ziya, S, and Pooled, VS. Dedicated queues when customers are delay-sensitive. Manag Sci. (2021) 67:3785–802. doi: 10.1287/mnsc.2020.3663

PubMed Abstract | Crossref Full Text | Google Scholar