Revisiting the complex time-varying effect of non-pharmaceutical interventions on COVID-19 transmission in the United States

Introduction: Although the global COVID-19 emergency ended, the real-world effects of multiple non-pharmaceutical interventions (NPIs) and the relative contribution of individual NPIs over time were poorly understood, limiting the mitigation of future potential epidemics.

Methods: Based on four large-scale datasets including epidemic parameters, virus variants, vaccines, and meteorological factors across 51 states in the United States from August 2020 to July 2022, we established a Bayesian hierarchical model with a spike-and-slab prior to assessing the time-varying effect of NPIs and vaccination on mitigating COVID-19 transmission and identifying important NPIs in the context of different variants pandemic.

Results: We found that (i) the empirical reduction in reproduction number attributable to integrated NPIs was 52.0% (95%CI: 44.4, 58.5%) by August and September 2020, whereas the reduction continuously decreased due to the relaxation of NPIs in following months; (ii) international travel restrictions, stay-at-home requirements, and restrictions on gathering size were important NPIs with the relative contribution higher than 12.5%; (iii) vaccination alone could not mitigate transmission when the fully vaccination coverage was less than 60%, but it could effectively synergize with NPIs; (iv) even with fully vaccination coverage >60%, combined use of NPIs and vaccination failed to reduce the reproduction number below 1 in many states by February 2022 because of elimination of above NPIs, following with a resurgence of COVID-19 after March 2022.

Conclusion: Our results suggest that NPIs and vaccination had a high synergy effect and eliminating NPIs should consider their relative effectiveness, vaccination coverage, and emerging variants.

1 Introduction

In the past 3 years, the value of non-pharmaceutical interventions (NPIs) in controlling infectious disease transmission has been greatly recognized in the context of the COVID-19 global pandemic. Implementing a series of containment NPIs, including school closing, workplace closing, canceling public events, gathering size restrictions, closing public transport, stay-at-home requirements, internal movement restrictions, and international travel restrictions (1), was one of the few tactics for such a new infectious disease. Many studies have reported the distinguished effects of NPIs on COVID-19 across the world (2–4), while long-term implementation of stringent NPIs also occasioned unintended issues in the economic recession (5), social problems (6, 7), and mental health outcomes (8). It has raised debates on whether and when an intervention could be lifted or eliminated (9, 10), which heavily depends on a comprehensive understanding of the effectiveness of NPIs based on real-world evidence.

Although we have trudged through the thorny forest with the end of the global health emergency of COVID-19 (11), there is still a risk of COVID-19 resurgence due to virus mutations. The risk of emerging infectious diseases has been increasing since the 21^st century, with several viral infectious diseases being pandemic within the last 20 years (12), which makes it more important to effectively implement NPIs to mitigate the epidemic. The practice of controlling COVID-19 provides an opportunity to clarify the effectiveness of NPIs, however, our understanding of the real-world effects of NPIs on mitigating COVID-19 transmission is not enough. The effectiveness of NPIs could change over time due to different compositions of variants of concern, socioeconomic features, vaccination, and policy compliance (13–16). Most previous studies focused on the early stage of the pandemic with the SARS-CoV-2 origin virus and the Alpha variant being dominant (3, 17, 18), however, evidence on the effectiveness in successive waves was relatively few, especially for the period of Omicron variant epidemic. Some modeling studies have discussed the varying effect of NPIs due to Omicron (19, 20), but the clues were not strong enough because the assumed population features and transmission parameters would be different from the real world.

Further, how vaccines worked with NPIs to eliminate the epidemic was also poorly understood. Several previous studies have explored the potential of lifting NPIs as the vaccination coverage increased to a pre-set level via mathematical models (20–23). Many of these modeling studies have provided constructive suggestions on delaying the relaxation of NPIs, but the reliability of simulation evidence would be challenged by the uncertainty of real-world vaccine efficacy and the emergence of new variants (24, 25). However, only one study has investigated the real-world combined effect of NPIs and vaccination and revealed a time-varying interaction effect between NPIs and vaccination (16).

Last but not least, multiple NPIs are always implemented and effective at the same time, but most previous studies failed to take this mechanism into consideration, which would limit the practical value in guiding policy decisions. Some studies were restricted to a specific combination of NPIs (e.g., lockdown) or regarded multiple NPIs as a whole by using a single index (13, 26, 27), which makes it hard to decide which kind of NPI could be modified. Another part of the study investigated the individual effect of a single NPI by constructing separate models for each NPI, ignoring the confounding of other NPIs (9, 15, 17). Some studies constructed a single model to simultaneously estimate the individual effect of multiple NPIs (18), but they did not consider the collinearity due to the high correlation among NPIs. Ignoring the collinearity issue can make results far away from the real effectiveness, leading to much uncertainty as to which NPIs are appropriate. Nevertheless, there is still no evidence untangling the complex effect of multiple highly correlated NPIs on COVID-19 transmission.

To provide more detailed evidence on the real-world effects of multiple NPIs across waves and pandemic variants, we revisited the complex time-varying role of NPIs in controlling COVID-19 transmission in the United States (United States) through a two-stage modeling framework. We first estimated bimonthly effects of NPIs, vaccination, and their interaction in forms of percentage reductions from time-specific basic reproduction number ( $R_{0,t})$ to effective reproduction number ( $R_t)$ at the state level through a Bayesian hierarchical model with a spike and slab prior. Then, we utilized meta-analysis to pool the state-specific results and obtain the overall impact of NPIs in the U.S. over time.

2 Materials and methods

2.1 Data sources

2.1.1 Outcome

To obtain a proper interpretation of the effectiveness of NPIs in mitigating COVID-19 transmission, we used the effective reproduction number ( $R_t$ ) to quantify the change in person-to-person transmission (27). Daily new diagnosed cases for the 51 states in the U.S. from 1 August 2020 to 31 July 2022 were obtained from The New York Times COVID-19 data repository, based on reports from state and local health agencies (28). To be consistent with Our World in Data and previous studies (16, 29), we estimated the daily state-specific $R_t$ using the Kalman filter, a Bayesian method modeling the growth rate of COVID-19 infections (30).

To consider the variation of basic reproduction numbers over time due to various compositions of variants of concern, we also estimated time-specific basic reproduction numbers ( $R_{0,t}$ ) to represent the basic transmission ability of the virus without any government response to the epidemic. First, we collected basic reproduction numbers for the original SARS-CoV-2 virus and the expansion factors for Alpha, Beta, Gamma, and Delta variants compared with the original virus from the literature (Supplementary Table S1). Due to a lack of evidence on the Omicron variant, we obtained its reported expansion parameter compared to the Delta variant. Then, the basic reproduction number for each variant was indirectly calculated by multiplying the expansion parameter with the basic reproduction number of the original virus or Delta variant (16, 31). Then, we collected the SARS-CoV-2 sequence data from the Global Initiative on Sharing All Influenza Data (GISAID) (32), and calculated the biweekly proportion of sequences of the above six variants between August 2020 and July 2022. Finally, we calculated $R_{0,t}$ by taking a weighted average of the basic reproduction numbers and proportions of the variants abovementioned.

2.1.2 Measure of NPIs

Consistent with previous studies, we used the stringency index of NPIs assembled by the Oxford COVID-19 Government Response Tracker (OxCGRT) to quantify the intensity of overall and individual NPIs implemented by the governments (1). It is a composite measure based on multiple response indicators reflecting more or less intense government responses to the epidemic. We mainly focused on containment policies, including school closing, workplace closing, canceling public events, gathering size restrictions, closing public transport, stay-at-home requirements, internal movement restrictions, and international travel restrictions. Each NPI had 3 to 4 levels of intensity and all of them were first normalized to a value between 0 and 100 by a flag variable with a higher score indicating a stricter response (1). We defined the stringency index as the arithmetic mean of these eight NPIs index, indicating the overall intensity of NPIs implemented by the government. For a better model fit, these indexes were divided by 100 to downscale to 0–1 in this study.

2.1.3 Other covariates

The fully vaccination coverages, i.e., proportions of people who completed the initial COVID-19 vaccination protocol, for the 51 states in the United States were collected from Our World in Data (33). Fully vaccination coverage was defined as the proportion of people who received all doses prescribed by the initial vaccination protocol. We did not consider the various efficacies of different types of vaccines because there was no accepted evidence on efficacies for different vaccines as well as their temporal variation. Using results from other populations and times would both introduce unknown confounding. We also considered temperature as a potential factor that would impact the relationship between NPIs and COVID-19 transmission (16). Thus, Hourly temperature gridded data (a resolution of 0.10 degrees) were obtained from the ERA5-Land dataset in the Climate Data Store maintained by the European Center for Medium-range Weather Forecasts (34). We calculated the daily mean temperature for each state by averaging data at all hours a day across all grids in the state and divided it by 10 to estimate the effect of temperature with per-10 degrees.

2.2 Statistical modeling

2.2.1 The main model

We used a two-stage modeling framework to assess the effect of NPIs and their relative contributions in the United States by pooling state-level effects via meta-analysis with the random effect model. For each state, we assumed the effects of NPIs and vaccination to be stable within 2 months and estimated them through a Bayesian hierarchical model. The outcome in states was the natural logarithm of the state-specific ratio of effective reproduction number ( $R_t^s$ ) to time-specific basic reproduction number ( $R_{0,t}^s$ ), namely $log\left(\frac{R_t^s}{R_{0,t}^s}\right)$ , where t represents the daytime in bimester b. We built a hierarchical linear model as follows,

$log\left(\frac{R_t^s}{R_{0,t}^s}\right)~\mathit{normal}\left({\mu }_t^s,0.5\right)\text{,}$

$\begin{array}{ll}{\mu }_t^s={\alpha }_b^s+{\displaystyle \sum }_{i=1}^8{\theta }_{i,b}^s{N}_{i,t}^s+{\beta }_{1,b}^s{V}_t^s+{\displaystyle \sum }_{i=1}^8{\gamma }_{i,b}^s{N}_{i,t}^s{V}_t^s+{\beta }_{2,b}^s{T}_t^s\text{,}& (1)\end{array}$

where $N_{i,t}^s$ is the intensity index of i^th NPI among the above eight containment in states at day t. $V_t^s$ and $T_t^s$ represent corresponding fully vaccination coverage and temperature, respectively. ${\theta }_{i,b}^s$ , ${\beta }_{1,b}^s$ , and ${\gamma }_{i,b}^s$ are the coefficients of individual NPI, vaccination and their interaction.

2.2.2 The shrinkage prior

The Bayesian hierarchical model with shrinkage priors is a compelling method to deal with high-dimensional and correlated structure of predictors (35). Many simulation and application studies have reported the robustness of the results in high-dimensional regressions by using shrinkage priors (36, 37). Depending on the number and forms of prior distribution settings for the coefficients, it could be roughly divided into discrete mixture shrinkage priors, such as the spike-and-slab prior, and global–local (GL) shrinkage priors, including the horseshoe prior and Dirichlet Laplace prior (37, 38). The mixture shrinkage priors are to set a discrete mixture of a normal distribution with high density around zero (the spike) and a normal distribution with a large shape parameter (38), whereas the global–local shrinkage priors are continuous mixture of normal densities, which used the global parameter controlling the overall shrinkage of all the coefficients toward zero and the local parameters modifying the coefficient-specific shrinkage (39).

The spike-and-slab prior is often considered as the “gold standard” for sparse Bayesian estimation (39), so that we utilized it to cope with the collinearity issue due to the high correlation among NPIs, which could shrink small coefficients toward zero and assign large coefficients to the slab (40). We set each ${\theta }_{i,b}^s$ or ${\gamma }_{i,b}^s$ with a spike-and-slab prior as follows. For the sake of brevity with no loss of generality, we removed the state superscripts and the prior distribution could be written as,

$\begin{array}{ll}{\vartheta }_{i,b}~{\delta }_{i,b}\mathit{normal}\left(0,{\sigma }_{i,b,\mathit{spike}}^2\right)+\left(1-{\delta }_{i,b}\right)\mathit{normal}\left(0,{\sigma }_{i,b,\mathit{slab}}^2\right)\text{,}& (2)\end{array}$

${\delta }_{i,b}~\mathit{beta}($1,1),

where ${\vartheta }_{i,b}$ is ${\theta }_{i,b}^s\mathit{or}{\gamma }_{i,b}^s$ in states which was constrained to be negative. ${\delta }_{i,b}$ is the inclusion probability of whether the coefficient equals zero with a noninformative prior of $\mathit{beta}($1,1) indicating that each coefficient has a 50% probability to be zero. ${\sigma }_{i,b,\mathit{spike}}^2$ and ${\sigma }_{i,b,\mathit{slab}}^2$ are the variances of normal distributions in which ${\sigma }_{i,b,\mathit{spike}}^2$ is much smaller than ${\sigma }_{i,b,\mathit{slab}}^2$ to make the probability density function closely around zero.

We put a half student-t prior with 3 degrees of freedom for these two hyperparameters to simultaneously ensure positivity and convergence (39), that is, ${\sigma }_{i,b,\mathit{spike}}^2~\mathit{half}\mathit{student}-t\left(3,0,0.05\right)$ and ${\sigma }_{i,b,\mathit{slab}}^2~\mathit{half}\mathit{student}-t\left(3,0,0.5\right)$ . We had a normal prior over the coefficients ${\beta }_{1/2,b}^s~\mathit{normal}$ (0,0.5) and a student-t prior with 3 degrees of freedom for the intercept ${\alpha }_b^s~\mathit{student}-t\left(3,0,1\right)$ .

2.2.3 Effectiveness and empirical effect of NPIs

To get the overall effect of integrated NPIs, we obtain the posterior distributions of the overall effect of eight integrated NPIs and their interaction with vaccination by summing the ${\theta }_{i,b}^s$ and ${\gamma }_{i,b}^s$ in each iteration, that is, ${\theta }_{\mathit{inte},b}^s={\displaystyle \sum }_{i=1}^8{\theta }_{i,b}^s$ and ${\gamma }_{\mathit{inte},b}^s={\displaystyle \sum }_{i=1}^8{\gamma }_{i,b}^s$ , respectively. Then, the relative effectiveness of NPIs and vaccination could be calculated by $1-exp\left({\theta }_{\mathit{inte},b}^s\right)$ and $1-exp\left({\beta }_{1,b}^s\right)$ . The estimated empirical effects taking the NPIs intensity and fully vaccination coverage into consideration are $1-exp\left({\theta }_{\mathit{inte},b}^{s}N\right)$ and $1-exp\left({\beta }_{1,b}^s\mathrm{V}\right)$ , respectively, which correspond to the realistic meaning of the reduction in $R_{0,t}$ regarding $R_t$ , i.e., $1-\frac{R_t}{R_{0,t}}$ , so as to the interaction between NPIs and vaccination. The relative contribution of i^th NPI was given as ${\theta }_{i,b}^s/{\displaystyle \sum }_{i=1}^8{\theta }_{i,b}^s$ (41), indicating the importance weight for each NPI in the overall effect of integrated NPIs. We estimated the relative effectiveness and empirical effect of integrated NPIs and the relative contribution of each NPI for every bimester (two-month) to account for seasonality and the impact of changing composition of dominant variant of concern.

To assess the model robustness, we modified the hyper-parameters in the spike-and-slab prior. Specifically, we changed the shape parameters in the half-student-t distribution to 0.3 and 0.03 for inclusion and exclusion, respectively. And we also alter the prior for ${\delta }_{i,b}$ from $\mathit{beta}($1,1) to $\mathit{beta}($3,5) with the assumption that only three of eight NPIs worked in a bimester. We also changed the linear model to a generalized linear model assuming the outcome of $R_t$ following gamma distribution which was conducted in previous research (16). Finally, we used the horseshoe prior, one of the most commonly used global–local shrinkage priors, with a $\mathit{half}-\mathit{student}-t\left(3,0,0.5\right)$ for the global parameter, to evaluate the influence of various shrinkage priors.

All the analyses were conducted in the R software (version 4.3.2). The Bayesian hierarchical model was performed through Markov Chain Monte Carlo (MCMC) methods by calling the Stan software via the ‘cmdstanr’ package (42). We ran four parallel chains for 2,000 iterations with the first 1,000 iterations of warmup to obtain 4,000 posterior samples for each state and bimester. All plots were drawn using the ‘ggplot2’ package.

3 Results

During the study period, the variants of Alpha, Delta, and Omicron successively became dominant in April 2020, July 2021, and January 2022, respectively (Figure 1A). During August 2020 and February 2021, governments in the U.S. implemented NPIs with the stringency index being around 55. Since March 2021, governments continuously relaxed the intensity of NPIs and went to an “open” style in April 2022. Most of the pairwise correlations between individual NPIs were higher than 0.6, indicating a potential of collinearity should be considered (Supplementary Figure S2). The U.S. government commenced the mass vaccination campaigns at the end of December 2020. The fully vaccination coverage quickly climbed to 50% in August 2021 and slowly increased to 67.6% on 31 July 2022 (Figure 1B).

Figure 1

Figure 1. The overall temporal distribution of the data in the United States from 1 August 2020 to 31 July 2022. (A) Daily diagnosed cases (black bar), pandemic variant (colored background, defined as variants of concern that had the highest proportion), and effective reproduction number (blue lines in the bottom). (B) The stringency index (blue lines) and documented fully vaccination coverage (red lines) across 51 states (thick lines for the whole country).

3.1 Estimated empirical effects of integrated NPIs

In August and September 2020, the estimated empirical reduction in $R_{0,t}$ attributable to integrated NPIs was 52.0% (95%CI: 44.4, 58.5%). The relative effect of integrated NPIs stabilized at around 50.2% ~ 53.7% until the intensity of NPIs decreased in April and May 2021 (Figure 2A). Thereafter, the empirical effect of integrated NPIs decreased to 8.8% ~ 12.2% during December 2021 and July 2022 no matter whether the fully vaccination coverage was continuously increasing. In the early stage of mass vaccination campaigns (February and March 2021), there was a negative effect of vaccination on reducing COVID-19 transmission, with reductions in $R_{0,t}$ of −10.1% (95%CI: −15.8, −4.8%), while its effect climbed to 91.5% (95%CI: 85.2, 95.1%) by January 2022, with the fully vaccination coverage reaching 60% in the U.S (Figure 2A). Due to the decreased effect of vaccination in the context of the Omicron pandemic, the overall empirical effect of integrated NPIs was not high enough to reach the target $R_{0,t}$ reductions in that period (Supplementary Figure S3).

Figure 2

Figure 2. The effects of NPIs, vaccination, and their interaction on reducing COVID-19 transmission in the United States over time. (A) The empirical reductions in $R_{0,t}$ for integrated NPIs, vaccination, and their combined effects (integrated eight NPIs + vaccination + interaction). (B) The reductions in $R_{0,t}$ for integrated NPIs when all NPIs set at the highest level. We can compare the absolute effects of NPIs across periods of different pandemic variants. (C) The spatial distribution of integrated effects of NPIs during December 2021 and January in 2022 where the daily diagnosed cases reached the peak due to the Omicron variant. (D) The (empirical) effects of vaccination and its interaction with NPIs.

3.2 Effect variations of integrated NPIs due to different pandemic variants of concern

In the original SARS-CoV-2 virus period, the effect of integrated NPIs was stable at about 74.5% ~ 78.1%. We found that the emergence of a new dominant variant of concern would decrease the effect of integrated NPIs. For instance, the effect decreased to 67.8% (95%CI: 56.8, 75.9%) in June–July 2021 as the Alpha variant became the dominant variant of concerns. And the effect decreased to 59.4% (95%CI: 45.6, 69.7%) in December 2021 and January 2022 with Omicron becoming dominant (Figure 2B). we observed that Louisiana, Kansas, and South Carolina had the lowest effects of integrated NPIs on $R_{0,t}$ reductions (Figure 2C).

3.3 Vaccination and its interaction with integrated NPIs

When the fully vaccination coverage was less than 60%, per-10% increase in fully vaccination coverage could merely reduce the $R_{0,t}$ . The effectiveness of vaccination increased to 33.4% (95%CI: 27.1, 39.2%) by January 2022 with the fully vaccination coverage exceeding 60%, but it reduced to 12.8% ~ 13.2% with the Omicron being dominant after February 2022 (Figure 2D). The interaction effect between vaccination and integrated NPIs was higher than 75% from August to November 2020. Although it decreased to around 40% in 2021, it was much higher than the main effect of vaccination (Figure 2D).

Taking the practical vaccination coverage into consideration, we found that the empirical effect of vaccination in the early stage of mass vaccination campaigns was also small, but the combined effects of integrated NPIs and vaccination in that stage were higher than their separate effects. From February 2021 to March 2022, syntheses of NPIs and vaccination were always enough to reach the target $R_{0,t}$ reductions (reducing the $R_{0,t}$ below 1) even the intensity of NPIs decreased. However, we saw the combined effect from April to July 2022 was not high enough, followed with a resurgence of COVID-19 after March 2022 (Figures 1A, 2A).

3.4 Relative contributions of individual NPIs

The relative contributions of individual NPIs over time are shown in Figure 3. In almost time, international travel restrictions, stay-at-home requirements, and restrictions on gathering size contributed the most to the overall effects of integrated NPIs, with all corresponding relative contributions higher than 12.5%. However, the relative effect of international travel restrictions between October 2021 and January 2022 was low, with the Delta variant being dominant. The relative contribution of school closing was higher than 12.5% before the Delta and Omicron variants being pandemic, but it decreased to 8.9% ~ 11.1% after July 2021.

Figure 3

Figure 3. The bimonthly relative contribution of individual NPIs to the integrated effects.

The average intensity of individual NPIs continuously decreased below 20 after March 2022, but daily diagnosed cases started to increase in that period (Supplementary Figure S1) and implementation of NPIs synthesis was unable to reach the target $R_{0,t}$ reductions in the following months (Supplementary Figure S3). We found that the relative contributions of school closing in Nebraska and Idaho, public events cancelation in Hawaii, restrictions on gathering size in California and Hawaii, as well as closed public events in Nebraska and Wisconsin were below 5% (Figure 4). However, the relative contributions of internal movement restrictions in Alaska, Florida, Hawaii, Louisiana, Nebraska, Oklahoma, and Wisconsin, stay-at-home requirements in Alaska, Hawaii, South Carolina, and Wisconsin, close public transport in Alabama and Hawaii, restrictions on gathering size in South Carolina and Wisconsin, public events cancelation in New Jersey, South Carolina, and Wisconsin, and workplace closing in Nebraska were higher than 15%.

Figure 4

Figure 4. Selected NPIs with the lowest and highest relative contributions across 51 states in the United States during February and March 2022. Each bubble represents the relative contribution of one specific NPI in a state, with the size being proportional to the strength of NPIs. We considered that those NPIs with relative contributions higher than 15% should be maintained or strengthened, especially for internal movement restrictions, stay-at-home requirements, public transport closure, restrictions on gathering size, and public events cancelation in some states (red small bubbles).

3.5 Sensitivity analysis

The distribution of R-hat statistics for all coefficients indicated that the main model in this study properly converged (Supplementary Figure S4). To assess the robustness of the model, sensitivity analyses were conducted by altering the hyper-parameters for the inclusion probability and variance of individual NPI coefficients. The results showed that the overall trends of the estimated effects of integrated NPIs, vaccination, and their interaction were consistent in all scenarios (Supplementary Figures S5–S7). The estimated empirical effects of NPIs and vaccination with the horseshoe prior were basically the same as the main results with the spike-and-slab prior in most bimesters, whereas the effect of NPIs in 2020 was lower when we used the horseshoe prior (Supplementary Figure S8).

4 Discussion

In this study, we constructed a Bayesian hierarchical model to estimate the real-world effects of integrated NPIs, vaccination, and the relative impacts of individual NPIs on reducing COVID-19 transmission among populations in the U.S. across multiple waves. We found that the effect of integrated NPIs alone continuously decreased since April 2021, along with the lifting of NPIs. The main effect of vaccination was low when the fully vaccination coverage below 60%, whereas it could still synergize with NPIs to effectively control the epidemic. International travel restrictions, stay-at-home requirements, and restrictions on gathering size were identified as the most important individual NPIs, and the elimination of these NPIs in many states would be responsible for the resurgence of the epidemic after April 2022. The proposed method in this study could provide a good tool to obtain more detailed clues to formulate cost-effectively comprehensive NPIs strategies in the future potential epidemic.

We found that the effectiveness of integrated NPIs was weakened in the short term when the Alpha and Omicron variants were dominant in the United States. There was no previous evidence on the real-world effectiveness variation of NPIs due to the emergence of new dominant variants of concern. This short-term decline in NPIs efficiency would be related to a higher transmission probability and lower policy adherence (31, 43, 44). The largest decline was observed in December 2021 and January 2022 when the daily diagnosed cases exceeded 0.6 million people (Figure 1A), closely following the epidemic caused by the Delta variant. Long-term implementation of NPIs and successive waves caused pandemic fatigue and reduced people’s adherence to policies (44, 45). More importantly, the results showed that the impact of NPIs was more correlated with the intensity of NPIs, with a decline in empirical effect along with the relaxation of NPIs (Figure 2A), which was consistent with previous estimation in Europe (16). And we observed a rebound in the COVID-19 epidemic after April 2022 with the intensity of NPIs decreasing to a very low level. It suggested that we should appropriately increase the intensity of NPIs when a new variant of concern becomes dominant.

Consistent with previous studies, we found vaccination alone could not mitigate the epidemic, especially in the early stage of mass vaccination campaigns. Many modeling studies have emphasized the necessity of maintaining the intensity of NPIs when the vaccination coverage is low (10, 16, 20). A more important finding in this study was that mass vaccination campaigns in the early stage could even increase the risk of transmission. Theoretically, mass vaccination campaigns could directly reduce individuals’ infection risk for vaccinated people and indirectly protect unvaccinated people by constructing herd immunity. However, the real-world efficacies of Pfizer-BioNTech and Moderna were lower than the results from clinical trials (46), and mass vaccination itself also increased the chance of contact and transmission due to large gatherings (47). Further, there is a threshold for fully vaccination coverage to achieve herd immunity, and it was estimated at least 60% of people in the U.S. needed to be fully vaccinated to obtain vaccine-derived herd immunity, which was consistent with the results in this study (48). Lack of herd immunity and large-scale gatherings were responsible for the increased risk of transmission for vaccination alone in the early period of mass vaccination campaigns, but fortunately, our results indicated that vaccination could synergize with NPIs and more attention should be paid to maintaining NPIs in that period.

Great heterogeneity of the effect rank of NPIs existed in previous studies (2). Ge et al. found that gathering restrictions played significant roles in controlling transmission across waves (15), but Liu et al. reported school closing as the most important NPI (18). The pairwise correlations of NPIs in this study indicated that a collinearity-related bias would exist in previous studies. There were very few studies considering this issue (49). In this study, we used a shrinkage prior in the Bayesian hierarchical model to solve this problem and estimated the relative contributions of individual NPIs. We found that international travel restrictions, stay-at-home requirements, and restrictions on gathering size were the most important NPIs in the U.S., and the relative contributions of individual NPIs would change over time.

According to the relative contributions of individual NPIs, we found that international travel restrictions, stay-at-home requirements, restrictions on gathering size, and cancelation of public events in some states should be strengthened after March 2022. There had long been calls to lift or eliminate NPIs because of the high socioeconomic costs and health trade-offs of NPIs, especially for school closing (50–52). We found school closing did not have an obvious impact on reductions in $R_{0,t}$ , which was lifted in almost states across the country. But more importantly, there was a lack of counterbalance of NPIs to reduce the $R_{0,t}$ below 1, following a resurgence of COVID-19. NPIs with high relative contributions, such as international travel restrictions, stay-at-home requirements, restrictions on gathering size, and cancelation of public events, might be eliminated too soon, especially in the states of Alaska, Hawaii, and Wisconsin. The results of this study suggested that mid-2022 was not the time to fully liberalize NPIs, and different states must lift individual NPIs according to their own conditions and should have corresponding hedging measures in other important NPIs.

By using a Bayesian hierarchical model with a shrinkage prior for highly correlated NPIs, this study provided elaborate evidence on the real-world effects of integrated and individual NPIs on mitigating COVID-19 transmission across time and populations. Some limitations in our analysis should be mentioned. First, due to lack of data availability, we used the biweekly reported proportions of sequences of main variants of concern to represent their actual proportions in the population. It introduced uncertainty in the effect estimation of NPIs and vaccination which should be interpreted with caution. Further, it was reported that demographics and comorbidities could also have great impact on the transmission of COVID-19 (53, 54), but there was not daily or monthly information on the state-level demographical status and comorbidity rate. More individual-level data were needed to investigate the role of these factors in COVID-19 transmission. Second, we were unable to consider the interactions among the eight NPIs in this study because of the unidentifiable interactions in such a sparse NPIs data which would also change over time. There was also no evidence on the interaction effects among NPIs, which should be further studied in the future. Third, we observed the threshold of fully vaccination coverage for herd immunity would be 60%, but more targeted analyses are needed to properly identify the threshold with consideration of variation in vaccine effectiveness across time.

In summary, we found that the overall impact of integrated NPIs on mitigating the COVID-19 pandemic was influenced by the new dominant variants of concern, especially for Omicron. The high correlation between intensity of NPIs and interaction with vaccination required us to simultaneously implement NPIs and vaccination when the fully vaccination coverage was low. School closing in some states could be lifted when the Omicron variant was dominant, but we should not eliminate internal movement restrictions and stay-at-home requirements in Alaska, Hawaii, and Wisconsin too soon to prevent the rebound of COVID-19.

Data availability statement

Publicly available datasets were analyzed in this study. This data can be found at: https://github.com/DRGonghuaWu/USANPIspaper.

Author contributions

GW: Data curation, Formal analysis, Investigation, Methodology, Writing – original draft, Writing – review & editing. WFZ: Formal analysis, Resources, Validation, Writing – review & editing. WW: Methodology, Validation, Visualization, Writing – review & editing. PW: Writing – review & editing. ZH: Writing – review & editing. YW: Writing – review & editing. JL: Writing – review & editing. WJZ: Conceptualization, Supervision, Validation, Writing – review & editing. ZD: Conceptualization, Funding acquisition, Resources, Supervision, Writing – review & editing. YH: Conceptualization, Funding acquisition, Resources, Supervision, Writing – review & editing.

Funding

The author(s) declare financial support was received for the research, authorship, and/or publication of this article. This study was funded by the Science and Technology Program of Guangzhou, China (202206080003) and National Natural Science Foundation of China (Nos. 82103947 and 81973150).

Acknowledgments

YH gratefully acknowledges the support of K. C. Wong Education Foundation.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpubh.2024.1343950/full#supplementary-material

References

1. Hale, T, Angrist, N, Goldszmidt, R, Kira, B, Petherick, A, Phillips, T, et al. A global panel database of pandemic policies (Oxford COVID-19 government response tracker). Nat Hum Behav. (2021) 5:529–38. doi: 10.1038/s41562-021-01079-8

PubMed Abstract | Crossref Full Text | Google Scholar

2. Mendez-Brito, A, El Bcheraoui, C, and Pozo-Martin, F. Systematic review of empirical studies comparing the effectiveness of non-pharmaceutical interventions against COVID-19. J Infect. (2021) 83:281–93. doi: 10.1016/j.jinf.2021.06.018

PubMed Abstract | Crossref Full Text | Google Scholar

3. Brauner, JM, Mindermann, S, Sharma, M, Johnston, D, Salvatier, J, Gavenciak, T, et al. Inferring the effectiveness of government interventions against COVID-19. Science. (2021) 371:9338. doi: 10.1126/science.abd9338

PubMed Abstract | Crossref Full Text | Google Scholar

4. Perra, N. Non-pharmaceutical interventions during the COVID-19 pandemic: a review. Phys Rep. (2021) 913:1–52. doi: 10.1016/j.physrep.2021.02.001

PubMed Abstract | Crossref Full Text | Google Scholar

5. Fezzi, C, and Fanghella, V. Tracking GDP in real-time using electricity market data: insights from the first wave of COVID-19 across Europe. Eur Econ Rev. (2021) 139:103907. doi: 10.1016/j.euroecorev.2021.103907

PubMed Abstract | Crossref Full Text | Google Scholar

6. Armitage, R, and Nellums, LB. COVID-19 and the consequences of isolating the elderly. The lancet. Public Health. (2020) 5:61. doi: 10.1016/s2468-2667(20)30061-x

PubMed Abstract | Crossref Full Text | Google Scholar

7. Campbell, AM. An increasing risk of family violence during the Covid-19 pandemic: strengthening community collaborations to save lives. Forensic Sci. Int. (2020) 2:100089. doi: 10.1016/j.fsir.2020.100089

Crossref Full Text | Google Scholar

8. Pfefferbaum, B, and North, CS. Mental health and the Covid-19 pandemic. N Engl J Med. (2020) 383:510–2. eng. doi: 10.1056/NEJMp2008017

Crossref Full Text | Google Scholar

9. Li, Y, Campbell, H, Kulkarni, D, Harpur, A, Nundy, M, Wang, X, et al. The temporal association of introducing and lifting non-pharmaceutical interventions with the time-varying reproduction number (R) of SARS-CoV-2: a modelling study across 131 countries. Lancet Infect Dis. (2021) 21:193–202. doi: 10.1016/s1473-3099(20)30785-4

PubMed Abstract | Crossref Full Text | Google Scholar

10. Yang, J, Marziano, V, Deng, X, Guzzetta, G, Zhang, J, Trentini, F, et al. Despite vaccination, China needs non-pharmaceutical interventions to prevent widespread outbreaks of COVID-19 in 2021. Nat Hum Behav. (2021) 5:1009–20. doi: 10.1038/s41562-021-01155-z

PubMed Abstract | Crossref Full Text | Google Scholar

11. WHO. WHO director-General’s opening remarks at the media briefing. Geneva: WHO (2023).

Google Scholar

12. Baker, RE, Mahmud, AS, Miller, IF, Rajeev, M, Rasambainarivo, F, Rice, BL, et al. Infectious disease in an era of global change. Nat Rev Microbiol. (2021) 20:193–205. doi: 10.1038/s41579-021-00639-z

PubMed Abstract | Crossref Full Text | Google Scholar

13. Singh, S, Shaikh, M, Hauck, K, and Miraldo, M. Impacts of introducing and lifting nonpharmaceutical interventions on COVID-19 daily growth rate and compliance in the United States. Proc Natl Acad Sci. (2021) 118:118. doi: 10.1073/pnas.2021359118

Crossref Full Text | Google Scholar

14. Paireau, J, Charpignon, M-L, Larrieu, S, Calba, C, Hozé, N, Boëlle, P-Y, et al. Impact of non-pharmaceutical interventions, weather, vaccination, and variants on COVID-19 transmission across departments in France. BMC Infect Dis. (2023) 23:190. doi: 10.1186/s12879-023-08106-1

PubMed Abstract | Crossref Full Text | Google Scholar

15. Ge, Y, Zhang, W-B, Liu, H, Ruktanonchai, CW, Hu, M, Wu, X, et al. Impacts of worldwide individual non-pharmaceutical interventions on COVID-19 transmission across waves and space. Int J Appl Earth Obs Geoinf. (2022) 106:102649. doi: 10.1016/j.jag.2021.102649

PubMed Abstract | Crossref Full Text | Google Scholar

16. Ge, Y, Zhang, WB, Wu, X, Ruktanonchai, CW, Liu, H, Wang, J, et al. Untangling the changing impact of non-pharmaceutical interventions and vaccination on European COVID-19 trajectories. Nat Commun. (2022) 13:3106. doi: 10.1038/s41467-022-30897-1 PMID 35661759

PubMed Abstract | Crossref Full Text | Google Scholar

17. Wong, MC, Huang, J, Teoh, J, and Wong, SH. Evaluation on different non-pharmaceutical interventions during COVID-19 pandemic: an analysis of 139 countries. J Infect. (2020) 81:e70–1. doi: 10.1016/j.jinf.2020.06.044 PMID 32579981

PubMed Abstract | Crossref Full Text | Google Scholar

18. Liu, Y, Morgenstern, C, Kelly, J, Lowe, R, and Jit, MGroup CC-W. The impact of non-pharmaceutical interventions on SARS-CoV-2 transmission across 130 countries and territories. BMC Med. (2021) 19:40. doi: 10.1186/s12916-020-01872-8 PMID 33541353

PubMed Abstract | Crossref Full Text | Google Scholar

19. Le Rutte, EA, Shattock, AJ, Chitnis, N, Kelly, SL, and Penny, MA. Modelling the impact of omicron and emerging variants on SARS-CoV-2 transmission and public health burden. Commun. Med. (Lond). (2022) 2:93. doi: 10.1038/s43856-022-00154-z PMID 35899148

PubMed Abstract | Crossref Full Text | Google Scholar

20. Wang, B-G, Wang, Z-C, Wu, Y, Xiong, Y, Zhang, J, and Ma, Z. A mathematical model reveals the influence of NPIs and vaccination on SARS-CoV-2 omicron variant. Nonlinear Dyn. (2022) 111:3937–52. doi: 10.1007/s11071-022-07985-4

PubMed Abstract | Crossref Full Text | Google Scholar

21. Moore, S, Hill, EM, Tildesley, MJ, Dyson, L, and Keeling, MJ. Vaccination and non-pharmaceutical interventions for COVID-19: a mathematical modelling study. Lancet Infect Dis. (2021) 21:793–802. doi: 10.1016/S1473-3099(21)00143-2 PMID 33743847

PubMed Abstract | Crossref Full Text | Google Scholar

22. Giordano, G, Colaneri, M, Filippo, AD, Blanchini, F, Bolzern, P, Dicolao, GD, et al. Modeling vaccination rollouts, SARS-CoV-2 variants and the requirement for non-pharmaceutical interventions in Italy. Nat Med. (2021) 27:993–8. doi: 10.1038/s41591-021-01334-5

PubMed Abstract | Crossref Full Text | Google Scholar

23. Sonabend, R, Whittles, LK, Imai, N, Perez-Guzman, PN, Knock, ES, Rawson, T, et al. Non-pharmaceutical interventions, vaccination, and the SARS-CoV-2 delta variant in England: a mathematical modelling study. Lancet. (2021) 398:1825–35. doi: 10.1016/S0140-6736(21)02276-5

Crossref Full Text | Google Scholar

24. Tartof, SY, Slezak, JM, Fischer, H, Hong, V, Ackerson, BK, Ranasinghe, ON, et al. Effectiveness of mRNA BNT162b2 COVID-19 vaccine up to 6 months in a large integrated health system in the USA: a retrospective cohort study. Lancet. (2021) 398:1407–16. doi: 10.1016/S0140-6736(21)02183-8 PMID 34619098

PubMed Abstract | Crossref Full Text | Google Scholar

25. Afzal, A, Saleel, CA, Bhattacharyya, S, Satish, N, Samuel, OD, and Badruddin, IA. Merits and limitations of mathematical modeling and computational simulations in mitigation of COVID-19 pandemic: a comprehensive review. Arch Comput. Methods Eng. (2022) 29:1311–37. doi: 10.1007/s11831-021-09634-2

PubMed Abstract | Crossref Full Text | Google Scholar

26. Banholzer, N, Lison, A, Ozcelik, D, Stadler, T, Feuerriegel, S, and Vach, W. The methodologies to assess the effectiveness of non-pharmaceutical interventions during COVID-19: a systematic review. Eur J Epidemiol. (2022) 37:1003–1024. doi: 10.1007/s10654-022-00908-y

PubMed Abstract | Crossref Full Text | Google Scholar

27. Lison, A, Banholzer, N, Sharma, M, Mindermann, S, Unwin, HJT, Mishra, S, et al. Effectiveness assessment of non-pharmaceutical interventions: lessons learned from the COVID-19 pandemic. Lancet Public Health. (2023) 8:e311–7. doi: 10.1016/S2468-2667(23)00046-4

PubMed Abstract | Crossref Full Text | Google Scholar

28. New York Times. Coronavirus (Covid-19) data in the United States Github. The New York Times (2021). Available at: https://github.com/nytimes/covid-19-data. (Accessed Dec 2, 2022).

Google Scholar

29. Mathieu, E, Ritchie, H, Rodés-Guirao, L, Appel, C, Giattino, C, Hasell, J, et al. Coronavirus Pandemic (COVID-19): Our world in data. (2020). Available at: https://ourworldindata.org/coronavirus. (Accessed Dec 2, 2022).

Google Scholar

30. Sartorius, B, Arroyo-Marioli, F, Bullano, F, Kucinskas, S, and Rondón-Moreno, C. Tracking R of COVID-19: a new real-time estimation using the Kalman filter. PLoS One. (2021) 16:e0244474. doi: 10.1371/journal.pone.0244474

PubMed Abstract | Crossref Full Text | Google Scholar

31. Liu, Y, and Rocklov, J. The effective reproductive number of the omicron variant of SARS-CoV-2 is several times relative to Delta. J Travel Med. (2022) 29:37. doi: 10.1093/jtm/taac037

PubMed Abstract | Crossref Full Text | Google Scholar

32. Elbe, S, and Buckland-Merrett, G. Data, disease and diplomacy: GISAID's innovative contribution to global health. Global Chall. (2017) 1:33–46. doi: 10.1002/gch2.1018

PubMed Abstract | Crossref Full Text | Google Scholar

33. Mathieu, E, Ritchie, H, Ortiz-Ospina, E, Roser, M, Hasell, J, Appel, C, et al. A global database of COVID-19 vaccinations. Nat Hum Behav. (2021) 5:947–53. doi: 10.1038/s41562-021-01122-8

PubMed Abstract | Crossref Full Text | Google Scholar

34. Muñoz, SJ. ERA5-land hourly data from 1950 to present Copernicus climate change service (C3S) climate data store (CDS). (2019).

Google Scholar

35. Yi, N, and Ma, S. Hierarchical shrinkage priors and model fitting for high-dimensional generalized linear models. Stat Appl Genet Mol Biol. (2012) 11:803. doi: 10.1515/1544-6115.1803

PubMed Abstract | Crossref Full Text | Google Scholar

36. Bhattacharyya, A., Mitra, R., and Rai, S.. Count data models and Bayesian shrinkage priors with real-world data applications. (2022).

Google Scholar

37. Bhattacharyya, A, Pal, S, Mitra, R, and Rai, S. Applications of Bayesian shrinkage prior models in clinical research with categorical responses. BMC Med Res Methodol. (2022) 22:126. doi: 10.1186/s12874-022-01560-6 PMID 35484507

PubMed Abstract | Crossref Full Text | Google Scholar

38. van Erp, S, Oberski, DL, and Mulder, J. Shrinkage priors for Bayesian penalized regression. J Math Psychol. (2019) 89:31–50. doi: 10.1016/j.jmp.2018.12.004

Crossref Full Text | Google Scholar

39. Piironen, J, and Vehtari, A. Sparsity information and regularization in the horseshoe and other shrinkage priors. arXiv. (2017) 2017:694. doi: 10.48550/arXiv.1707.01694

Crossref Full Text | Google Scholar

40. Antonelli, J, Wilson, AA-O, and Coull, BA. Multiple exposure distributed lag models with variable selection. Biostatitics. (2022) 25:kxac038. doi: 10.1093/biostatistics/kxac038

PubMed Abstract | Crossref Full Text | Google Scholar

41. Keil, AP, Buckley, JP, O’Brien, KM, Ferguson, KK, Zhao, S, and White, AJ. A quantile-based g-computation approach to addressing the effects of exposure mixtures. Environ Health Perspect. (2020) 128:47004. doi: 10.1289/ehp5838

PubMed Abstract | Crossref Full Text | Google Scholar

42. Team SD. Stan modeling language users guide and reference manual. (2023). Available at: https://mc-stan.org.

Google Scholar

43. Liu, Y, and Rocklov, J. The reproductive number of the Delta variant of SARS-CoV-2 is far higher compared to the ancestral SARS-CoV-2 virus. J Travel Med. (2021) 28:124. doi: 10.1093/jtm/taab124

PubMed Abstract | Crossref Full Text | Google Scholar

44. Petherick, A, Goldszmidt, R, Andrade, EB, Furst, R, Hale, T, Pott, A, et al. A worldwide assessment of changes in adherence to COVID-19 protective behaviours and hypothesized pandemic fatigue. Nat Hum Behav. (2021) 5:1145–60. doi: 10.1038/s41562-021-01181-x

PubMed Abstract | Crossref Full Text | Google Scholar

45. N'Konzi, JN, Chukwu, CW, and Nyabadza, F. Effect of time-varying adherence to non-pharmaceutical interventions on the occurrence of multiple epidemic waves: a modeling study. Front Public Health. (2022) 10:1087683. doi: 10.3389/fpubh.2022.1087683

PubMed Abstract | Crossref Full Text | Google Scholar

46. Zheng, C, Shao, W, Chen, X, Zhang, B, Wang, G, and Zhang, W. Real-world effectiveness of COVID-19 vaccines: a literature review and meta-analysis. Int J Infect Dis. (2022) 114:252–60. doi: 10.1016/j.ijid.2021.11.009 PMID 34800687

PubMed Abstract | Crossref Full Text | Google Scholar

47. WHO. Framework for decision-making: implementation of mass vaccination campaigns in the context of COVID-19. WHO (2021). Available at: https://www.who.int/docs/default-source/coronaviruse/framework-for-decision-making-implementation-of-mass-vaccination-campaigns-in-the-context-of-covid19-slide-deck.pdf?sfvrsn=438dcccS.

Google Scholar

48. Gumel, AB, Iboi, EA, Ngonghala, CN, and Ngwa, GA. Toward achieving a vaccine-derived herd immunity threshold for COVID-19 in the U.S. front. Public Health. (2021) 9:709369. doi: 10.3389/fpubh.2021.709369 PMID 34368071

PubMed Abstract | Crossref Full Text | Google Scholar

49. Haug, N, Geyrhofer, L, Londei, A, Dervic, E, Desvars-Larrive, A, Loreto, V, et al. Ranking the effectiveness of worldwide COVID-19 government interventions. Nat Hum Behav. (2020) 4:1303–12. doi: 10.1038/s41562-020-01009-0

PubMed Abstract | Crossref Full Text | Google Scholar

50. Panchal, U, Salazar de Pablo, G, Franco, M, Moreno, C, Parellada, M, Arango, C, et al. The impact of COVID-19 lockdown on child and adolescent mental health: systematic review. Eur Child Adolesc Psychiatry. (2021) 32:1151–77. doi: 10.1007/s00787-021-01856-w

PubMed Abstract | Crossref Full Text | Google Scholar

51. Bayham, J, and Fenichel, EP. Impact of school closures for COVID-19 on the US health-care workforce and net mortality: a modelling study. Lancet Public Health. (2020) 5:e271–8. doi: 10.1016/s2468-2667(20)30082-7

PubMed Abstract | Crossref Full Text | Google Scholar

52. Munro, AA-O, and Faust, SN. Children are not COVID-19 super spreaders: time to go back to school. Arch Dis Child. (2020) 105:618–9. doi: 10.1136/archdischild-2020-319474

Crossref Full Text | Google Scholar

53. Bhattacharyya, A, Seth, A, Srivastava, N, Imeokparia, M, and Rai, S. Coronavirus (COVID-19): a systematic review and Meta-analysis to evaluate the significance of demographics and comorbidities. Res Sq. (2021). doi: 10.21203/rs.3.rs-144684/v1

PubMed Abstract | Crossref Full Text | Google Scholar

54. Das, P, Nadim, SS, Das, S, and Das, P. Dynamics of COVID-19 transmission with comorbidity: a data driven modelling based approach. Nonlinear Dyn. (2021) 106:1197–211. doi: 10.1007/s11071-021-06324-3 PMID 33716405

PubMed Abstract | Crossref Full Text | Google Scholar