Heterogeneity in the Effectiveness of Non-pharmaceutical Interventions During the First SARS-CoV2 Wave in the United States

Background: Attempts to quantify effect sizes of non-pharmaceutical interventions (NPI) to control COVID-19 in the US have not accounted for heterogeneity in social or environmental factors that may influence NPI effectiveness. This study quantifies national and sub-national effect sizes of NPIs during the early months of the pandemic in the US.

Methods: Daily county-level COVID-19 cases and deaths during the first wave (January 2020 through phased removal of interventions) were obtained. County-level cases, doubling times, and death rates were compared to four increasingly restrictive NPI levels. Socio-demographic, climate and mobility factors were analyzed to explain and evaluate NPI heterogeneity, with mobility used to approximate NPI compliance. Analyses were conducted separately for the US and for each Census regions (Pacific, Mountain, east/West North Central, East/West South Central, South Atlantic, Middle Atlantic and New England). A stepped-wedge cluster-randomized trial analysis was used, leveraging the phased implementation of policies.

Results: Aggressive (level 4) NPIs were associated with slower COVID-19 propagation, particularly in high compliance counties. Longer duration of level 4 NPIs was associated with lower case rates (log beta −0.028, 95% CI −0.04 to −0.02) and longer doubling times (log beta 0.02, 95% CI 0.01–0.03). Effects varied by Census region, for example, level 4 effects on doubling time in Pacific states were opposite to those in Middle Atlantic and New England states. NPI heterogeneity can be explained by differential timing of policy initiation and by variable socio-demographic county characteristics that predict compliance, particularly poverty and racial/ethnic population. Climate exhibits relatively consistent relationships across Census regions, for example, higher minimum temperature and specific humidity were associated with lower doubling times and higher death rates for this period of analysis in South Central, South Atlantic, Middle Atlantic, and New England states.

Conclusion and Relevance: Heterogeneity exists in both the effectiveness of NPIs across US Census regions and policy compliance. This county-level variability indicates that control strategies are best designed at community-levels where policies can be tuned based on knowledge of local disparities and compliance with public health ordinances.

Introduction

During the first COVID-19 wave (15th January to 31th May 2020), the disease spread rapidly across the globe, infecting over 3 million people with the SARS-CoV-2 virus. One-third of all reported cases and one-quarter of all deaths were in the United States. Public health interventions to reduce the spread of COVID-19 in the US varied in timing and level of aggressive measures undertaken. No official US policy existed to prevent COVID-19 transmission until January 31, 2020, when a presidential order blocked the entry of non-US citizens into the US traveling from China. The state of Washington declared the first State of Emergency (February 29), followed by California (March 4) and Maryland (March 5); however, many counties implemented restrictive policies prior to state action. For example, three counties in Washington (King, Pierce, Snohomish) and four in Arkansas (Grant, Jefferson, Pulaski, Saline) ordered school closings on March 12 vs. statewide closures on March 17; counties in Pennsylvania, California, and New Jersey closed non-essential businesses prior to the state; and counties in California and Idaho issued restrictions on mass gatherings before state policies.

Early action by counties was motivated by both the ability to legislate response and the multifaceted issues that impact disease risk at local levels. Such issues include racial disparities, with a disproportionate number of Black and Hispanic Americans reported to be infected or dying (1–3), urban vs. rural characteristics that influence transmission and policy implementation (4), and disease spillover from neighboring counties, which can be exacerbated by economic disparities and shared environmental risks (5). There is growing consensus that SARS-CoV-2 is an airborne pathogen, spread primarily through respiratory aerosols (6–8) that can be influenced by both micro- and macro-environments. Much work has focused on indoor micro-environments to help understand superspreading events, such as in restaurants, call centers, and large social gatherings (9–11). The relationship between COVID-19 transmission and macro-environmental factors is more difficult, with early work evaluating relative humidity (RH), temperature, and UV exposure; however, findings have varied with some studies finding no effect (12), inverse relationships (13), or mixed effects (14, 15). Elevated humidity has been associated with an increase in organic aerosols and higher levels of small particulate air pollutants (PM2.5, PM10, and O₃) (16, 17), which have been correlated with transmission (18, 19); however, the simultaneous use of RH and temperature in models of viral respiratory illness is not recommended due to their direct functional relationship (20).

Among other factors, the absence of vaccines during the COVID-19's first wave resulted in the application of non-pharmaceutical interventions (NPI, also known as community mitigation strategies) to control COVID-19. Unfortunately, without a randomized trial to quantify the effect size of NPIs or the potential causes of heterogeneity across different regions of the US, policymakers relied on modeling studies and early evidence from Asia to guide decisions. These included susceptible-exposed-infectious-recovered (SEIR) models attempting to quantify the effects of isolation and contact tracing (21) and how varying NPI effectiveness may influence demands on critical care resources (22); and studies in Wuhan and Hong Kong reporting how strict interventions (i.e., quarantine, social distancing, shelter in place and active case detection) reduced the COVID-19 reproductive number (R₀) (23, 24). Over the past year, several studies have attempted to quantify NPI effects, with a recent meta-analysis summarizing studies from around the world that quantified how various strategies helped contain COVID incidence (25). In the US, the majority of studies trying to quantify NPI effects have focused on the state-level (i.e., state-level policies impacting state-level cases and deaths) (26–29). For example, White and Hebert-Dufresne (27) analyzed five state-level policy effects, finding that only restaurant restrictions significantly correlated with higher doubling times; Chernozhukov et al. (26) study four state policies using a structural equation modeling approach, showing important effects of masking, business closures, and stay-at-home orders on both cases and deaths; and Auger (28) studied the effect of statewide school closures on COVID-19 cases and deaths using a negative binomial interrupted time series analysis, finding a 62% decline in cases and 58% decline in mortality following school closure. However, these studies ignore county-level heterogeneity and restrict their analyses to a specific set of policies without considering the joint effect of two policies implemented simultaneously. Aggregating county-level data ignores important social determinants as well as the reliance on county governments by states to make essential decisions regarding policy implementation and enforcement that explain variations in NPI effectiveness (29, 30). The most comprehensive county-level analysis to date in the US was conducted by Ebrahim et al. (30), who found both widespread county-level variation in policies and identified business closures as having the most important effect on COVID-19 cases; however, this study was limited to just one-third of US counties and relied on the estimation of R0 rather than using reported cases.

The goal of this study is to evaluate national and sub-national effects of the four levels of NPIs using county-level data on policies, cases and socio-environmental factors during the first wave in the US until implementation to Phase 1 reopening (lifting of policies), if specified, otherwise to May 29. We leverage the phased implementation of policies at the county level, using a stepped-wedge cluster-randomized trial (SW-CRT) framework (31). We quantify NPI effects on COVID-19 daily case incidence, doubling time, and reported deaths across nine US Census regions.

Methods

Outcome Data

Multiple data sources were used to confirm county-level daily cases and deaths from SARS-CoV2 infection. Data from the Johns Hopkins University Center for System Science and Engineering Coronavirus Resource Center (JHU-CSSE, https://coronavirus.jhu.edu/) were compared to county data reported on state health department's websites, using the state data when discrepancies were noted (i.e., counties from 34 states using the JHU-CSSE data were discrepant with state-reported county data). In addition, any county whose cumulative cases or deaths declined over time was flagged and adjusted using state- or county data. The final data are counties from all 50 US states and the District of Columbia extending from January 22 through May 29, 2020. US territories (American Samoa, Guam, the Northern Mariana Islands, Puerto Rico, and the U.S. Virgin Islands) were excluded.

Policy Data

The effective date of each public health intervention and the phased reopening at the state and county levels was initially extracted from online policy databases (32–36). For discrepancies or missing county policies, we obtained policy dates in two steps: (1) searching the county's state government website for reported county policies; and (2) if state websites did not report county policies or if the county reported at least one COVID-19 case prior to the issuance of any state order, we conducted a systematic search of gray literature for each county's policies, focusing on county websites (if existent) and local news websites. We categorized 12 policies into 4 levels of disease control following the New Zealand alert system and Oxford classification (37, 38): Level 1 (low)—governor declaration of a State of Emergency; Level 2 (moderate)—school closures, restricting access (visits) to nursing homes, or closing restaurants and bars; Level 3 (high) – non-essential business closures, suspending non-violent arrests, suspending elective medical procedures, suspending evictions, or restricting mass gatherings of at least 10 people; and Level 4 (aggressive)—sheltering in place / stay-at-home, public mask requirements, or travel restrictions. These levels are mostly cumulative, meaning counties tended to implement policies sequentially and jointly, for example, 67% of counties implemented almost all level 3 policies at the same time, while 23% implemented them within 7 days of their initial level 3 policy. Note that our initial analysis found no effect of the two federal policies blocking entry to the US for non-US citizens (i.e., from China issued January 31 and from Schengen European countries issued March 11) on COVID-19 morbidity or mortality propagation; thus, we classified them as the “non-intervention” period. Finally, NPI effects were measured up to 5 days after county reopening, defined as opening non-essential businesses (with capacity restrictions), allowing public gatherings of more than 10 people, opening public spaces, or easing shelter in place orders. All but 12 states entered some phase of reopening by May 29 and six states allowed counties to open at their discretion (CA, IA, MD, NE, OR, WY).

Policy compliance was measured by comparing the number of trips recorded in each county from 2020 to 2019 based on data from the Maryland Transportation Institute and Center for Advanced Transportation Technology (https://data.bts.gov/Research-and-Statistics/Trips-by-Distance/w96p-f2qv). Several studies have demonstrated the utility of mobility data as proxy measures of policy compliance (30, 39–41). Using results from Nouvellet et al. (42), we defined gradients of mobility decline associated with reductions in R0. A four-level variable for compliance was created, where non-compliance (level 0) was defined as a mobility difference from 2019 to 2020 of <15%, low compliance (level 1) as a decline in mobility of 15–30%, moderate compliance (level 2) as a decline of 30–50% and high compliance (level 3) as mobility declines >50%. We then summed the time-lagged 10-day compliance and divided by maximum compliance (30) to create a scaled measure, from 0 to 10, representing the average level of policy compliance over the past 10 days. A value closer to 10 indicates high compliance (low mobility compared to 2019), while values closer to zero indicate lower compliance (more mobility). We expect to see values close to zero prior to the initiation of polices and as policies were removed.

Demographic and Environmental Data

Demographic data are from the US Census Bureau. This includes county-level age, sex and racial composition, migration, and educational data from the 2018 American Community Survey (43), land area to compute population density (1,000 people per square-km) (44), and poverty (45). Analyses were stratified by the nine US Census regions (Pacific, Mountain, West North Central, East North Central, West South Central, East South Central, South Atlantic, Middle Atlantic, New England) to evaluate differential policy effects.

We use the USDA Rural-Urban Continuum Code that categorizes counties into nine levels of rural-urban characteristics (46). Three levels indicate metropolitan areas of (1) 1 million or more people, (2) 250,000 to 1 million people, and (3) fewer than 250,000 people. Four urban levels classified by size and adjacency to a metropolitan area: (4) 20,000 or more people adjacent; (5) 20,000 or more, not adjacent; (6) 2,500 to 19,999, adjacent; and (7) 2,500 to 19,999, not adjacent. And two rural levels: (8) <2,500 population, adjacent to a metropolitan area; and (9) <2,500 population, not adjacent.

Environmental data are from the North American Land Data Assimilation System (NLDAS). The NLDAS provides several daily hydrometeorological measures, for which we define 10-day temporal lags of minimum air temperature (Celsius), specific humidity (g/kg) and bias-corrected shortwave radiation (W/m²).

Statistical Methods

Outcomes were defined at the county level as the number of new daily cases, new deaths, and case doubling time. Doubling time is the number of days required to double the cumulative case count on a particular day. Policy levels were time-varying, coded as a 1 while a level was active and 0 otherwise, and as the number of days since an intervention level was initiated.

Descriptive differences in policy implementation were examined using Chi-square and t-tests. To evaluate intervention levels and socio-environmental factors, negative binomial mixed models were fit using an approach similar to a Hussey and Hughes SW-CRT analysis (47). The model is specified as follows:

$\begin{array}{l}ln(y_{it})=ln(N_i)+x_0{\beta }_0+x_1{\beta }_1+\dots x_p{\beta }_p+u_i\end{array}$

Where y_it is the outcome (cases, deaths, doubling time) for county i on study day t; ln(N_i) is the offset where N_i is the population density for county i; parameters x_pβ_p represent the fixed predictors (x_p) and their associated parameters; and u_i is the random county effect. For the new cases and death models, we use population density as the offest, but no offset is included for doubling time. In addition, for case and death models, period (study day) is included as a continuous variable while the doubling time model uses period as a categorical variable (similar to the SW-CRT modeling approach). When entered as a continuous variable, we evaluated inclusion of linear and quadratic period terms. Each outcome was fit for the country as a whole and separately for each US Census region, combining East and West North Central states, East and West South Central states, and Middle Atlantic and New England states. Finally, to test policy effects, we evaluate the time-varying policy variables, i.e., duration of intervention, which are entered into case and doubling time models simultaneously, but as individual time-varying policy effects in death rate models (i.e. comparing policy 2 vs. 1 or nothing, policy 3 vs. 0–2, etc.). Since there are four policies, we use the Holm-Bonferroni multiple comparisons correction to evaluate significance (48).

Case and doubling time models were identified using one randomly selected census region and then evaluating model fit using AIC (49). Variables considered include: rural-urban continuum code; minority (Black and Hispanic) and total population density; net county migration rate in 2018; percent of the 2018 county population Black (alone or mixed race), Hispanic (alone or mixed race), living in poverty, or with a college education or higher; and climate parameters (10-day lags for minimum temperature, specific humidity, and UV radiation). Once a final model was determined, it was used to fit to all regions combined (adding an indicator for census region) and for each individual census region. Final models are shown in the full regression tables (Supplementary Table B0 for the country as a whole; Supplementary Tables B1, C1, D1 for COVID-19 models of cases, doubling time, and deaths by Census region, respectively). Climate variables were evaluated only using doubling time and death outcomes. Final models were fit using SAS 9.4 with Gaussian adaptive quadrature.

Secondary to our evaluation of policy effects, we use the above models to describe social and environmental disparities observed across Census regions. In addition, we conduct an analysis on factors associated with policy compliance by comparing change in county compliance during the first 15 and 30 days of March to socioeconomic and environmental characteristics. Therefore, since we define compliance as a 10-day lagged sum of compliance, we fit two models to estimate factors associated with compliance change from March 1 to 15 (i.e., change compliance from April 21-March 1 to March 5–15) and change from March 1 to 30. The compliance model includes random county effects within states and estimates both standardized and non-standardized beta coefficients. Standardized Beta coefficients only standardize predictor variables, not the response; therefore, the interpretation of the standardized beta is a one standard deviation unit change in a covariate being associated with a change in compliance. We note that on March 1, 98.8% of counties did not have any COVID policies; by March 15, 50.9% had enacted at least policy level 2; and by March 30, 98.9% of counties established policy level 3 or 4.

Results

We obtained complete data from 3,142 counties. 339 counties (10.8%) from 26 states created policies prior to their state government, the majority (211 or 62%) were located in Texas, Nebraska, Missouri, and Pennsylvania. Counties initiating policies prior to the state were more likely to be located in metropolitan areas (17.5% metro counties vs. 6.8% non-metro countries adopted early policies, p < 0.001), have populations with higher educational levels, higher percentages of Hispanic population, and fewer people living in poverty (respectively, counties with early policy adoption had 26.3, 15.9, and 13.8% of their populations with a college degree, Hispanic descent [mixed or alone], and living in poverty, vs. 21.0, 8.5, and 15.3% for countries that did not adopt early policies, p < 0.001).

Speed of policy adoption varied, with fewer days spanning initial case detection and initiation of NPIs in counties located in North and South Central states, while counties in New England, Middle Atlantic, and Pacific states had more days between initial case detection and policy initiation (Figure 1). However, in these later three regions, the first COVID-19 case was detected an average of 10 days prior to North and South Central states. Counties in these three regions (Pacific, Middle Atlantic and New England) had the longest duration of level 1 policies, but the shortest duration of time from initial case detection to the most strict (level 4) policies (Table 1). County and state governments in New England and Mountain states were more likely to initiate policies before the first reported COVID-19 death compared to other regions (data not shown). Once initiated, policy duration averaged 5.7, 3.6, 11.9, and 44.3 days for levels 1–4, respectively, with significant variation by state and some states having zero days for any particular level (Supplementary Table A1, Table 1).

FIGURE 1

Figure 1. Time between first COVID-19 case to policy initiation by US census region (mean days and 95% confidence interval). Policies classified under each level of intervention: Level 2—school closures, restricting access to nursing homes, closing restaurants and bars; Level 3—non-essential business closures, suspending non-violent arrests, suspending elective medical procedures, suspending evictions, restricting mass gatherings of at least 10 people; Level 4—sheltering in place/stay-at-home, mask requirements in public, travel restrictions. States classified for each region are as follows: Pacific—CA, WA, OR, HI, AK; Mountain—MT, WY, ID, NV, UT, CO, AZ, NM; West North Central—ND, SD, NE, KS, MN, IA, MO; East North Central—WI, IL, MI, IN, OH; West South Central—TX, OK, AR, LA; East South Central—KY, TN, MS, AL; South Atlantic—FL, GA, SC, NC, VA, WV, DC, MD, DE; Middle Atlantic—NY, PA, NJ; New England—ME, CT, NH, MA, CT, RI.

TABLE 1

Table 1. Median policy duration by US census region.

Compliance with policies was highly variable. As expected, prior to the initiation of most policies (January and February 2020), compliance measures were low (i.e., no policies to follow). Most states and counties initiated strict policies in March 2020, which is reflected by the sharp increase in compliance in most regions during this month (Figure 2). In Middle Atlantic and New England States, there was high, homogenous compliance with few counties reporting moderate changes to mobility; however, counties in Mountain, West North Central, East South Central and South Atlantic exhibited high heterogeneity in compliance. Even within states, considerable heterogeneity was observed; for example, four states did not have any counties implement level 4 policies: Iowa, Nebraska, North Dakota and South Dakota. Among these four, policy compliance increased rapidly, but only counties in Iowa maintained high and relatively homogenous policy compliance across the state (Supplementary Table A1, Supplementary Figure A1). Similarly, four states had over 70 days with a level 4 policy (California, Hawaii, Illinois, New Jersey) with three experiencing high, homogenous compliance across the state and one (Illinois) having increasing heterogeneity in compliance over time (Supplementary Figure A1).

FIGURE 2

Figure 2. Compliance with policy interventions for US counties by census region. Each dot represents a county in the US and each county has a daily time series of compliance. The red line is a fitted penalized b-spline. US Census Regions: Pacific—CA, WA, OR, HI, AK; Mountain—MT, WY, ID, NV, UT, CO, AZ, NM; West North Central—ND, SD, NE, KS, MN, IA, MO; East North Central—WI, IL, MI, IN, OH; West South Central—TX, OK, AR, LA; East South Central—KY, TN, MS, AL; South Atlantic—FL, GA, SC, NC, VA, WV, DC, MD, DE; Middle Atlantic—NY, PA, NJ; New England—ME, CT, NH, MA, CT, RI.

Policy Effects on COVID-19 Propagation and Mortality

Case Rates

Model results for the country as a whole indicate that, under conditions of high policy compliance (scaled value of 8), intervention level 4 achieved a 50% reduction in COVID-19 case rates in 16 days (95% CI for 16-day case reduction: 40.2–61.8%), compared to 22 days for intervention level 3 (95% CI: 35.1–71.5%, Figure 3, Table 2). Intervention levels 1 and 2 never achieve significant reductions in case rates; in fact, longer duration of level 1 policies was positively associated with cases. Under conditions of low compliance (scaled value of zero), only duration of level 4 policies achieved a significant decline, while level 1 policies were inversely associated with COVID-19 cases. The time needed for level 4 policies to achieve a 50% decline increased to 20 days under these conditions.

FIGURE 3

Figure 3. Mean effect of each policy level on the COVID-19 case rate if the policy was implemented for 15 days, low and high compliance. The solid blue line is the mean effect size for each intervention with 95% confidence intervals (shaded blue). A value of 1 represents no change, 1.5 is a 50% increase in cases and 0.5 is a 50% decrease. The left column is for low compliance (no change in behavior in 2020 compared to 2019, scaled value 0) and the right is for high compliance (scaled value 8). Effect sizes are from negative binomial models reported in Supplementary Table B0.

TABLE 2

Table 2. Model predicted policy effects on case rates, doubling time, and death rates for all US counties.

Policy effects were variable across US Census regions (Table 3). For an average level of policy compliance (scaled value of 6), intervention level 4 was significantly associated with declines in COVID-19 case rates in Pacific and Mountain regions, yet did not achieve significance in other regions (Table 3, Supplementary Figure B1, Supplementary Table B1). Further, intervention level 3 was associated with a reduction in cases in the Pacific region, but an increase in cases in North Central States. Intervention level 1 was the only intervention not associated with any changes to COVID-19 rates across all US Census regions.

TABLE 3

Table 3. Multivariate model results^* for case incidence and doubling time by US census region.

As noted, policy compliance impacted policy effectiveness at national and Census region levels. Each unit increase in compliance was associated with a log beta decline in case rates of 0.002 and 0.001 for policy level 3 and 4, respectively (95% CI, −0.0024 to −0.0016; −0.0012 to −0.0008 respectively) at the national level. Compliance during level 4 policies was significantly associated with reduced case rates across all US Census regions (Supplementary Table B1).

Doubling Time

For the country as a whole, only duration of policy level 4 achieved statistical significance to increase doubling time (Table 2). Levels 2 and 3 had p-values under 0.05, but after adjusting for multiple comparisons, we cannot reject the null hypothesis of no effect; however, it is noteworthy their effect sizes were positive, indicating that the level 2 and 3 policies trended toward reduction of overall COVID-19 propagation. Each day on intervention level 4 was associated with an increase in log beta doubling time of 0.02 (95% CI: 0.01–0.03, p < 0.0001). When calculating the predicted doubling time from the data, level 4 policies achieve a peak 40 days after initiation with an estimated doubling time of 24 days (95% CI, 19.1–29.5) compared to 23 days after initiation of level 3 policies for an estimated doubling time of 17.6 days (95% CI, 14.5–22.3) (Figure 4). Levels 1 and 2 never achieve increased doubling times.

FIGURE 4

Figure 4. Model-predicted doubling time by policy intervention level. Model predictions are from the final Doubling Time model shown in Supplementary Table B0.

By US Census region, duration of level 4 policies in Pacific, Mountain and North Central states have a positive effect on doubling time, but in other regions, duration of level 4 policies reduced doubling time (Table 3, Supplementary Table C2). In addition, level 3 policy duration was significantly related to longer doubling times in Pacific and Mountain states, but shorter doubling times in Middle Atlantic and New England states. Level 1 policy duration was significantly associated with shorter doubling times in South Central, South Atlantic, Middle Atlantic and New England states, and longer doubling times in Pacific and North Central states.

Policy compliance was an important factor to explain doubling time. At the national level, higher policy compliance increased doubling time (log beta 0.036, 95% CI 0.03–0.04), yet higher compliance during policies 1 through 3 reduced doubling time. In other words, both policy level 3 and compliance increase doubling times; however, as compliance increases, the effect of policy level 3 declines. By Census region, the effect of compliance varied, increasing doubling time in North Central and South Atlantic states, but reducing doubling times in others.

Deaths

This analysis includes data from 73,676 COVID-19 deaths, the majority in Middle Atlantic (36%), East North Central (19%), New England (15%), South Atlantic (11%), and Pacific (7%) states. For the country as a whole, duration of policy levels 3 and 4 were both significantly associated with higher death rates (Table 2); however, the distribution of deaths indicate potential heterogeneity. Indeed, model results by Census region indicate duration of levels 1 and 2 had no effect on death rates for any Census region (Supplementary Table D1); level 3 policy duration was significantly related to lower COVID-19 death rates in North Central states, yet higher death rates in South Central, South Atlantic, Middle Atlantic, and New England states; and duration of level 4 was only related to lower death rates in Pacific states (other regions with p-values under 0.05 were not significant after the Holm-Bonferroni adjustment). We note that models for death rates had to be simplified with fewer covariates to allow for model convergence.

The effect of policy duration on death rates is sensitive to measures of compliance. This is particularly true for level 3 policies as seen in Figure 5, where high levels of compliance resulted in significantly lower death rates that low levels of compliance in North Central, South Central, and South Atlantic states. Policy compliance did not influence the effectiveness of other policies.

FIGURE 5

Figure 5. Risk ratio and 95% confidence intervals of COVID-19 deaths by duration of policy for each US census region. Shaded areas represent the 95% confidence interval for each region. US Census Regions: Pacific—CA, WA, OR, HI, AK; Mountain—MT, WY, ID, NV, UT, CO, AZ, NM; North Central—ND, SD, NE, KS, MN, IA, MO, WI, IL, MI, IN, OH; South Central—TX, OK, AR, LA, KY, TN, MS, AL; South Atlantic—FL, GA, SC, NC, VA, WV, DC, MD, DE; Middle Atlantic—NY, PA, NJ; New England—ME, CT, NH, MA, CT, RI.

Social and Environmental Disparities

As noted in Methods, we conducted a secondary evaluation of social and environmental factors associated with COVID-19 cases and deaths, as well as factors related to policy compliance. These variables include county-level measures of: poverty, unemployment and income; average level of educational attainment; socio-demographic characteristics (population density, minority population, %Black, %Hispanic); and hydro-meteorological characteristics. Model results related to policy compliance are reported in Table 4.

TABLE 4

Table 4. Socioeconomic characteristics associated with policy compliance (changes in mobility) during the first 15 and 30 days of March 2020.

Poverty

Poverty and income data used in this study are from the US Census Bureau's Small Area Income and Poverty Estimate (SAIPE). At the national level, as expected, higher levels of poverty are associated with shorter doubling times and higher death rates, yet, counterintuitively, lower case rates. When evaluating poverty effects by Census region, poverty was not associated with doubling times in any region, but was positively associated with case rates in Mountain states and inversely related to cases in all other regions. In contrast, elevated poverty was consistently associated with elevated COVID-19 death rates in all regions. Although poverty was not associated with 15-day change in compliance, a county's poverty level was the most important factor explaining low compliance with any policy after 30 days.

Education

Education data are 5-year averages estimated by the American Community Survey. At the national level, the percent of the county with at least a Bachelor's degree was not associated with case rates, but was associated with increased doubling times. By Census region, higher education was consistently associated with increased doubling time, yet variability existed when analyzing case rates with education associated with higher case rates in Mountain, Middle Atlantic, and New England states, but lower cases in South Central states. Higher education was also consistently associated with increased policy compliance, for both the 15 and 30 day changes in compliance.

Socio-Demographic

Social and demographic data are county racial, ethnic and total population characteristics: %Black, %Hispanic, population density, and the USDA Urban-Rural Classification. None of these factors were identified as related to death rates and population density was not evaluated in case rate models as population size is used as an offset in the model. At the national level, %Black and %Hispanic populations were associated with elevated case rates, which was consistent across Census regions. Case rates were also consistently higher in metropolitan areas as well as non-metropolitan areas with more than 20,000 people compared to smaller counties (i.e., non-metro areas under 20,000 people and rural counties). By Census region, %Black and %Hispanic were both associated with elevated cases in North Central, South Atlantic, Middle Atlantic, and New England states. %Black was also associated with elevated cases in South Central states while %Hispanic was associated with elevated cases in Pacific states. Case rates by urban-rural classification varied considerably by Census region: Pacific and South Atlantic states had similar patterns with smaller counties having higher rates that larger counties; Middle Atlantic and New England states had the opposite patterns (large counties had higher rates compared to smaller counties); and the other Census regions had mixed patterns. Counties with the highest policy compliance levels tended to have larger %Hispanic, lower %Black populations, and be in counties with larger population sizes (i.e., non-metropolitan counties with >20 K people and larger).

Climate

Hydrometeorological data were used to define 10-day temporal lags of minimum air temperature, humidity, and shortwave radiation. The relationship between climate parameters and COVID-19 was variable, and these relationships often differed between case doubling time and deaths (Supplementary Figures D1, D2). Solar radiation, for example, which is frequently invoked as a negative forcing on COVID-19 transmission on account of its relationship with UV radiation intensity (50, 51), is associated with increased case doubling time (i.e., decreased transmission) in coastal regions but with decreased doubling time in the North Central and South Atlantic regions. For deaths, most northern census regions (Middle Atlantic, New England, North Central, Mountain) had tend toward decreased deaths with increased solar radiation, but southern regions had mixed relationships.

Increases in specific humidity and minimum temperature were associated with decreased doubling time and increased deaths for the county as a whole (specific humidity: log beta −0.006, 95% CI −0.009 to −0.003 for doubling time, 0.015, 95% CI 0.003–0.027 for deaths; minimum temperature: log beta −0.005, 95% CI −0.006 to −0.003 for doubling time, log beta 0.015, 95% CI 0.0075-0.0225 for deaths). In the South Central, South Atlantic and Middle Atlantic/New England Census regions, minimum temperature and specific humidity exhibited the same inverse relationship with doubling time (i.e., higher values, shorter doubling times) and positive relationship with death rates (higher values, higher death rates). Other regions had contrasting relationships. For example, specific humidity was significantly associated with lower doubling times in Pacific states, but higher doubling times in North Central; however, specific humidity had a (non-significant) downward trend with death rates in those two regions.

Regarding policy compliance, after 30 days, no climate factors were associated with compliance.

Discussion

This study evaluates the effectiveness of four non-pharmaceutical intervention categories on COVID-19 case rates, doubling time, and deaths at the county level in the US, and the heterogeneity that exists across Census regions. We find that during the first wave of COVID-19, the most restrictive NPI policies (level 4) were the most effective at reducing case rates and increasing doubling time of COVID-19 compared to any other policy level. In addition, we observed that higher levels of policy compliance, as measured by changes in county-level mobility from 2019 to 2020, resulted in larger reductions in cases but lower increases in doubling time for these restrictive NPI. Analysis of NPI effectiveness across Census regions revealed strong variation across regions and within states. Level 4 policies were associated with reduced case incidence only in Pacific states and increased doubling time only in Mountain states, yet associated with higher case rates in North Central States and lower doubling times in multiple regions, noting in particular that the effect sizes for doubling time in Mountain vs. Middle Atlantic/New England states were exactly the same value but in opposite directions.

Surprisingly, duration of level 4 policies was associated with higher rates of death at the national level, but when analyzed separately by Census region, associated with lower rates of death in Pacific, North Central, and South Atlantic states (also trended to lower rates in Mountain and South Central regions, but was not statistically significant). Given than Middle Atlantic and New England states comprised 51% of all deaths reported during the first wave, the relationship in these two regions likely dominated the national trend. The initiation of level 4 policies was slowest in these regions, allowing for both case and mortality momentum that resulted in the positive association observed. It is clear from Supplementary Table D1 that level 4 policies not only reduced death rates in other regions, but higher compliance may complement policy effectiveness.

Why do NPIs exhibit such variation in effects? As alluded to above, policy compliance likely plays a key role, which we measured as the change in mobility over a 10-day period in 2019 vs. 2020. It is not surprising that we find compliance to be associated with enhanced policy effectiveness and reduced COVID-19 burden, i.e., higher compliance was significantly related to lower case rates and higher doubling times for each day on level 4 policies in all Census regions, as well as lower death rates in most regions during both level 3 and level 4 policies. However, we found considerable variation in compliance both within Census regions and within states (Figure 2, Supplementary Figure A1) indicating that adherence to policies has strong community and census-level mediating factors. This is consistent with our findings that that compliance varies by levels of poverty, educational level, racial/ethnic composition, and rurality. Recent studies corroborate these findings and also highlight the importance of perceived risk, occupation, and local social pressures (52–55). Importantly, we find poverty levels to be the most important factor in explaining compliance, which was also found in other studies (52, 56). This relationship between poverty, policy compliance, and COVID-19 risk is tied to the overall literature of population vulnerability and the burdens of low-income households to maintain food security and financial stability during a time when businesses and employment opportunities were closing (57).

Overall, the results of this study are consistent with findings from Wuhan (23) and Europe (58) that reported significant declines in the effective reproductive number following implementation of NPIs that included quarantine, travel restriction, shelter-in-place, school and business closures, and social distancing. Results are also consistent with studies in the US highlighting the effect of shelter-in-place (5, 59) and closure of schools, restaurants and businesses (26–28). However, in contrast to these studies, we combine multiple NPIs into an ordered grouping of policies that tended to be implemented simultaneously, thereby avoiding the potential biases in assigning attributable risk reduction to individual policies when their roll-out occurs concurrently with others. For example, 51% of counties had school and restaurant closures occur <2 days apart (level 2 NPI), and 55% initiated restrictions on mass gatherings and non-essential business closures within 2 days (level 3 NPI). It is very difficult, statistically, to measure independent policy effects in an observational study when those policies are implemented in such close temporal proximity. The stepped-wedge approach, although not designed for observational data, is a novel attempt to control for these overlapping periods.

The stepped wedge approach was only applied to the doubling time models, primary due to difficulties in achieving model convergence for case and death rate models. For doubling time, we produced predicted estimates for each NPI adjusting for all covariates entered into the model (Figure 4, Supplementary Figure C1). These predicted doubling times adjust for all the observed heterogeneity across counties, demonstrating that policy impact is likely not continuous in reality, but achieves a peak after a certain amount of time. Overall, we report level 4 achieving peak doubling time after 40 days and 23 days for level 3. This peaking is likely associated with reduced perceptions of local risk and reduced compliance even when policies are still in effect. For Census regions, they all appear to reach a peak doubling time for level 4 policies with the exception of Middle Atlantic and East North Central, which tended to have less heterogeneity in policy compliance compared to other regions as indicated in Figure 2. This plateauing of intervention effects remains an important area of research, which is tied to issues related to adherence, transportation, and racial and economic disparities. The timing of policy implementation may also have introduced heterogeneity across regions. We observed that states experiencing early cases (Pacific, New England, and Mid-Atlantic) had significantly longer gap times between case detection and NPI initiation than states experiencing their first COVID-19 case later. This difference in policy initiation time is likely due to non-coastal US Regions learning from experience of coastal regions to implement NPIs more quickly. Unfortunately, early response during the first wave, which proved effective, may have given policy-makers false confidence as 12 states reported case spikes on June 23 and 7 states reported highs for hospitalization (https://covidtracking.com/)—all but one of these states began removing social distancing protections by May 11 and all but four are located in Mountain, North and South Central regions.

In addition to evaluating policy effects, this study reported the effects of temperature, specific humidity, and solar radiation related to COVID-19. These mixed results highlight a number of important points on climate predictors of COVID-19: (1) the regional heterogeneity of the relationship between climate parameters and SARS-CoV-2 is consistent with the variability observed in studies of MERS-CoV infection and climate (60–63); (2) climate analyses are sensitive to choice of response variable and analysis period, and may not provide stable results at the stage of epidemic data collection considered in this study—for example, our results tend to run in the opposite direction of those reported by Ma et al. (64), but the studies use different response variables (R_t vs. doubling rate), as well as different periods of analysis, ranges of climate variability, and statistical methods; (3) there is substantial inter-regional variability in climate sensitivities, such that large scale analyses are not necessarily representative of regional climate influence; and (4) it is difficult to isolate climate effects from those of other predictors, and analyses that examine climate variables without adequate control for policy and other factors are prone to spurious climate associations. We recommend further research be conducted at different spatial scales (community, census tract, etc.) to better characterize the climate-COVID-19 relationship.

Limitations

There are several limitations to note in our study. First, the analysis does not have an accurate representation of the availability of testing (or the number of tests administered) at the county level for the time series. As this availability changed over time for all counties in the US, we cannot accurately characterize the population at risk for COVID-19 case detection. However, we are more confident in COVID-related deaths reported. Second, our measure of compliance is based on reduced mobility in countries, but not a direct measure of adherence. In some states, policy levels 3 and 4 were viewed as an affront to civil liberties. While the use of mobility data has been shown to be an appropriate proxy for compliance, studies focusing on smaller geographic units may be able to obtain more accurate data on compliance to measure NPI “dose.” Third, we do not look at individual interventions. However, this was a choice as the sets of NPIs are considered a more appropriate response than single interventions, which have never been employed historically without others. Finally, we would ideally be applying these methods to a randomized design, which is impossible for COVID-19. Thus, our inferences are drawn from observational data regardless of our SW-CRT analytical framework.

Conclusion

The most aggressive NPIs (shelter-in-place, public mask requirements, and travel restrictions) were the only policies that are consistently associated with a reduction in COVID-19 cases and doubling times in the US between January 2020 and the phased re-opening of states. However, when analyzing by Census region, considerable variation of NPI effectiveness is observed, likely due to variations in policy adherence. Socio-environmental factors, including poverty, racial/ethnic status and educational levels, contribute to heterogeneity of COVID-19 propagation, NPI adherence and NPI effectiveness. These results may inform public health policy as states continue to manage the ongoing pandemic.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Author Contributions

WP and ST conceived the study. WP, ST, BZ, PL, and CW contributed to the design of the study. WP, RD, and BZ organized the data. WP, DF, and G-VI contributed to analysis. WP wrote the first draft of the manuscript. All authors contributed to manuscript revision, read, and approved the submitted version.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpubh.2021.754696/full#supplementary-material

References

1. Yancy CW. COVID-19 and African Americans. JAMA. (2020) 323:1891–2. doi: 10.1001/jama.2020.6548

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Garg S, Kim L, Whitaker M, O'Halloran A, Cummings C, Holstein R, et al. Hospitalization rates and characteristics of patients hospitalized with laboratory-confirmed coronavirus disease 2019 — COVID-NET, 14 states. Morb Mortality Wkly Rep. (2020) 69:458–64. doi: 10.15585/mmwr.mm6915e3

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Millett GA, Jones AT, Benkeser D, Baral S, Mercer L, Beyrer C, et al. Assessing differential impacts of COVID-19 on black communities. Ann. Epidemiol. (2020) 47:37–4. doi: 10.1016/j.annepidem.2020.05.003

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Prusaczyk B. Strategies for disseminating and implementing COVID-19 public health prevention practices in rural areas. J Rural Health. (2021) 37:142–44. doi: 10.1111/jrh.12432

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Lyu W, Wehby GL. Comparison of estimated rates of coronavirus disease 2019 (COVID-19) in border counties in iowa without a stay-at-home order and border counties in Illinois with a stay-at-home order. JAMA Netw Open. (2020) 3:e2011102. doi: 10.1001/jamanetworkopen.2020.11102

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Prather KA, Marr LC, Schooley RT, McDiarmid MA, Wilson ME, Milton DK. Airborne transmission of SARS-CoV-2. Science. (2020) 370:303–4. doi: 10.1126/science.abf0521

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Klompas M, Baker MA, Rhee C. Airborne transmission of SARS-CoV-2: theoretical considerations and available evidence. JAMA. (2020) 324:441–2. doi: 10.1001/jama.2020.12458

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Azuma K, Yanagi U, Kagi N, Kim H, Ogata M, Hayashi M. Environmental factors involved in SARS-CoV-2 transmission: effect and role of indoor environmental quality in the strategy for COVID-19 infection control. Environ Health Prev Med. (2020) 25:66. doi: 10.1186/s12199-020-00904-2

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Mahale P, Rothfuss C, Bly S, Kelley M, Bennett S, Huston SL, et al. Multiple COVID-19 outbreaks linked to a wedding reception in rural Maine - August 7-September 14, 2020. MMWR Morb Mortal Wkly Rep. (2020) 69:1686–90. doi: 10.15585/mmwr.mm6945a5

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Ghinai I, Woods S, Ritger KA, McPherson TD, Black SR, Sparrow L, et al. Community transmission of SARS-CoV-2 at two family gatherings - Chicago, Illinois, February-March 2020. MMWR Morb Mortal Wkly Rep. (2020) 69:446–50. doi: 10.15585/mmwr.mm6915e1

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Park SY, Kim YM, Yi S, Lee S, Na BJ, Kim CB, et al. Coronavirus disease outbreak in call center, South Korea. Emerg Infect Dis. (2020) 26:1666–70. doi: 10.3201/eid2608.201274

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Yao Y, Pan J, Liu Z, Meng X, Wang W, Kan H, et al. No association of COVID-19 transmission with temperature or UV radiation in Chinese cities. Eur Respir J. (2020) 55:2000517. doi: 10.1183/13993003.00517-2020

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Wu Y, Jing W, Liu J, Ma Q, Yuan J, Wang Y, et al. Effects of temperature and humidity on the daily new cases and new deaths of COVID-19 in 166 countries. Sci Total Environ. (2020) 729:139051. doi: 10.1016/j.scitotenv.2020.139051

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Ma Y, Zhao Y, Liu J, He X, Wang B, Fu S, et al. Effects of temperature variation and humidity on the death of COVID-19 in Wuhan, China. Sci Total Environ. (2020) 724:138226. doi: 10.1016/j.scitotenv.2020.138226

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Qi H, Xiao S, Shi R, Ward MP, Chen Y, Tu W, et al. COVID-19 transmission in Mainland China is associated with temperature and humidity: a time-series analysis. Sci Total Environ. (2020) 728:138778. doi: 10.1016/j.scitotenv.2020.138778

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Jia L, Xu Y. Effects of relative humidity on ozone and secondary organic aerosol formation from the photooxidation of benzene and ethylbenzene. Aerosol Sci Technol. (2014) 48:1–12. doi: 10.1080/02786826.2013.847269

CrossRef Full Text | Google Scholar

17. Carlton AG, Wiedinmyer C, Kroll JH. A review of Secondary Organic Aerosol (SOA) formation from isoprene. Atmos Chem Phys. (2009) 9:4987–5005. doi: 10.5194/acp-9-4987-2009

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Wu X, Nethery RC, Sabath BM, Braun D, Dominici F. Air pollution and COVID-19 mortality in the United States: Strengths and limitations of an ecological regression analysis. Sci Adv. (2020) 6:eabd4049. doi: 10.1126/sciadv.abd4049

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Fernandez D, Gine-Vazquez I, Liu I, Yucel R, Nai Ruscone M, Morena M, et al. Are environmental pollution and biodiversity levels associated to the spread and mortality of COVID-19? A four-month global analysis. Environ Pollut. (2021) 271:116326. doi: 10.1016/j.envpol.2020.116326

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Davis RE, McGregor GR, Enfield KB. Humidity: a review and primer on atmospheric moisture and human health. Environ Res. (2016) 144(Pt A):106–16. doi: 10.1016/j.envres.2015.10.014

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Hellewell J, Abbott S, Gimma A, Bosse NI, Jarvis CI, Russell TW, et al. Feasibility of controlling COVID-19 outbreaks by isolation of cases and contacts. Lancet Glob Health. (2020) 8:e488–96. doi: 10.1016/S2214-109X(20)30074-7

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Kissler SM, Tedijanto C, Goldstein E, Grad YH, Lipsitch M. Projecting the transmission dynamics of SARS-CoV-2 through the postpandemic period. Science. (2020) 368:860–68. doi: 10.1126/science.abb5793

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Pan A, Liu L, Wang C, Guo H, Hao X, Wang Q, et al. Association of Public Health Interventions With the Epidemiology of the COVID-19 Outbreak in Wuhan, China. JAMA. (2020) 323:1915–23. doi: 10.1001/jama.2020.6130

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Cowling BJ, Aiello A. Public health measures to slow community spread of COVID-19. J Infect Dis. (2020) 221:1749–51. doi: 10.1093/infdis/jiaa123

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Perra N. Non-pharmaceutical interventions during the COVID-19 pandemic: a review. Phys Rep. (2021) 913:1–52. doi: 10.1016/j.physrep.2021.02.001

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Chernozhukov V, Kasahara H, Schrimpf P. Causal impact of masks, policies, behavior on early covid-19 pandemic in the U.S. J Econom. (2021) 220:23–62. doi: 10.1016/j.jeconom.2020.09.003

PubMed Abstract | CrossRef Full Text | Google Scholar

27. White ER, Hebert-Dufresne L. State-level variation of initial COVID-19 dynamics in the United States. PLoS ONE. (2020) 15:e0240648. doi: 10.1371/journal.pone.0240648

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Auger KA, Shah SS, Richardson T, Hartley D, Hall M, Warniment A, et al. Association between statewide school closure and COVID-19 incidence and mortality in the US. JAMA. (2020) 324:859–70. doi: 10.1001/jama.2020.14348

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Jalali AM, Peterson BM, Galbadage T. Early COVID-19 interventions failed to replicate 1918 St. Louis vs. Philadelphia Outcomes in the United States. Front Public Health. (2020) 8:579559. doi: 10.3389/fpubh.2020.579559

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Ebrahim S, Ashworth H, Noah C, Kadambi A, Toumi A, Chhatwal J. Reduction of COVID-19 incidence and nonpharmacologic interventions: analysis using a US County-level policy data set. J Med Internet Res. (2020) 22:e24614. doi: 10.2196/24614

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Hemming K, Haines TP, Chilton PJ, Girling AJ, Lilford RJ. The stepped wedge cluster randomised trial: rationale, design, analysis, and reporting. BMJ. (2015) 350:h391. doi: 10.1136/bmj.h391

PubMed Abstract | CrossRef Full Text | Google Scholar

32. National Governor's Association. Available online at: https://www.nga.org/coronavirus/#resElections. https://www.nga.org/coronavirus/#resElections (accessed April 24, 2020).

33. MultiState Associates. Available online at: https://www.multistate.us (accessed April 24, 2020). Available online at: https://docs.google.com/spreadsheets/d/e/2PACX-1vRlJWZJ7OkGUW57_rdA2n3xBJ3qjW6u4Z9N6K9Y5L4bM_6H7-S308qdKmJfpVstYWf300nyujvZPFSy/pubhtml?urp=gmail_link#.

34. Council of State Governments. Available online at: https://web.csg.org/covid19/executive-orders/. https://web.csg.org/covid19/executive-orders/ (accessed April 24, 2020).

35. Officials AoSaTH. Available online at: https://coronavirus-astho.hub.arcgis.com/. https://coronavirus-astho.hub.arcgis.com/ (accessed April 28, 2020).

36. Kaiser Family Foundation. Available online at: https://www.kff.org/health-costs/issue-brief/state-data-and-policy-actions-to-address-coronavirus/#casesdeathstrend; https://www.kff.org/health-costs/issue-brief/state-data-and-policy-actions-to-address-coronavirus/#casesdeathstrend: Kaiser Family Foundation (accessed April 29, 2020).

37. COVID-19 Alert System. Unite against COVID-19. (2020). Available online at: https://covid19.govt.nz/alert-system/covid-19-alert-system/ (accessed April 29, 2020).

38. Hale T, Webster S, Petherick A, Phillips T, Kira B. Oxford COVID-19 Government Response Tracker. Data use policy: Creative Commons Attribution CC BY standard. Oxford: Blavatnik School of Government (2020).

39. Kissler SM, Kishore N, Prabhu M, Goffman D, Beilin Y, Landau R, et al. Reductions in commuting mobility correlate with geographic differences in SARS-CoV-2 prevalence in New York City. Nat Commun. (2020) 11:4674. doi: 10.1038/s41467-020-18271-5

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Dainton C, Hay A. Quantifying the relationship between lockdowns, mobility, and effective reproduction number (Rt) during the COVID-19 pandemic in the Greater Toronto Area. BMC Public Health. (2021) 21:1658. doi: 10.1186/s12889-021-11684-x

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Deforche K, Vercauteren J, Muller V, Vandamme AM. Behavioral changes before lockdown and decreased retail and recreation mobility during lockdown contributed most to controlling COVID-19 in Western countries. BMC Public Health. (2021) 21:654. doi: 10.1186/s12889-021-10676-1

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Nouvellet P, Bhatia S, Cori A, Ainslie KEC, Baguelin M, Bhatt S, et al. Reduction in mobility and COVID-19 transmission. Nat Commun. (2021) 12:1090. doi: 10.1038/s41467-021-21358-2

PubMed Abstract | CrossRef Full Text | Google Scholar

43. 2014–2018 American Community Survey 5-Year Estimates (2020). Available online at: https://data.census.gov/mdat/#/ (accessed April 24, 2020).

44. US Census Bureau TIGER/Line Shapefiles (2020). Available online at: https://www.census.gov/geographies/mapping-files/time-series/geo/tiger-line-file.2019.html (accessed April 24, 2020).

45. SAIPE State and County Estimates for 2018. (2020). Available online at: https://www.census.gov/data/datasets/2018/demo/saipe/2018-state-and-county.html

46. Economic Research Service (ERS) 2013 Rural-Urban Continuum Codes (2020). Available online at: https://www.ers.usda.gov/data-products/rural-urban-continuum-codes/.

47. Hussey MA, Hughes JP. Design and analysis of stepped wedge cluster randomized trials. Contemp Clin Trials. (2007) 28:182–91. doi: 10.1016/j.cct.2006.05.007

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Abdi H. Holm's sequential bonferroni procedure. In: Salkind N, editor. Encyclopedia of Research Design. Vol 1. Thousand Oaks, CA: Sage (2010).

Google Scholar

49. Akaike H. Likelihood of a model and information criteria. J Econometr. (1981) 16:3–14. doi: 10.1016/0304-4076(81)90071-3

CrossRef Full Text | Google Scholar

50. Schuit M, Ratnesar-Shumate S, Yolitz J, Williams G, Weaver W, Green B, et al. airborne SARS-CoV-2 is rapidly inactivated by simulated sunlight. J Infect Dis. (2020) 222:564–71. doi: 10.1093/infdis/jiaa334

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Merow C, Urban MC. Seasonality and uncertainty in COVID-19 growth rates. Proc Natl Acad Sci. (2020) 117:27456–64. doi: 10.1073/pnas.2008590117

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Papageorge NW, Zahn MV, Belot M, van den Broek-Altenburg E, Choi S, et al. Socio-demographic factors associated with self-protecting behavior during the COVID-19 pandemic. J Popul Econ. (2021). doi: 10.1007/s00148-020-00818-x. [Epub ahead of print].

PubMed Abstract | CrossRef Full Text | Google Scholar

53. Hills S, Eraso Y. Factors associated with non-adherence to social distancing rules during the COVID-19 pandemic: a logistic regression analysis. BMC Public Health. (2021) 21:352. doi: 10.1186/s12889-021-10379-7

PubMed Abstract | CrossRef Full Text | Google Scholar

54. Gouin JP, MacNeil S, Switzer A, Carrese-Chacra E, Durif F, Knauper B. Socio-demographic, social, cognitive, and emotional correlates of adherence to physical distancing during the COVID-19 pandemic: a cross-sectional study. Can J Public Health. (2021) 112:17–28. doi: 10.17269/s41997-020-00457-5

PubMed Abstract | CrossRef Full Text | Google Scholar

55. Coroiu A, Moran C, Campbell T, Geller AC. Barriers and facilitators of adherence to social distancing recommendations during COVID-19 among a large international sample of adults. PLoS ONE. (2020) 15:e0239795. doi: 10.1371/journal.pone.0239795

PubMed Abstract | CrossRef Full Text | Google Scholar

56. Hawkins D. Differential occupational risk for COVID-19 and other infection exposure according to race and ethnicity. Am J Ind Med. (2020) 63:817–20. doi: 10.1002/ajim.23145

PubMed Abstract | CrossRef Full Text | Google Scholar

57. Sharma SV, Haidar A, Noyola J, Tien J, Rushing M, Naylor BM, et al. Using a rapid assessment methodology to identify and address immediate needs among low-income households with children during COVID-19. PLoS ONE. (2020) 15:e0240009. doi: 10.1371/journal.pone.0240009

PubMed Abstract | CrossRef Full Text | Google Scholar

58. Flaxman S, Mishra S, Gandy A, Unwin HJT, Mellan TA, Coupland H, et al. Estimating the effects of non-pharmaceutical interventions on COVID-19 in Europe. Nature. (2020) 584:257–61. doi: 10.1038/s41586-020-2405-7

PubMed Abstract | CrossRef Full Text | Google Scholar

59. Castillo RC, Staguhn ED, Weston-Farber E. The effect of state-level stay-at-home orders on COVID-19 infection rates. Am J Infect Control. (2020) 48:958–60. doi: 10.1016/j.ajic.2020.05.017

PubMed Abstract | CrossRef Full Text | Google Scholar

60. Altamimi A, Ahmed AE. Climate factors and incidence of Middle East respiratory syndrome coronavirus. J Infect Public Health. (2020) 13:704–8. doi: 10.1016/j.jiph.2019.11.011

PubMed Abstract | CrossRef Full Text | Google Scholar

61. Gardner EG, Kelton D, Poljak Z, Van Kerkhove M, von Dobschuetz S, Greer AL. A case-crossover analysis of the impact of weather on primary cases of Middle East respiratory syndrome. BMC Infect Dis. (2019) 19:113. doi: 10.1186/s12879-019-3729-5

PubMed Abstract | CrossRef Full Text | Google Scholar

62. Ward MP, Xiao S, Zhang Z. The role of climate during the COVID-19 epidemic in New South Wales, Australia. Transbound Emerg Dis. (2020) 67:2313–7. doi: 10.22541/au.158879258.84484606

PubMed Abstract | CrossRef Full Text | Google Scholar

63. Alghamdi IG, Hussain II, Almalki SS, Alghamdi MS, Alghamdi MM, El-Sheemy MA. The pattern of Middle East respiratory syndrome coronavirus in Saudi Arabia: a descriptive epidemiological analysis of data from the Saudi Ministry of Health. Int J Gen Med. (2014) 7:417–23. doi: 10.2147/IJGM.S67061

PubMed Abstract | CrossRef Full Text | Google Scholar

64. Ma Y, Pei S, Shaman J, Dubrow R, Chen K. Role of meteorological factors in the transmission of SARS-CoV-2 in the United States. Nat Commun. (2021) 12:3602. doi: 10.1038/s41467-021-23866-7

PubMed Abstract | CrossRef Full Text | Google Scholar