Excess death estimates compared with state-reported and observed COVID-19 deaths, New Jersey and the United States, 2020–2022

Deaths associated with COVID-19 in the United States are currently estimated to be over 1.2 million, but the true burden of mortality due to the SARS-CoV-2 virus is unknown. Methods for identifying and reporting deaths related to COVID-19 differ between jurisdictions, and concerns about overreporting and underreporting exist. Excess death estimates for the pandemic period, based on data from the National Center for Health Statistics, may be used to approximate the number of COVID-19-associated deaths. In this analysis, we first describe the process by which the New Jersey Department of Health identified, classified, and reported COVID-19-associated deaths from January 2020 through December 2022. The National Center for Health Statistics’ excess deaths estimates are first compared with New Jersey’s reported COVID-19-associated deaths, and then with the observed COVID-19-associated deaths in the entire United States, by month, from January 2020 through December 2022. New Jersey’s reported COVID-19-associated deaths (n = 35,555) accounted for (and slightly exceeded) the state’s excess deaths estimated by the National Center for Health Statistics for 2020–2022 (n = 30,365). However, the overall number of United States observed COVID-19 deaths for 2020–2022 (n = 1,094,230) for the study period did not account for all estimated excess deaths in the nation for the same period (n = 1,233,366). The general congruence of New Jersey’s reported COVID-19 deaths and the National Center for Health Statistics’ excess death estimates may be due in part to New Jersey’s early detailed classification system for identifying and reporting deaths associated with COVID-19, leading to more accurate COVID-19 death reporting by the state.

Introduction

The COVID-19 pandemic has had a devastating impact on the United States (US) with over 1 million deaths attributed to the virus as of September 2023 (1). The true toll of the pandemic may be even higher, as supported by studies indicating deaths due to COVID-19 have been undercounted (2–4). The Council of State and Territorial Epidemiologists (CSTE) released a case definition for identifying and classifying COVID-19-associated deaths on December 22, 2021 (5). Prior to then, there was no standard definition for reporting COVID-19-associated deaths, and jurisdictions may have used different methods to count these deaths. Using the same criteria to count COVID-19 deaths in all US jurisdictions allows for meaningful comparisons between communities and contributes to a more accurate national picture of the true mortality burden of the pandemic.

Since COVID-19-associated deaths may have been undercounted (2–4), one way to measure the potential burden of mortality related to the COVID-19 pandemic is to estimate the number of excess deaths. Estimates of excess deaths can be calculated in a variety of ways, and both the World Health Organization (WHO) and the US Centers for Disease Control and Prevention (CDC) have developed methodologies to estimate excess mortality associated with COVID-19. WHO’s methodology continues to be developed to account for nations that have sparse or missing COVID-19-associated death data. CDC’s excess death estimates provide data for both the US overall and for individual states/jurisdictions, which were required to report COVID-19-associated deaths during the pandemic. CDC defines excess deaths as the difference between the observed number of deaths and the expected number of deaths during a specific period (6). Excess deaths can be due to all causes, including COVID-19 and other infectious and chronic diseases. In this analysis, we describe the process by which the New Jersey Department of Health (NJDOH) identified, classified, and reported COVID-19-associated deaths from January 2020 through December 2022. Additionally, National Center for Health Statistics’ (NCHS) excess death estimates for New Jersey (NJ) and the US are compared with NJ’s reported COVID-19 deaths and with US observed COVID-19 deaths, respectively, from 2020 to 2022. These comparisons can help determine how differing reporting processes may affect the congruence between COVID-19 death counts and excess death estimates. By examining COVID-19-associated deaths reported in the US, there can be a better understanding of the true health impact the pandemic had on the nation.

Materials and methods

Data and analysis

The study period was from January 1, 2020, to December 31, 2022. Two data sets were used: CDC’s NCHS National and State Estimates of Excess Deaths data set (6) and the CDC Weekly United States COVID-19 Cases and Deaths by State data set (7). Since the week-ending dates differ for the two data sets, all numbers provided as weekly counts in these data sets were aggregated into monthly counts for comparison purposes. The following metrics were used to compare the two data sets: NCHS excess death estimates for NJ and the US, NJ COVID-19 deaths reported to CDC, and US observed COVID-19 deaths.

The NCHS National and State Estimates of Excess Deaths data set (6) was used for all-cause excess death estimates for NJ and the US. This data was extracted from the national Electronic Death Reporting System (EDRS) on March 6, 2023. The data set provides weekly estimates of excess deaths for national, state, and other jurisdictions, by date of death. Excess death estimates were calculated as the difference between the “observed count and one of two thresholds [either the average expected count or the upper bound of the 95% prediction interval (threshold)], by week and jurisdiction” (6). Negative values, where the observed count fell below the threshold, were set to zero.

The CDC Weekly United States COVID-19 Cases and Deaths by State data set (7) provides weekly case and death counts reported by states and territories to the CDC. The data set for NJ’s reported COVID-19 deaths was extracted on August 25, 2023. National observed death counts (provided in the NCHS National and State Estimates of Excess Deaths data set) were compared with NCHS excess death estimates. Counts for US observed COVID-19 deaths were calculated by subtracting national observed death counts for ‘All-causes – except COVID-19’ from national observed death counts for ‘All-causes’ for each month in the study period. ‘All-causes – except COVID-19’ death estimates excluded deaths with an International Classification of Diseases, Tenth Edition (ICD-10) code of U07.1 as an underlying or multiple cause of death; ‘All-causes’ included all-cause mortality.

Classifying COVID-19 cases and ascertaining COVID-19 deaths

Classifying COVID-19 deaths as confirmed or probable in NJ between 2020 and 2022 was based on meeting the criteria for COVID-19 case ascertainment as published by CSTE in position statements released in April 2020, August 2020, and August 2021 (8–10), according to which case definition was in effect when the case expired.

Confirmed COVID-19 deaths included deceased individuals with confirmatory laboratory evidence (i.e., detection of SARS-CoV-2 RNA in a clinical specimen using a molecular amplification detection test; or detection of SARS-CoV-2 RNA in a clinical or autopsy specimen using a molecular amplification test; or detection of SARS-CoV-2 RNA in a post-mortem respiratory swab or clinical specimen using a diagnostic molecular amplification test performed by a CLIA-certified provider; or detection of SARS-CoV-2 by genomic sequencing, depending on which case definition was in place at the time of death). Probable COVID-19 deaths included individuals with presumptive laboratory evidence (i.e., detection of a specific antigen in a clinical specimen or detection of a specific antibody in serum, plasma, or whole blood indicative of a new or recent infection; or detection of SARS-CoV-2 by antigen test in a respiratory specimen; or detection of SARS-CoV-2 specific antigen in a post-mortem obtained respiratory swab or clinical specimen using a diagnostic test performed by a CLIA-certified provider), depending on the case definition in effect at the time of death. Probable COVID-19 deaths also included individuals who had died without laboratory confirmation but for whom COVID-19 disease or SARS-CoV-2 was listed as a cause of death or a significant condition contributing to death on the death certificate, as well as individuals with symptoms compatible with COVID-19 and a known COVID-19 exposure who were not tested for the SARS-CoV-2 virus.

Since the surveillance definition did not take the elapsed time between illness onset or date of positive specimen collection and date of death into consideration for associating a death to COVID-19, NJDOH collaborated with the CDC to establish additional guidelines for the review of cases and deaths before categorizing them as COVID-19-associated. Thus, COVID-19 deaths were also classified as confirmed if they met the following criteria:

(1) Individuals with confirmatory laboratory evidence who expired ≤30 days of a confirmatory lab test, except if a fully explanatory alternative cause of death is known that is causally unrelated to COVID-19 (such as a homicide or accident), OR

(2) Individuals with confirmatory laboratory evidence who expired >30 days after confirmatory lab test AND met at least one of the following criteria:

(A) COVID-19 or equivalent is mentioned on the death certificate, or

(B) COVID-19 findings/symptoms (such as ARDS or pneumonia) were documented in NJ’s electronic Communicable Disease Registry and Surveillance System (CDRSS) or on the death certificate, OR there is epidemiological evidence suggesting death is otherwise temporally and causally related to COVID-19, and:

(i) The individual did not experience complete recovery back to baseline state of health (per information from local health department, hospital infection preventionist, or long-term care facility), and

(ii) The individual did not have a fully explanatory alternative cause of death that is determined to be causally unrelated to COVID-19 (i.e., homicide, accident, etc.).

The following criteria were used in NJ to classify COVID-19 deaths as probable:

(1) COVID-19 or equivalent is mentioned on the death certificate, OR

(2) Individuals with presumptive laboratory evidence that met COVID-19 clinical (via CDRSS or death certificate) or epidemiological criteria (according to the surveillance case definition) who expired ≤30 days after presumptive lab test, OR

(3) Individuals with presumptive laboratory evidence that met COVID-19 clinical (CDRSS or death certificate) criteria according to the surveillance case definition who expired >30 days after presumptive lab test, and:

(A) The individual did not experience complete recovery back to baseline state of health (determined as described above), and

(B) The individual did not have a fully explanatory alternative cause of death that is determined to be causally unrelated to COVID-19 (such as a homicide or accident), OR

(4) Individuals who met COVID-19 clinical (CDRSS or death certificate) and epidemiological criteria according to the surveillance case definition (no positive COVID-19 test results, meaning either no COVID-19 testing was performed or negative test results were present but preceded the individual’s terminal illness onset as determined by CDRSS record review) who expired ≤30 days of symptom onset, OR

(5) Individuals who met COVID-19 clinical (CDRSS or death certificate) and epidemiological criteria according to the surveillance case definition (no positive COVID-19 test results, as defined above) who expired >30 days of symptom onset, and.

(A) The individual did not experience complete recovery back to baseline state of health, and

(B) The individual did not have a fully explanatory alternative cause of death that is determined to be causally unrelated to COVID-19 (i.e., accident, homicide).

Between 2020 and 2022, NJDOH used two sources to quantify deaths associated with COVID-19: Electronic death records data and public health disease surveillance and investigation. Electronic death records data from the New Jersey Electronic Death Records System (NJ-EDRS) were pulled and matched to COVID-19 cases in CDRSS using demographic data manually reviewed by NJDOH staff 5–7 days a week. Information on matched cases was utilized to update the mortality status of known COVID-19 cases whose deaths were not reported by the LHD, to verify COVID-19 as the cause of death on known fatalities of COVID-19 cases, and to identify COVID-19 cases that were not reported through public health reporting and surveillance. The majority of COVID-19-associated deaths reported by NJ were identified through this mechanism, which minimized lag time between the date of death and the reporting date as NJDOH had access to NJ-EDRS data in real-time. This also decreased the workload on local health departments (LHD) to identify and report COVID-19-associated deaths to NJDOH. LHDs received reports of COVID-19-associated deaths from healthcare facilities and local death registrars. Additionally, they identified confirmed and probable deaths through investigations of persons who tested positive for COVID-19, were symptomatic close contacts of a COVID-19 case, or were associated with a COVID-19 outbreak or cluster.

Results

NJDOH began reporting deaths associated with COVID-19 in March 2020. As shown in Figure 1, NJ’s reported COVID-19 deaths closely track NCHS excess death estimates for the state, although reporting in some months did exceed or fall below excess death estimates due to lag times in death certificate reporting, incomplete medical histories, and human error. NJ-reported COVID-19 deaths exceeded estimated excess deaths the most in June and July 2020, from February through May 2021, and in February 2022 (Table 1). NJ reported substantially fewer COVID-19 deaths than excess death estimates in March and April 2020, in January 2022, and in October and December 2022. Overall, NJ-reported COVID-19-associated death counts (n = 35,555) ultimately accounted for and slightly exceeded the total excess deaths estimated by NCHS for NJ for 2020–2022 (n = 30,365).

Figure 1

Figure 1. NJ Reported COVID-19 deaths compared with NCHS excess death estimates, 2020–2022. Sources: CDC COVID-19 provisional counts weekly death estimates (6), CDC weekly United States COVID-19 cases and death by state (7).

Table 1

Table 1. Differences between NCHS all-cause excess death estimates (NJ and US) and reported (NJ)/observed (US) COVID-19 deaths by month, January 2020 – December 2022.

Observed COVID-19 deaths in the US are compared with NCHS excess death estimates for the US in Figure 2. Observed deaths exceeded excess death estimates the greatest from February through May 2021, in January 2022, and in March and April 2022 (Table 1). Observed US deaths were markedly lower than estimated deaths for several months earlier in the pandemic (April, July and August 2020), as well as later in 2021 (August through December), and in much of 2022 (February and June through December). Decreased congruity is seen with this comparison, with larger numbers of excess estimated deaths than observed COVID-19 deaths for close to half the months in the study period. The overall number of US-observed COVID-19 deaths from January 2020 through December 2022 (n = 1,094,230) falls short of accounting for all estimated excess deaths in the US for the same period (n = 1,233,366) by over 11%.

Figure 2

Figure 2. US observed COVID-19 deaths compared with NCHS excess death estimates, 2020–2022. Source: CDC COVID-19 provisional counts weekly excess deaths by Jurisdiction (6).

Discussion

Overall, NJ’s reported COVID-19 deaths from 2020 to 2022 closely align with excess death estimates produced by NCHS and account for all estimated excess deaths during that period. This is consistent with the absence of any other historical events causing widespread fatalities in either NJ or the US at that time. The general congruence of NJ-reported COVID-19 deaths and NCHS excess death estimates may be partly the result of establishing an early detailed classification system for identifying, counting and reporting deaths associated with COVID-19, leading to accurate COVID-19 death reporting in NJ. The inclusion of criteria for classifying both confirmed and probable COVID-19 deaths in NJ, as well as the presence of two mechanisms by which deaths associated with COVID-19 were identified by NJDOH (public health investigation/disease surveillance and electronic death records/death certificate data), has established a robust system that identified and reported COVID-19-associated deaths accurately overall. Despite the general agreement between the NJ-reported COVID-19 deaths and the NCHS excess death estimates, NJ-reported COVID-19-associated deaths for the study period exceeded overall NJ excess death estimates by approximately 17%. This difference may at least partially due to the continued high levels of circulating COVID-19 during the study period in NJ, which could have resulted in higher COVID-19 deaths than provisional excess death estimates had accounted for. Additionally, the number of NJ-reported deaths associated with COVID-19 may be a more accurate reflection of the true mortality burden of the pandemic in the state, given the indirect effects on health for the public (such as deferred illness screenings and treatments during hospital surges, increases in depression and social isolation, and elevated abuse of substances, all of which negatively affect health and contribute to higher mortality). Unlike in NJ, the total number of observed COVID-19 deaths in the US did not account for all the estimated excess deaths for the same period. This suggests a potential underreporting of COVID-19 deaths across the US.

Limitations include the possibility that the weighting method used in excess death estimates could over-adjust for underreporting, thereby producing inflated excess death estimates (6). However, given the lack of accessible and reliable testing for COVID-19 at the beginning of the pandemic (therefore artificially decreasing deaths associated with COVID-19 since no testing was performed), the issue that decedents may not have been tested for COVID-19 even after testing became more widely available (and so may not have had a COVID-19 diagnosis listed as a cause of death on their death certificates), and the differing methods and criteria used to identify, classify and count COVID-19-associated deaths, underreporting of these deaths is a distinct possibility for many jurisdictions. Differing methods can lead to different results; therefore, jurisdictional comparisons of COVID-19 deaths should be approached with great caution. Additionally, because the excess death estimates used in this study were for all-cause mortality, some of the pandemic’s secondary effects on mental health, substance abuse patterns, healthcare services, etc. are likely captured in these estimates. However, since many of these effects have not been quantified and measured, it would not be possible to determine the extent to which they influenced trends in mortality in this study. Finally, since aggregated US data were not available in the CDC Weekly United States COVID-19 Cases and Deaths by State data set, the category ‘US observed COVID-19 deaths’ was calculated as a proxy to compare with NCHS excess death estimates for the US jurisdiction, and a direct comparison could not be performed. Comparisons between excess death estimates and reported/observed deaths by age group and gender also were not performed, as these breakdowns were not included in the data sets.

There continues to be debate about the underreporting and/or overreporting of COVID-19-associated deaths (11, 12). Despite this, NCHS excess death estimates are available to help validate the numbers of jurisdiction-reported COVID-19 deaths. These estimates are used by the public health community and serve as the standard by which many jurisdictions calculate excess deaths caused by COVID-19 during the pandemic. NJDOH established a process for identifying, classifying and reporting COVID-19-related deaths early in the pandemic. The process included reporting both confirmed and probable deaths; this approach seems to have allowed for NJ’s reported COVID-19 deaths to track similarly to NCHS excess death estimates throughout most of the pandemic, and ultimately account for the jurisdiction’s estimated excess deaths from 2020 to 2022. This may be a useful approach for other jurisdictions where excess death estimates exceed reported COVID-19 deaths to more accurately understand the true burden of mortality caused by the pandemic. Accurate COVID-19 death reporting allows for understanding the human toll of the pandemic and targeting resources to jurisdictions with the greatest need. Additionally, improving death reporting and registration systems would allow for improved monitoring not just for COVID-19, but for any future pandemics as well.

Data availability statement

Publicly available datasets were analyzed in this study. This data can be found at: https://covid.cdc.gov/covid-data-tracker/#datatracker-home, https://data.cdc.gov/Case-Surveillance/Weekly-United-States-COVID-19-Cases-and-Deaths-by-/pwn4-m3yp.

Author contributions

DR: Writing – review & editing, Writing – original draft, Visualization, Methodology, Formal analysis, Conceptualization. LM: Writing – original draft, Visualization, Methodology, Data curation. MO: Writing – review & editing, Methodology. NB: Writing – review & editing, Supervision, Conceptualization. EL: Writing – review & editing, Supervision, Conceptualization.

Funding

The author(s) declare that financial support was received for the research, authorship, and/or publication of this article. This publication is supported by the New Jersey Department of Health and the Centers for Disease Control and Prevention of the U.S. Department of Health and Human Services (HHS) as part of a financial assistance award totaling $10,658,181.43. The contents are those of the author(s) and do not necessarily represent the official views of, nor an endorsement by, CDC/HHS, or the U.S. Government.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

1. Centers for Disease Control and Prevention. COVID data tracker. Centers for Disease Control and Prevention. CDC; (2020). Available at: https://covid.cdc.gov/covid-data-tracker/#datatracker-home. Accessed September on 20, 2023.

Google Scholar

2. Paglino, E, Lundberg, DJ, Wrigley-Field, E, Zhou, Z, Wasserman, J, Raquib, R, et al. Excess natural-cause mortality in US counties and its association with reported COVID-19 deaths. Proc Natl Acad Sci. (2024). 121:e2313661121. doi: 10.1073/pnas.2313661121

Crossref Full Text | Google Scholar

3. Campolieti, M. COVID-19 deaths in the USA: Benford's law and under-reporting. J Public Health (Oxf). (2022) 44:e268–71. doi: 10.1093/pubmed/fdab161

PubMed Abstract | Crossref Full Text | Google Scholar

4. COVID-19 Excess Mortality Collaborators. Estimating excess mortality due to the COVID-19 pandemic: a systematic analysis of COVID-19-related mortality, 2020-21. Lancet. (2022) 399:1513–36. doi: 10.1016/S0140-6736(21)02796-3

PubMed Abstract | Crossref Full Text | Google Scholar

5. Council of State and Territorial Epidemiologists. Interim Guidance for Public Health Surveillance Programs for Classification of COVID-19-associated Deaths among COVID-19 Cases. (2021). Available at: https://preparedness.cste.org/wp-content/uploads/2022/03/20211222_interim-guidance.pdf. Accessed on September 20, 2023.

Google Scholar

6. Centers for Disease Control and Prevention. COVID-19 Provisional Counts Weekly Excess Deaths by Jurisdiction. National Center for Health Statistics. (2023). Available at: https://www.cdc.gov/nchs/nvss/vsrr/covid19/excess_deaths.htm. Accessed on September 20, 2023.

Google Scholar

7. Centers for Disease Control and Prevention. Weekly United States COVID-19 Cases and Deaths by State. (2023). Available at: https://data.cdc.gov/Case-Surveillance/Weekly-United-States-COVID-19-Cases-and-Deaths-by-/pwn4-m3yp. Accessed on August 25, 2023.

Google Scholar

8. Council of State and Territorial Epidemiologists. Standardized surveillance case definition and national notification for 2019 novel coronavirus disease (COVID-19). (2020). Available at: https://cdn.ymaws.com/www.cste.org/resource/resmgr/2020ps/interim-20-id-01_covid-19.pdf. Accessed on October 5, 2023.

Google Scholar

9. Council of State and Territorial Epidemiologists. Update to the standardized surveillance case definition and national notification for 2019 novel coronavirus disease (COVID-19). (2020). Available at: https://cdn.ymaws.com/www.cste.org/resource/resmgr/ps/positionstatement2020/Interim-20-ID-02_COVID-19.pdf. Accessed on October 5, 2023.

Google Scholar

10. Council of State and Territorial Epidemiologists. Update to the standardized surveillance case definition and national notification for 2019 novel coronavirus disease (COVID-19). (2021). Available at: https://cdn.ymaws.com/www.cste.org/resource/resmgr/ps/ps2021/21-ID-01_COVID-19.pdf. Accessed on October 5, 2023.

Google Scholar

11. Stokes, A, Lundberg, D, Wrigley-Field, E, and Chen, YH. COVID-19 deaths in the US continue to be undercounted, research shows, despite claims of ‘overcounts’. (2023). Available at: https://theconversation.com/covid-19-deaths-in-the-us-continue-to-be-undercounted-research-shows-despite-claims-of-overcounts-198266. Accessed on October 25, 2023.

Google Scholar

12. Wen, LS. We are overcounting COVID deaths and hospitalizations, that is a problem. (2023). Available at: https://www.phillytrib.com/commentary/we-are-overcounting-covid-deaths-and-hospitalizations-thats-a-problem/article_1beb4dfd-11d3-5f82-872d-f3474b789108.html. Accessed on October 25, 2023.

Google Scholar