Facing the Omicron variant—how well do vaccines protect against mild and severe COVID-19? Third interim analysis of a living systematic review

Background: The severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) Omicron variant is currently the dominant variant globally. This third interim analysis of a living systematic review summarizes evidence on the effectiveness of the coronavirus disease 2019 (COVID-19) vaccine (vaccine effectiveness, VE) and duration of protection against Omicron.

Methods: We systematically searched literature on COVID-19 for controlled studies, evaluating the effectiveness of COVID-19 vaccines approved in the European Union up to 14/01/2022, complemented by hand searches of websites and metasearch engines up to 11/02/2022. We considered the following comparisons: full primary immunization vs. no vaccination, booster immunization vs. no vaccination, and booster vs. full primary immunization. VE against any confirmed SARS-CoV-2 infection, symptomatic, and severe COVID-19 (i.e., COVID-19-related hospitalization, ICU admission, or death) was indicated, providing estimate ranges. Meta-analysis was not performed due to high study heterogeneity. The risk of bias was assessed with ROBINS-I, and the certainty of the evidence was evaluated using GRADE.

Results: We identified 26 studies, including 430 to 2.2 million participants, which evaluated VE estimates against infections with the SARS-CoV-2 Omicron variant. VE against any confirmed SARS-CoV-2 infection ranged between 0–62% after full primary immunization and between 34–66% after a booster dose compared to no vaccination. VE range for booster vs. full primary immunization was 34–54.6%. After full primary immunization VE against symptomatic COVID-19 ranged between 6-76%. After booster immunization VE ranged between 3-84% compared to no vaccination and between 56-69% compared to full primary immunization. VE against severe COVID-19 ranged between 3-84% after full primary immunization and between 12-100% after booster immunization compared to no vaccination, and 100% (95% CI 71.4-100) compared to full primary immunization (data from only one study). VE was characterized by a moderate to strong decline within 3–6 months for SARS-CoV-2 infections and symptomatic COVID-19. Against severe COVID-19, protection remained robust for at least up to 6 months. Waning immunity was more profound after primary than booster immunization. The risk of bias was moderate to critical across studies and outcomes. GRADE certainty was very low for all outcomes.

Conclusions: Under the Omicron variant, the effectiveness of EU-licensed COVID-19 vaccines in preventing any SARS-CoV-2 infection is low and only short-lasting after full primary immunization, but can be improved by booster vaccination. VE against severe COVID-19 remains high and is long-lasting, especially after receiving the booster vaccination.

Introduction

The Omicron variant of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) [Phylogenetic Assignment of Named Global Outbreak (Pango) lineage designation B.1.1.529] was first detected in South Africa in November 2021. Since then, the variant spread rapidly across countries and has largely replaced all other variants globally [>99.8% of all sequences submitted to the Global Initiative on Sharing All Influenza Data (GISAID) were Omicron in week 5 of 2022] (1). Evidence suggests that the Omicron variant has a growth rate advantage compared to the previously dominant Delta variant, leading to its overtake as the dominant variant globally, while the Delta variant now only represents 0.1% of the collected samples (1).

High rates of asymptomatic infection and symptomatic COVID-19 disease among people previously infected with SARS-CoV-2 or fully vaccinated with a COVID-19 vaccine raise concerns that the currently available vaccines are less or no longer effective against the Omicron variant. To summarize the existing evidence on the effectiveness and the duration of protection conferred by COVID-19 vaccines licensed in the European Union (EU) with respect to the Omicron variant and compare it to the Delta variant, we synthesized the evidence within an ongoing living systematic review (LSR), conducted by the Robert Koch Institute (RKI) in collaboration with the National Immunisation Technical Advisory Groups (NITAGs) network coordinated by the European Centre for Disease Prevention and Control (ECDC) (2).

Methods

Literature search

The LSR follows the Preferred Reporting Items for Systematic Review and Meta-analysis (PRISMA) guidelines (Supplementary Material 1, Part 1) and was registered in the Prospective Register of Systematic Reviews (PROSPERO registration ID no. CRD42020208935; updated on 9 March 2022). All amendments since the initial registration are available online. The methods have been previously described in detail (2). In brief, we included studies of any design that had a comparison group and investigated vaccine effectiveness (VE) against SARS-CoV-2 infection of any severity of COVID-19 vaccines approved by the European Medicine Agency (EMA) in people ≥12 years of age (see Supplementary Material 1, Part 2, for complete PICO question). For this third update of the LSR, we only considered studies that reported on outcomes that are due to the SARS-CoV-2 Omicron variant or that occurred during a dominant circulation of the Omicron variant. There were no restrictions with regard to publication language or status.

We searched the COVID-19 literature database constructed by the RKI library (see Supplementary Material 1, Part 3, for the description of the database and the search strategy) for studies published between 23 October 2021 and 14 January 2022. We added literature identified by a hand search of websites and metasearch engines, indicated in Supplementary Material 1, Part 3, up to 11 February 2022. Potentially relevant publications were screened by examination of the title/abstract and full-text level by pairs of independent investigators (AP, SK, MB, LSD, VP, and/or WKS).

Data extraction

Data from the included studies (see PROSPERO registration for details) were extracted in duplicate and summarized in tables (AP, MB, VP, and WKS). We considered VE data on the following three comparisons by vaccine type (mRNA-based, vector-based, heterologous scheme, any vaccine): (1) full primary immunization vs. no vaccination (i.e., placebo, no vaccination, or vaccine not directed against COVID-19), (2) booster immunization vs. no vaccination, and (3) booster immunization vs. full primary immunization. We used the term “full primary immunization” to indicate a completed primary vaccination series, as defined by the study itself. Booster immunization indicates the vaccine dose following the full primary immunization. Outcomes of interest were VE against polymerase chain reaction- (PCR) or antigen-test confirmed SARS-CoV-2 infection of “any type” (i.e., studies did not indicate underlying symptoms), “symptomatic COVID-19,” and “severe COVID-19” (including hospitalization, ICU admission, or death due to SARS-CoV-2 infection). To investigate VE at different time points since vaccination, data were stratified into four time periods (≈14 days, >14 days up to 3 months, >3 months up to 6 months, and >6 months).

Data synthesis

As there was substantial heterogeneity across studies, we abstained from conducting a meta-analysis but summarized the studies as follows: we assessed the range of VE of any mRNA- or vector-based vaccines or for heterologous schedules against the outcomes described for the different time strata by including VE estimates from all studies that provide VE estimates for the respective time point (Table 1). If studies reported VE data for more granular time points within the defined time stratum, it was always the estimate of the latest time point within the stratum that contributed to the indicated range (e.g., if studies reported VE after 2–4 and 5–9 weeks after vaccination, the estimate of 5–9 weeks was included in the depicted effect range). VE estimates closest to 14 days were included only for the time stratum of “≈14 days”. The latter stratum included further VE data that were only assessed at “≥14 days” after full primary vaccination, “≥7 days” post-booster vaccination, or when the time point of assessment after full primary immunization was not reported at all. To visualize VE over time after full primary and booster vaccination, the estimates contributing to the VE range for the respective time category were included in forest plots. Additionally, we assessed the percentage difference of VE over time in studies reporting VE estimates for at least two different time points. We provide the range of observed minimal and maximal differences across studies. For more detailed information, please refer to the table of all extracted information provided in Supplementary Material 2.

Risk of bias and quality assessments

ROBINS-I was used to assess the risk of bias (26). The certainty of the evidence included in the LSR was rated using the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach (27, 28).

TABLE 1

Table 1 Effectiveness of COVID-19 vaccines against SARS-CoV-2 Omicron variant infection (infection (any type), symptomatic COVID-19, and severe COVID-19 (hospitalization, ICU admission, or death)).

Results

Study Screening

We identified a total of 8,428 entries in the database, until 14 January 2022. Another 38 potentially relevant studies were added by hand search up to 11 February 2022. After title/abstract and full-text screening, data from 26 studies (3–25, 29–31) were extracted (Supplementary Material 1, Part 4, Figure 1). If studies referred to previously published references for further information, these references were considered and information extracted if necessary. For pre-print studies with several versions available, the most recent update published until 11 February 2022 was included.

FIGURE 1

Figure 1 PRISMA flow diagram.

Characteristics of included studies

The included studies, of which most (21/26) were not yet peer-reviewed, reported VE estimates against infections (and related outcomes) with only the Omicron variant (5/26) or included VE data against infections with the Delta variant for comparison (21/26). Fourteen studies assessed VE using a test-negative design, eight based on a cohort design, three on a case–control design, and one study reported infection rates per vaccination status within a transmission study.

In the studies, the SARS-CoV-2 variant causing the investigated outcome was identified either from time periods with a known dominant circulation of the Omicron (resp. Delta) variant in the corresponding study location, from whole-genome sequencing (WGS), or through S-gene target failure (SGTF) in PCR assays. Studies using the latter method subdivided the SARS-CoV-2 positive samples by the detection or non-detection of the S-gene in a three gene PCR assay. As the SGTF is characteristic for the Omicron variant but rare in the Delta variant, studies used the detection of the S-gene as a proxy for the Delta variant and the “non-detection” as a proxy for the Omicron variant. As most studies report overall VE estimates without further differentiating between WGS- or SGTF-variant assessment, the data were synthesized only into “dominance” and “sequencing/SGTF.”

Studies were conducted in 10 different countries and mainly used national electronic registries or claimed data from the general population for laboratory, immunization, and patient characteristics. Only three studies investigated VE for specific populations such as healthcare workers, veterans, or patients under hemodialysis therapy. The minimum age of included study participants was 12 years (if reported). However, none of the studies provided subgroup data for children or adolescents.

Twenty-two studies reported VE estimates for full primary immunization (as defined per study) and 23 for a booster dose (additional dose after full primary immunization). One of the latter studies compared the effect of a second booster dose (in the study referring to a fourth dose of a mRNA vaccine) with a single booster dose (third dose of a mRNA vaccine). VE estimates for several time points after full primary immunization have been reported by 11 studies, and 8 studies reported VE data for several time points after booster immunization.

Most studies investigated VE of mRNA-based vaccines (13 on Comirnaty and 9 on Spikevax), seven studies reported VE for vector-based vaccines (four on Vaxzevria and three on COVID-19 vaccines from Janssen), and seven studies did not differentiate VE per vaccine. None of the studies provided data for Nuvaxovid.

Prevention of infection with SARS-CoV-2 Omicron variant (without differentiation between asymptomatic or symptomatic cases)

Twelve studies (including between 1,220 and 2.2 million participants) reported the effectiveness of COVID-19 vaccines in preventing infection of any type with SARS-CoV-2 Omicron variant (without differentiation between asymptomatic or symptomatic cases). After full primary immunization, VE across all studies ranged between 0% and 62% at “≈14 days” post-vaccination, compared to no vaccination. For the time periods of “>14 days up to 3 months,” “>3 months up to 6 months,” and “>6 months” after vaccination, VE ranges, assessed across all reported VE estimates, were 4.2%–42.8%, 0%–23%, and 0%–8.6%, respectively (Table 1, Figure 2). The data from the two studies reporting VE for at least two time points show a decline of VE between >14 days to up to 6 months by 16%–34% after vaccination with mRNA-based vaccines (Figure 2) (3, 4). As there was no VE data over time identified for vector-based vaccines, unspecified vaccines, or heterologous schedules, the respective VE decline could not be assessed.

FIGURE 2

Figure 2 Vaccine effectiveness estimates against SARS-CoV-2 infection (any type) of the Omicron variant after full primary immunization and booster dose, as reported in the study for the defined time strata after immunization. For booster immunization the COVID-19 vaccine used for primary immunization is indicated.

After booster vaccination, VE estimates at “≈14 days” post-vaccination across all studies ranged between 34% and 76%, compared to no vaccination, and between 14% and 53%, compared to full primary immunization (9, 11). Follow-up data were insufficient to evaluate the waning of immunity after booster vaccination.

The study investigating the VE of four vs. three doses of mRNA-based vaccines reported a VE estimate of 47% (95% CI, 44–47) against Omicron infection at 12 or more days after the fourth dose (29). VE ranges against infection with the Delta variant for the different time points are provided in Supplementary Material 1, Part 5.

To our knowledge, no study provided VE estimates for the direct comparison of different vaccine formulations. When comparing provided estimates across studies, the VE of mRNA-based vaccines is suggested to be higher than that of vector-based vaccines [16% (0–37) to 55.2% (23.5–73.7), respectively, −4% (−97−43) to 11.4 (NR to NR)] at ...≈14 days after full primary immunization. For later time points and booster immunization, VE could not be compared as data for vector-based vaccines were not available.

The risk of bias was serious to critical for all assessed studies (see Figure 3). A key concern was no or insufficient adjustment for confounders.

Prevention of symptomatic COVID-19

Seven studies, including between 430 and 2.2 million participants, estimated the effectiveness of COVID-19 vaccines in preventing symptomatic infection with the SARS-CoV-2 Omicron variant (Supplementary Material 1, Part 4). Compared to unvaccinated individuals, VE at ∼14 days after full primary immunization summarized from all studies reporting on this time point ranged between 6% and 76%. For the time period of “>14 days up to 3 months,” “>3 months up to 6 months,” and “>6 months,” VE ranges were 12%–54%, 6.1%–20.8%, and 0%–13%, respectively. VE estimates from studies that report data for at least two time points suggest a decrease in protective immunity against symptomatic COVID-19 between 45% and 63% for mRNA-based vaccine recipients and 50% for vector-based vaccine recipients over the time period of >14 days up to >6 months after full primary immunization (Figure 4) (13, 14).

FIGURE 3

Figure 3 Risk of Bias assessments for the outcome "SARS-CoV-2 infection (any type)".

After booster vaccination, VE against symptomatic infection ranged between 19% and 73.9% at “≈14 days” post-vaccination, compared to no vaccination, and between 50% and 68%, compared to full primary immunization (12, 16). For the time point of “>14 days up to 3 months,” estimates were only reported for the comparison with no vaccination and ranged between 43.7% and 65.4%. No study reported VE for later observation periods. The two studies that provide data for both these time periods show a 9%–28% decrease in VE after booster vaccination with an mRNA-based vaccine and between 32% and 34% after heterologous vaccination schemes (Figure 4) (13, 14). However, 95% confidence intervals between observation periods were partially overlapping.

To our knowledge, no study provided VE estimates directly comparing different vaccine formulations. Of the included studies, only a few studies investigated the VE of vector-based vaccines against symptomatic diseases. Based on these, VE against symptomatic COVID-19 compared across studies is suggested to be higher for mRNA-based vaccines at the time points of “≈14 days,” “>14 days up to 3 months,” and “>3 months up to 6 months” after full primary immunization and at “≈14 days” after booster immunization (Table 1).

The risk of bias was serious to critical for all but one of the assessed studies (see Figure 5). The remaining study was rated to have a moderate risk. All factors considered relevant for confounding were taken into account; however, residual confounding could not be ruled out.

Prevention of severe COVID-19 (hospitalization, ICU admission, or death)

VE against severe COVID-19 was assessed in 17 studies including 1,220 to 2.2 million participants (Supplementary Material 1, Part 4). VE estimates in full primary vaccinated compared to unvaccinated individuals ranged from 3% to 84% at “≈14 days” post-vaccination, between 21% and 95% at “>14 days up to 3 months,” between 0% and 91% at “>3 months up to 6 months,” and between 32.7% and 86% at “>6 months” after vaccination. Studies reporting VE for at least two time points indicate a decline by up to 40% for mRNA-based vaccines and 15%–67% for vector-based vaccines between 14 days and ≥6 months after vaccination (13, 22). However, 95% confidence intervals were wide and overlapping across time points. One study reported no difference in the first and last time point estimate (20), and another study reported a small non-significant increase [VE at 30–180 days: 73.7% (95% CI, 46.8–87); VE at ≥210 days: 80.7% (95% CI, 71.3–87) (14)] (see Figure 6, Supplementary Material 2).

FIGURE 4

Figure 4 Vaccine effectiveness estimates against symptomatic COVID-19 due to SARS-CoV-2 infection of the Omicron variant after full primary immunization and booster dose, as reported in the study for the defined time strata after immunization. For booster immunization the COVID-19 vaccine used for primary immunization is indicated.

FIGURE 5

Figure 5 Risk of Bias assessments for the outcome "symptomatic COVID-19" due to SARS-CoV-2 infection of the Omicron variant.

FIGURE 6

Figure 6 Vaccine effectiveness estimates against severe COVID-19 (incl. hospitalization, ICU admission or death) due to SARS-CoV-2 infection of the Omicron variant after full primary immunization and booster dose, as reported in the study for the defined time strata after immunization. For booster immunization the COVID-19 vaccine used for primary immunization is indicated.

FIGURE 7

Figure 7 Risk of bias assessments for the outcome "severe COVID-19" due to SARS-CoV-2 infection of the Omicron variant.

For booster vaccinated compared to unvaccinated individuals, VE ranged from 12% to 100% at “≈14 days” post-vaccination and between 78% and 93.7% at “>14 days up to 3 months”. According to three studies, VE after booster vaccination remained stable over this time period (14, 20, 22), irrespective of the vaccine or scheme used (mRNA-, vector-based, or heterologous vaccination). Only one study showed a decline of VE of approximately 12% over the respective time period after mRNA-based vaccination (13). VE data was not available for later time periods.

For the comparison against full primary immunization, we calculated the VE of booster vaccination against severe disease based on data from one study (12) that only reported data for 7 days after booster vaccination [VE: 100% (95% CI, 71.4–100)].

When comparing VE data of different vaccine formulations against severe COVID-19 across studies, no substantial differences were identified. At ∼14 days after full primary immunization, the VE of mRNA-based vaccines was slightly lower than the VE of vector-based vaccines; at the time points of “>3 months to 6 months” and “>6 months” after full primary immunization, the VE of mRNA-based vaccines was slightly higher than VE of vector-based vaccines at respective time points. VE estimates after booster immunization were similar for both vaccine formulations at the time points of “≈14 days” and “>14 days up to 3 months.” To our knowledge, no study provided VE estimates directly comparing different vaccine formulations.

The study assessing the effect of a fourth vs. a third mRNA-based vaccine dose reported VE against severe disease of 75% (95% CI, 57–86) at ≥12 days after additional booster vaccination (29). Data to assess the duration of protection were not available.

In all but one studies, the risk of bias was serious to critical (see Figure 7). The remaining study was rated to have a moderate risk due to the potential for residual confounding.

Effectiveness against Omicron variant, compared to Delta variant

Twenty-one studies reported VE estimates both against infections with the Omicron and with the Delta variant (Supplementary Material 1, Part 4).

In addition to our PICO question, we included eight studies that estimated the risk reduction of infections with the Omicron variant compared to those with the Delta variant (Supplementary Material 1, Part 6). Risk ratios from these studies suggest that the risk for any type of SARS-CoV-2 infection and symptomatic COVID-19 despite vaccination is higher for the Omicron variant than for the Delta variant. Instead, the risk of severe disease is lower in vaccinated people infected with the Omicron variant compared to those vaccinated and infected with the Delta variant (Supplementary Material 1, Part 7).

Publication bias

Potential publication bias could not be explored through statistical testing and generating funnel plots, as none of the comparisons/outcomes/time points involved sufficient studies.

GRADE

Overall, the GRADE certainty of the evidence is very low for all outcomes due to the underlying study limitations and serious heterogeneity.

Discussion

This third update of our LSR provides evidence on VE and duration of protection of EU-approved COVID-19 vaccines against any type of infection, symptomatic infection, and COVID-19-associated severe disease (i.e., hospitalization, ICU admission, or death) caused by the SARS-CoV-2 Omicron variant.

Although evidence is uncertain about the exact level of protection against all investigated outcomes, both after full primary and after booster vaccination, data suggest that the VE was higher after booster immunization when compared to full primary immunization. Results suggest a rapid decline in vaccine-induced protection after the completion of the full primary vaccination series. The effect is profound for infections of any type but less pronounced for severe diseases. This is in line with the findings of a recently published meta-regression analysis on pre-Omicron variants (32). VE could be restored to high levels of protection by the booster dose, although the first follow-up data also suggest a waning effect (13, 14, 21). However, VE against severe disease caused by Omicron remained high for at least 3 months post-booster immunization (no data for longer follow-up available). Compared to VE against the Delta variant, vaccines were less effective for all reported outcomes.

In light of the more transmissible Omicron variant and the advancing immunization campaigns across countries, evidence of the need and the timing of booster vaccination is of increasing public health interest. Thus, we decided to adapt our PICO questions and inclusion criteria and considered also studies that compare booster-vaccinated individuals with primary-vaccinated ones. The effect of waning immunity was assessed by stratifying VE data into multiple observation periods.

As included studies were highly heterogeneous (e.g., in terms of analyzed time points after vaccination, study population, applied vaccine schedules, if reported at all) and had a serious to critical risk of bias, we decided not to perform meta-analyses but provide ranges of reported VE estimates across studies to increase transparency and prevent misinterpretation. Due to the heterogeneity of the studies, the VE ranges provided for all of the outcomes are very wide, even when disaggregated by vaccine type. This is especially true for the VE estimates comparing full primary immunization with unvaccinated, which may mainly be the result of differences among the study populations, who were not accounted for in the adjustment measures, e.g., level of naturally acquired immunity, underlying comorbidities, and/or socio-economic parameters. The studies adjusted for very different confounders, but none adjusted for all variables the study team considered relevant. For any type of infection including symptomatic COVID-19, the study team considers age, sex, region, time, underlying comorbidities, previous SARS-CoV-2 infection, occupation, previous influenza vaccination, nationality, ethnicity, and socioeconomic status important confounders, whereas for severe COVID-19, age, sex, time, underlying comorbidities, previous COVID-19 treatment, duration since symptom onset, region, and previous SARS-CoV-2 infection are considered important confounders.

Recent in vitro data suggest that substrains of the Omicron variant “escape” the immune response induced by immunization in different forms, possibly leading to different VE against different substrains of the Omicron variant. In mid-January 2022, an increase in substrain BA.2 was observed in most countries. Of the included studies, only one reported VE data stratified by underlying substrains BA.1 and BA.2 of the Omicron variant. These data suggest that VE against these substrains did not differ substantially [VE after 25+ weeks after two doses: BA.1, 9% (7–10%); BA.2, 13% (−26%–40%); VE after 2 weeks after booster immunization: BA.1, 63% (63%–64%); BA.2, 70% (58%–79%) (33)]. More recent studies confirm this finding (9, 34), making it unlikely that the wide VE ranges are due to the different substrains circulating at the time when the included studies were conducted.

A standardized assessment for time points of VE evaluation as suggested by the WHO (35) and for the confounders considered would facilitate the comparison of evidence across studies and allow statistical synthesis of the evidence.

Most studies included here used the test-negative design. This study design was initially introduced to estimate VE against seasonal influenza and thought to control for differences in seeking medical care (36). In the COVID-19 era, the test-negative studies might be prone to bias caused by specific testing strategies or behaviors at the study location. However, those are mainly not reported in the studies, making it difficult to interpret reported estimates when individuals are not tested due to underlying symptoms. Most studies did not indicate the vaccination schedule applied for full primary immunization or the booster dose. For immunocompromised people, a three-dose primary vaccination schedule is recommended by the WHO Strategic Advisory Group of Experts on Immunization to improve immune response. As this recommendation is implemented in many countries¹, it is possible that VE estimates for a booster schedule include those that received third doses within an “optimized” schedule for immunocompromised people.

This third update provides a comprehensive overview of the currently available evidence on VE against infection with the SARS-CoV-2 Omicron variant. We provide an in-depth analysis of the VE estimates extracted from the included studies that were identified following a pre-registered protocol. We conducted a thorough risk of bias assessment of the studies and evaluated the certainty of evidence using the GRADE approach.

Due to the highly dynamic publication landscape in this field, additional studies have been published since our last search that are not captured by this analysis. We are aware of at least two studies reporting on adolescents, which were published after our data cut (37, 38). As in adults, lower VE against infections and severe disease was observed for the Omicron variant when compared to the Delta variant. However, data on the duration of protection are contradicting. While one study reported a decrease in VE against hospitalizations by 6%–19% at more than 5 months after full primary immunization (37), the second study did not identify a decrease in VE against any infection but was based only on few events (38). It should be noted that most studies (21/26) included in this systematic review have not yet undergone the peer-review process. This should be taken into account when interpreting the reported results. In addition to studies not included, as they were published after our final search date, we noticed that some studies updated data on pre-print servers after initial publication including data on longer follow-up periods. We therefore cannot exclude that the authors revised the available pre-print versions including additional data after we completed data extraction for this update. Furthermore, we cannot exclude a risk of potential bias as real-life observations are based on retrospective analyses that are not systematically registered. Thus, intent and possibility of publication might depend on observed results. However, due to the increase in publications on pre-print servers, the risk of publication bias is probably low.

Conclusion

Current evidence suggests that the effectiveness of COVID-19 vaccines licensed in the EU is low after full primary vaccination and improved after booster vaccination in preventing infections with the SARS-CoV-2 Omicron variant. For both full primary vaccination and booster immunization, it is characterized by a rapid decrease over time. VE against severe courses of COVID-19 remains generally high.

The studies included in this update were very heterogeneous. Therefore, the pooling of estimates was not appropriate. To allow statistical synthesis of effects, studies need to be comparable for clinical and meta-epidemiological aspects. Thus, certain standards for VE studies, as suggested by WHO, are useful to better inform vaccination guidelines and reliably assess the public health impact of vaccination campaigns.

Data availability statement

The original contributions presented in the study are included in the article/Supplementary Material. Further inquiries can be directed to the corresponding author.

Author contributions

WKS was first and second reviewer, responsible for data extraction, performed bias assessment and drafted the manuscript. VP was first and second reviewer, responsible for data extraction, performed bias assessment, conducted GRADE assessment and contributed to the manuscript. AP was first and second reviewer, responsible for data extraction, performed bias assessment and contributed to the manuscript. MB was first reviewer, responsible for data extraction, performed bias assessment and contributed to the manuscript. MW, SK, LSD, and BG were first reviewers and contributed to the manuscript. MT provided input into the interpretation of the results and developed forest plots. SVB and JK contributed to the manuscript. OW held general oversight of the conducted work and revised the manuscript. TH conceived the study and contributed to the manuscript. All authors contributed to the interpretation of the data and provided important intellectual content to the manuscript. All authors contributed to the article and approved the submitted version.

Acknowledgments

We sincerely thank the authors Andrews et al. and Kahn et al. for providing us with additional data.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fimmu.2022.940562/full#supplementary-material

Footnotes

1. ^ An additional dose is recommended for immunocompromised people in all eight countries where included studies that reported VE estimates on booster immunization were conducted (Canada, Denmark, Netherlands, Norway, Qatar, South Africa, United Kingdom, and United States of America).

References

1. World Health Organization. Weekly epidemiological update on COVID-19 - 22 march 2022 [Online] (2022). Available at: https://www.who.int/publications/m/item/weekly-epidemiological-update-on-covid-19—22-march-2022 (Accessed March 28, 2022).

Google Scholar

2. Harder T, Koch J, Vygen-Bonnet S, Kulper-Schiek W, Pilic A, Reda S, et al. Efficacy and effectiveness of COVID-19 vaccines against SARS-CoV-2 infection: interim results of a living systematic review, 1 January to 14 may 2021. Euro Surveill (2021) 64(4):383–94. doi: 10.2807/1560-7917.ES.2021.26.28.2100563

CrossRef Full Text | Google Scholar

3. Hansen CH, Schelde AB, Moustsen-Helm IR, Emborg H-D, Krause TG, Mølbak K, et al. Vaccine effectiveness against SARS-CoV-2 infection with the omicron or delta variants following a two-dose or booster BNT162b2 or mRNA-1273 vaccination series: A Danish cohort study. medRxiv (2021). doi: 10.1101/2021.12.20.21267966

CrossRef Full Text | Google Scholar

4. Tseng HF, Ackerson BK, Luo Y, Sy LS, Talarico CA, Tian Y, et al. Effectiveness of mRNA-1273 against SARS-CoV-2 omicron and delta variants. medRxiv (2022). doi: 10.1101/2022.01.07.22268919

CrossRef Full Text | Google Scholar

5. Young-Xu Y. Effectiveness of mRNA COVID-19 vaccines against omicron among veterans. medRxiv (2022). doi: 10.1101/2022.01.15.22269360

CrossRef Full Text | Google Scholar

6. Spensley K, Gleeson S, Martin P, Thomson T, Clarke CL, Pickard G, et al. Comparison of vaccine effectiveness against the omicron (B.1.1.529) variant in patients receiving haemodialysis. medRxiv (2022). doi: 10.1101/2022.01.25.22269804

CrossRef Full Text | Google Scholar

7. Willett BJ, Grove J, Maclean OA, Wilkie C, Logan N, Lorenzo GD, et al. The hyper-transmissible SARS-CoV-2 omicron variant exhibits significant antigenic change, vaccine escape and a switch in cell entry mechanism. medRxiv (2022). doi: 10.1101/2022.01.03.21268111

CrossRef Full Text | Google Scholar

8. Jalali N, Brustad HK, Frigessi A, Macdonald E, Meijerink H, Feruglio S, et al. Increased household transmission and immune escape of the SARS-CoV-2 omicron variant compared to the delta variant: evidence from Norwegian contact tracing and vaccination data. medRxiv (2022). doi: 10.1101/2022.02.07.22270437

CrossRef Full Text | Google Scholar

9. Andeweg SP, De Gier B, Eggink D, Van Den Ende C, Van Maarseveen N, Ali L, et al. Protection of COVID-19 vaccination and previous infection against omicron BA.1, BA.2 and delta SARS-CoV-2 infections. medRxiv (2022). doi: 10.1101/2022.02.06.22270457

CrossRef Full Text | Google Scholar

10. Buchan SA, Chung H, Brown KA, Austin PC, Fell DB, Gubbay JB, et al. Effectiveness of COVID-19 vaccines against omicron or delta symptomatic infection and severe outcomes. medRxiv (2022). doi: 10.1101/2021.12.30.21268565

CrossRef Full Text | Google Scholar

11. Lyngse FP, Mortensen LH, Denwood MJ, Christiansen LE, Møller CH, Skov RL, et al. SARS-CoV-2 omicron VOC transmission in Danish households. medRxiv (2021). doi: 10.1101/2021.12.27.21268278

CrossRef Full Text | Google Scholar

12. Abu-Raddad LJ, Chemaitelly H, Ayoub HH, Almukdad S, Tang P, Hasan MR, et al. Effectiveness of BNT162b2 and mRNA-1273 COVID-19 boosters against SARS-CoV-2 omicron (B.1.1.529) infection in Qatar. medRxiv (2022). doi: 10.1101/2022.01.18.22269452

CrossRef Full Text | Google Scholar

13. Uk Health Security Agency. A COVID-19 vaccine surveillance report – week 4 [Online] (2022). Available at: https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/1050721/Vaccine-surveillance-report-week-4.pdf (Accessed March 18, 2022).

Google Scholar

14. Chemaitelly H, Ayoub HH, Almukdad S, Tang P, Hasan MR, Yassine HM, et al. Duration of protection of BNT162b2 and mRNA-1273 COVID-19 vaccines against symptomatic SARS-CoV-2 omicron infection in Qatar. medRxiv (2022). doi: 10.1101/2022.02.07.22270568

CrossRef Full Text | Google Scholar

15. Nunes MC, Sibanda S, Baillie VL, Kwatra G, Aguas R, Madhi SA, et al. SARS-CoV-2 omicron symptomatic infections in previously infected or vaccinated south African healthcare workers. medRxiv (2022). doi: 10.1101/2022.02.04.22270480

CrossRef Full Text | Google Scholar

16. Accorsi EK, Britton A, Fleming-Dutra KE, Smith ZR, Shang N, Derado G, et al. Association between 3 doses of mRNA COVID-19 vaccine and symptomatic infection caused by the SARS-CoV-2 omicron and delta variants. JAMA (2022) 327:639–51. doi: 10.1001/jama.2022.0470

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Collie S, Champion J, Moultrie H, Bekker L-G, Gray G. ffectiveness of BNT162b2 vaccine against omicron variant in south Africa. New Engl J Med (2021) 386:494–6. doi: 10.1056/NEJMc2119270

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Thompson MG, Natarajan K, Irving SA, Rowley EA, Griggs EP, Gaglani M, et al. Effectiveness of a third dose of mRNA vaccines against COVID-19-Associated emergency department and urgent care encounters and hospitalizations among adults during periods of delta and omicron variant predominance - VISION network, 10 states, august 2021-January 2022. MMWR Morbidity mortality weekly Rep (2022) 71:139–45. doi: 10.15585/mmwr.mm7104e3

CrossRef Full Text | Google Scholar

19. Lewnard JA, Hong VX, Patel MM, Kahn R, Lipsitch M, Tartof SY. Clinical outcomes among patients infected with omicron (B.1.1.529) SARS-CoV-2 variant in southern California. medRxiv (2022). doi: 10.1101/2022.01.11.22269045

CrossRef Full Text | Google Scholar

20. Tartof SY, Slezak JM, Puzniak L, Hong V, Xie F, Ackerson BK, et al. BNT162b2 (Pfizer–biontech) mRNA COVID-19 vaccine against omicron-related hospital and emergency department admission in a Large US health system: A test-negative design. (2022). doi: 10.2139/ssrn.4011905

CrossRef Full Text | Google Scholar

21. Lauring AS, Tenforde MW, Chappell JD, Gaglani M, Ginde AA, Mcneal T, et al. Clinical severity and mRNA vaccine effectiveness for omicron, delta, and alpha SARS-CoV-2 variants in the united states: A prospective observational study. medRxiv (2022). doi: 10.1101/2022.02.06.22270558

CrossRef Full Text | Google Scholar

22. Nyberg T, Ferguson N, Nash S, Webster H, Flaxman S, Andrews N, et al. Comparative analysis of the risks of hospitalisation and death associated with SARS-CoV-2 omicron (B.1.1.529) and delta (B.1.617.2) variants in England. SSRN (2022). doi: 10.2139/ssrn.4025932

CrossRef Full Text | Google Scholar

23. Veneti L, Bøås H, Bråthen Kristoffersen A, Stålcrantz J, Bragstad K, Hungnes O, et al. Reduced risk of hospitalisation among reported COVID-19 cases infected with the SARS-CoV-2 omicron BA.1 variant compared with the delta variant, Norway, December 2021 to January 2022. Eurosurveillance (2022) 27:2200077. doi: 10.2807/1560-7917.ES.2022.27.4.2200077

CrossRef Full Text | Google Scholar

24. Gray GE, Collie S, Garrett N, Goga A, Champion J, Zylstra M, et al. Vaccine effectiveness against hospital admission in south African health care workers who received a homologous booster of Ad26.COV2 during an omicron COVID19 wave: Preliminary results of the sisonke 2 study. medRxiv (2021). doi: 10.1101/2021.12.28.21268436

CrossRef Full Text | Google Scholar

25. Sheikh A, Kerr S, Woolhouse M, Mcmenamin J, Robertson C. Severity of omicron variant of concern and vaccine effectiveness against symptomatic disease: national cohort with nested test negative design study in Scotland [Online] (2021). Edinburgh Research Explorer. Available at: https://www.research.ed.ac.uk/en/publications/severity-of-omicron-variant-of-concern-and-vaccine-effectiveness (Accessed February 11, 2022).

Google Scholar

26. Sterne JA, Hernán MA, Reeves BC, Savović J, Berkman ND, Viswanathan M, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ (2016) 355:i4919. doi: 10.1136/bmj.i4919

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Guyatt G, Oxman AD, Akl EA, Kunz R, Vist G, Brozek J, et al. GRADE guidelines: 1. introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol (2011) 64:383–94. doi: 10.1016/j.jclinepi.2010.04.026

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Schünemann HJ, Cuello C, Akl EA, Mustafa RA, Meerpohl JJ, Thayer K, et al. GRADE guidelines: 18. how ROBINS-I and other tools to assess risk of bias in nonrandomized studies should be used to rate the certainty of a body of evidence. J Clin Epidemiol (2019) 111:105–14. doi: 10.1016/j.jclinepi.2018.01.012

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Bar-On YM, Goldberg Y, Mandel M, Bodenheimer O, Amir O, Freedman L, et al. Protection by 4th dose of BNT162b2 against omicron in Israel. medRxiv (2022). doi: 10.1101/2022.02.01.22270232

CrossRef Full Text | Google Scholar

30. Davies M-A, Kassanjee R, Rosseau P, Morden E, Johnson L, Solomon W, et al. Outcomes of laboratory-confirmed SARS-CoV-2 infection in the omicron-driven fourth wave compared with previous waves in the Western cape province, south Africa. medRxiv (2022). doi: 10.1101/2022.01.12.22269148

CrossRef Full Text | Google Scholar

31. Kahn F, Bonander C, Moghaddassi M, Rasmussen M, Malmqvist U, Inghammar M, et al. Risk of severe COVID-19 from the delta and omicron variants in relation to vaccination status, sex, age and comorbidities – surveillance results from southern Sweden. medRxiv (2022). doi: 10.1101/2022.02.03.22270389

CrossRef Full Text | Google Scholar

32. Feikin DR, Higdon MM, Abu-Raddad LJ, Andrews N, Araos R, Goldberg Y, et al. Duration of effectiveness of vaccines against SARS-CoV-2 infection and COVID-19 disease: results of a systematic review and meta-regression. Lancet (2022) 399(10328):924–44. doi: 10.1016/S0140-6736(22)00152-0

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Uk Health Security Agency. COVID-19 vaccine surveillance report - week 4. London, United Kingdom: UK Health Security Agency (2022).

Google Scholar

34. Kirsebom FCM, Andrews N, Stowe J, Toffa S, Sachdeva R, Gallagher E, et al. COVID-19 vaccine effectiveness against the omicron BA.2 variant in England. medRxiv (2022). doi: 10.1101/2022.03.22.22272691

CrossRef Full Text | Google Scholar

35. World Health Organization. Evaluation of COVID-19 vaccine effectiveness - interim guidance [Online] (2021). Available at: https://www.who.int/publications/i/item/WHO-2019-nCoV-vaccine_effectiveness-measurement-2021.1 (Accessed March 18, 2022).

Google Scholar

36. Jackson ML, Nelson JC. The test-negative design for estimating influenza vaccine effectiveness. Vaccine (2013) 31:2165–8. doi: 10.1016/j.vaccine.2013.02.053

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Klein NP, Stockwell MS, Demarco M, Gaglani M, Kharbanda AB, Irving SA, et al. Effectiveness of COVID-19 pfizer-BioNTech BNT162b2 mRNA vaccination in preventing COVID-19-Associated emergency department and urgent care encounters and hospitalizations among nonimmunocompromised children and adolescents aged 5-17 years - VISION network, 10 states, April 2021-January 2022. MMWR Morbidity mortality weekly Rep (2022) 71:352–8. doi: 10.15585/mmwr.mm7109e3

CrossRef Full Text | Google Scholar

38. Fowlkes AL, Yoon SK, Lutrick K, Gwynn L, Burns J, Grant L, et al. Effectiveness of 2-dose BNT162b2 (Pfizer BioNTech) mRNA vaccine in preventing SARS-CoV-2 infection among children aged 5-11 years and adolescents aged 12-15 years - PROTECT cohort, July 2021-February 2022. MMWR Morbidity mortality weekly Rep (2022) 71:422–8. doi: 10.15585/mmwr.mm7111e1

CrossRef Full Text | Google Scholar