Is the Harm-to-Benefit Ratio a Key Criterion in Vaccine Approval?

For the individual wellbeing or the therapy of an illness, a medical treatment should be more beneficial than harmful regarding possible adverse events of those treated. From our medical-layman point of view, until the end of 2020, we considered this part of the Hippocratic Oath, to be accepted and implemented as a rule world-wide.

Usually, concerning medical drugs, the focus of attention is on the frequency of severe adverse, i.e., potentially harmful, events (SAE), whose occurrences range from “very rare” (< 0.01% of treatments) to “very common” (≥ 10%) (1) . Yet, the essentials of drugs or vaccines are curing or preventing, respectively, a potentially severe illness, which would either be potentially beneficial. Apparently, it is uncommon on a broad scale to directly, quantitatively relate the frequencies of competing harmful and beneficial events; rather, (potentially rare) SAE and (potentially high) efficacies are communicated separately. Levels of SAE frequencies that might well be acceptable in administering medical drugs to cure acute illness, may not at all be acceptable for vaccines, which are administered preventively to healthy subjects. Here, a rarely occurring SAE may indeed spoil the preventive effect, if a severe disease progression (SX) of disease X is likewise rare. When approving the mass administration of vaccines (or even make them mandatory), one hazards the consequences of SAE increasing proportionally with the doses administered. The resulting balance of these potential harms and the conceived benefits, as well as the absolute number of SAE induced, should thus be a key criterion for the decision in favor of or against the approval of vaccines.

For the time being, in this communication, we are concerned with quantifying the basic balancing relation of harms and benefits of a vaccine introduced. To this end, let N_{SAE, vac} and N_{SAE, com} be the numbers of severe adverse events in the vaccine and comparison (often: placebo) groups, respectively, as well as N_{SX, vac} and N_{SX, com} the respective numbers of severe cases of disease X. Then, ΔN_SAE and ΔN_SX symbolize the additional, as compared to the occurrences in the comparison group (index “_com”), number of SAE and the lesser number of SX cases, respectively, as a consequence of a given number of vaccine (index “_vac”) doses. If the harm-to-benefit ratio

$\begin{array}{l}hbr=\frac{N_{SAE,vac}-N_{SAE,com}}{N_{SX,com}-N_{SX,vac}}=\frac{\text{Δ}{N}_{SAE}}{\text{Δ}{N}_{SX}}=\frac{\text{Δ}{\tilde{N}}_{SAE}}{\text{Δ}{\tilde{N}}_{SX}},\end{array}$

with $\text{Δ}{\tilde{N}}_{SAE}=\frac{\text{Δ}{N}_{SAE}}{\left|N_{all}\right|}$, $\text{Δ}{\tilde{N}}_{SX}=\frac{\text{Δ}{N}_{SX}}{\left|N_{all}\right|}$, and the norm $\left|N_{all}\right|=\sqrt{{(\text{Δ}{N}_{SAE})}^2+{(\text{Δ}{N}_{SX})}^2}$, were greater than one, SAE due to the vaccine would more likely do severe harm to a treated person than the treated would benefit by being potentially saved from becoming severely ill due to disease X. Applying the same standard for hbr as for the frequency of SAE when administering medical drugs, it appears reasonable to aim at hbr < 0.1, i.e., at maximum one additional (short-term) SAE in ten prevented SX cases.

Our attention was attracted by the accumulating evidence of adverse events due to the world-wide SARS-CoV-2 vaccination campaign [e.g., (2)] being far more frequent than advertised initially, together with notes of the practically vanishing efficacy of the vaccines against infections with new variants of SARS-CoV-2 [e.g., (3)]. Further guided by work (4, 5) in which the hbr had been quantified in the field by relating mortality as a consequence of SAE presumably due to SARS-CoV-2 vaccination (Dutch national register “Lareb”) to COVID-19 deaths probably prevented by SARS-CoV-2 vaccination (in an Israel campaign), and being inherently inquisitive, we had a closer look at the safety and efficacy data published with the available (phase-3) papers (6–9) on pivotal clinical trials of mRNA/vector vaccines. Then simply picking up the documented counts of both severe adverse events (N_{SAE, vac}, N_{SAE, com}) and severe-critical COVID-19 cases (N_{SC19, vac}, N_{SC19, com}), in each the vaccine and comparison group of a trial, we could calculate the corresponding harm-to-benefit ratio $hbr=\frac{N_{SAE,vac}-N_{SAE,com}}{N_{SC19,com}-N_{SC19,vac}}$. Earlier, most of these trials had already been analyzed (10) with a similar approach; however, events of several severity degrees and sorts (adverse and COVID-19) had been summed within each the vaccination and the control group, therewith obscuring the individual event significances regarding risks and benefits in the trials.

One may ponder whether the grades of severity of, on the one hand, SAE due to SARS-CoV-2 vaccination and, on the other hand, “severe-critical COVID case” differ in quality. However, this is an issue to be debated on the public stage by clinical experts. Such had not been done within the pivotal clinical trial studies we have examined here. For initiation, an excessive written review of the scientific and clinical literature as well as the medical rules and guidelines currently in effect would be required indeed. In our present analysis, we imply that the use of one and the same adjective “severe” for rating both health issues in these pivotal trials indicates comparable grades of severity.

Drawing upon “severe-critical” to compare with “severe adverse” and accordingly calculating the severe-event harm-to-benefit ratio, we found that only two studies documented a ratio lower than one (Table 1), hbr = 0.6 or 0.9 in a particular analysis, respectively, for “Ad26.COV2.S”, as well as hbr = −0.4 for “Sputnik V”. The negative value for “Sputnik V” results, mathematically, from the number of SAE in the comparison group (receiving only the vaccine buffer composition) exceeding that in the vaccine group, which is a puzzling phenomenon, practically impossible for us to interpret with the information available from (9) . Figure 1 illustrates at a glance the studies' hbr values and their classification. It is evident that the overall short-term performance of the vaccines “BNT162b2” and “mRNA-1273”, based on the Supplementary Material published with these studies, can not be rated other than “harmful”, entirely unbalanced. “Ad26.COV2.S” performs nearly “harmful”, definitely unbalanced. A “gambling” classification is suitable, because its administration is a bet on unknown long-term harms being less likely then short-term harms already known. Astonishingly enough, the vaccine “Sputnik V” seems to perform as a “universal remedy”.

TABLE 1

Table 1. Absolute counts of severe adverse events, N_SAE, and severe cases of CoViD-19, N_SC19, each for the vaccine (index “_vac”) and the comparison (mostly: placebo) group (index “_com”), as documented by the phase-3 studies (data source); the harm-to-benefit ratio hbr resulting from these counts, and the Time after full vaccination, when the count, until the end of the study, of CoViD-19 cases began; N_t documents the number of subjects in the vaccine group.

FIGURE 1

Figure 1. Juxtaposition, within a harm-benefit diagram, of the vaccines' observed (short-term) harm-to-benefit ratios (hbr) according to Table 1. To make both juxtaposed, hbr-contributing differences, ΔN_SAE: = N_{SAE, vac}−N_{SAE, com} (harm events) and ΔN_SC19: = N_{SC19, com}−N_{SC19, vac} (benefit events), comparable, we normalized these numbers as $\text{Δ}{\tilde{N}}_{SAE}:=\frac{\text{Δ}{N}_{SAE}}{\sqrt{\text{Δ}{N}_{SAE}^2+\text{Δ}{N}_{SC19}^2}}$ and $\text{Δ}{\tilde{N}}_{SC19}:=\frac{\text{Δ}{N}_{SC19}}{\sqrt{\text{Δ}{N}_{SAE}^2+\text{Δ}{N}_{SC19}^2}}$, respectively, for then projecting them on the unit circle. Note that the hbr is uniquely represented by the angle $\phi =\mathrm{\text{atan2}}\left(\frac{\text{Δ}{N}_{SAE}}{\text{Δ}{N}_{SC19}}\right)=\mathrm{\text{atan2}}\left(\frac{\text{Δ}{\tilde{N}}_{SAE}}{\text{Δ}{\tilde{N}}_{SC19}}\right)$, with “atan2()” denoting the four-quadrant inverse tangent function (12). The red-shaded area above the “harm = benefit”, i.e., hbr = 1 line marks the harmfully unbalanced area: hbr > 1 indicates harm to exceed benefit in the corresponding studies. The green-shaded area marks what we would consider the “medically beneficial” (also: balanced or acceptable, respectively) area of hbr ≤ 0.1 (while ΔN_SC19 > ΔN_SAE), at least in the short term. The yellow-shaded area marks the remaining cases, which we would term “gambling” conditions, because they yield only just less short-term harm than benefit, and a vaccinated human bets on the unknown long-term harms being lower than those in the short term. Dark-shaded sectors around hbr pointers indicate a 10% uncertainty interval, which was constructed by disturbing numerator and denominator by 10% each and choosing the maximum and minimum resulting hbr. For the “Sputnik V” study (9) data, a (beneficial) negative hbr indicates a positive benefit alongside with a negative harm, i.e., lesser SAE in the vaccinated than in the comparison group. Note that a negative (and then harmful) hbr might also have occurred by doing harm (ΔN_SAE > 0) while being accompanied by a negative benefit (IIInd quadrant), which had luckily not been observed in these studies.

The numbers presented in our Table 1 and Figure 1 have been widely visible since their publication. We wonder why neither a reviewer nor an approving authority seems to have noticed these jaw-dropping values of harm-to-benefit ratios. Us having missed some other vaccine-beneficial rationale would be an explanation of good nature. A crucial question has thus emerged: What scientific data and criteria other than the severe-event harm-to-benefit ratio, or rationales, have the above vaccines' approvals by authorities been exactly based on?

Author Contributions

FM performed an initial data research and analysis, did early calculations, and wrote the initial manuscript draft. MG refined the calculation rationale and method and finally extracted all necessary numbers from the analyzed studies' data material. RR invented and drafted the graphical illustration of the calculation and results and finally generated the figure. MG and RR added passages and revised the initial draft. All authors conjointly converged iteratively by writing and discussing to the eventually submitted manuscript draft. All authors contributed to the article and approved the submitted version.

Funding

Publication costs were covered by the Open Access Fund of the University of Koblenz-Landau.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

1. WHO Vaccine Safety Basics. (2022). Available online at: https://vaccine-safety-training.org/frequency-and-severity.html

Google Scholar

2. Oster ME, Shay DK, Su JR, Gee J, Creech CB, Broder KR, et al. Myocarditis cases reported after mRNA-based COVID-19 vaccination in the US From December 2020 to August 2021. JAMA. (2022) 327:331–40. doi: 10.1001/jama.2021.24110

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Singanayagam A, Hakki S, Dunning J, Madon KJ, Crone MA, Koycheva A, et al. Community transmission and viral load kinetics of the SARS-CoV-2 delta (B.1.617.2) variant in vaccinated and unvaccinated individuals in the UK: a prospective, longitudinal, cohort study. Lancet Infect Dis. (2022) 22:183–95. doi: 10.1016/S1473-3099(21)00648-4

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Walach H, Klement RJ, Aukema W. The safety of COVID-19 vaccinations–should we rethink the policy? Sci Public Health Policy Law. (2021) 3:87–99. Available online at: https://www.publichealthpolicyjournal.com/_files/ugd/adf864_8c97b2396c2842b3b05975bfbd8254cb.pdf

PubMed Abstract | Google Scholar

5. Walach H, Klement RJ, Aukema W. The risk-benefit ratio of Covid-19 vaccines: Publication policy by retraction does nothing to improve it. Clin Transl Discov. (2022) 2:e35. doi: 10.1002/ctd2.35

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Polack FP, Thomas SJ, Kitchin N, Absalon J, Gurtman A, Lockhart S, et al. Safety and efficacy of the BNT162b2 mRNA Covid-19 vaccine. N Engl J Med. (2020) 383:2603–15. doi: 10.1056/NEJMoa2034577

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Baden LR, El Sahly HM, Essink B, Kotloff K, Frey S, Novak R, et al. Efficacy and safety of the mRNA-1273 SARS-CoV-2 vaccine. N Engl J Med. (2021) 384:403–16. doi: 10.1056/NEJMoa2035389

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Sadoff J, Gray G, Vandebosch A, Cárdenas V, Shukarev G, Grinsztejn B, et al. Safety and efficacy of single-dose Ad26.COV2.S vaccine against Covid-19. N Engl J Med. (2021) 384:2187–201. doi: 10.1056/NEJMoa2101544

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Logunov DY, Dolzhikova IV, Shcheblyakov DV, Tukhvatulin AI, Zubkova OV, Dzharullaeva AS, et al. Safety and efficacy of an rAd26 and rAd5 vector-based heterologous prime-boost COVID-19 vaccine: an interim analysis of a randomised controlled phase 3 trial in Russia. Lancet. (2021) 397:671–81. doi: 10.1016/S0140-6736(21)00234-8

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Classen JB. US COVID-19 vaccines proven to cause more harm than good based on pivotal clinical trial data analyzed using the proper scientific endpoint, “all cause severe morbidity”. Trends Internal Med. (2021) 1:1–6. doi: 10.33425/2771-5906.1004

CrossRef Full Text | Google Scholar

11. Voysey M, Clemens SAC, Madhi SA, Weckx LY, Folegatti PM, Aley PK, et al. Safety and efficacy of the ChAdOx1 nCoV-19 vaccine (AZD1222) against SARS-CoV-2: an interim analysis of four randomised controlled trials in Brazil, South Africa, and the UK. Lancet Infect Dis. (2021) 397:99–111. doi: 10.1016/S0140-6736(20)32661-1

PubMed Abstract | CrossRef Full Text | Google Scholar

12. The The inverse tangent function atan2() Available online at: https://en.wikipedia.org/wiki/Atan2

Google Scholar