Quentin Haas1,2
Nikolay Borisov1,2
David Vicente Alvarez3,4Sohrab Ferdowsi3,4
Leonhard von Meyenn1,2
Douglas Teodoro3,4Poorya Amini1,2*- 1Risklick AG, Spin-off University of Bern, Bern, Switzerland
- 2Clinical Trial Unit Bern, University of Bern, Bern, Switzerland
- 3HES-SO University of Applied Sciences and Arts Western Switzerland, Geneva, Switzerland
- 4Department of Radiology and Medical Informatics, University of Geneva, Geneva, Switzerland
The 2019 coronavirus (COVID-19) pandemic revealed the urgent need for the acceleration of vaccine development worldwide. Rapid vaccine development poses numerous risks for each category of vaccine technology. By using the Risklick artificial intelligence (AI), we estimated the risks associated with all types of COVID-19 vaccine during the early phase of vaccine development. We then performed a postmortem analysis of the probability and the impact matrix calculations by comparing the 2020 prognosis to the contemporary situation. We used the Risklick AI to evaluate the risks and their incidence associated with vaccine development in the early stage of the COVID-19 pandemic. Our analysis revealed the diversity of risks among vaccine technologies currently used by pharmaceutical companies providing vaccines. This analysis highlighted the current and future potential pitfalls connected to vaccine production during the COVID-19 pandemic. Hence, the Risklick AI appears as an essential tool in vaccine development for the treatment of COVID-19 in order to formally anticipate the risks, and increases the overall performance from the production to the distribution of the vaccines. The Risklick AI could, therefore, be extended to other fields of research and development and represent a novel opportunity in the calculation of production-associated risks.
Introduction
The 2019 coronavirus (COVID-19) pandemic revealed an urgent need for the acceleration of vaccine development worldwide (1). Within months of the start of the COVID-19 pandemic, several pharmaceutical companies announced their objective to develop vaccines against COVID-19 by using different technologies (2–4). Each of the technologies presents advantages and pitfalls regarding the development of efficient COVID-19 vaccines (5–7). However, the situation requires that all the alternatives should be explored to shorten the course of the COVID-19 pandemic (8, 9). An increased speed of development induces risks, which must be taken seriously (10, 11). Hence, we analyzed the risks of the different technologies to produce the vaccines proposed by the pharmaceutical companies providing vaccines against COVID-19. This analysis was performed using scientific publications from early 2020 and utilizing the artificial intelligence (AI)-based search engine Risklick (12, 13). By calculating the risk score using a Probability and Impact Matrix and a semi-automated AI interface, we estimated the major risks faced by the pharmaceutical companies providing the vaccines.
A year later, our risk analysis on COVID-19 vaccine development was compared to the contemporary situation. Overall, we observed that the AI-based analysis anticipated the shortage of production and distribution faced by the pharmaceutical companies providing the RNA vaccines. The AI-based analysis highlighted the other issues faced by the pharmaceutical companies providing the vaccines, such as the delayed arrival of vaccines based on technologies other than RNA. Each issue faced by the vaccine providers is, as expected in the analysis, connected to the technology of their vaccine. Hence, our analysis exposes how AI-based technologies could become essential to the future of treatment development and the anticipation of associated risks.
Methods
Data
The Risklick AI collects and updates clinical trial data from a wide variety of sources such as the Clinical Trials Registr and datasets from the WHO each day. The metadata related to the publications are acquired from 14 international sources [BioRxiv; MedRxiv; Medline; Embase; Pubmed; Cinahl; Web of Science; Scopus; Cochrane; the International Clinical Trials Registry Platform (ICTRP); Dimensions; Living Evidence; Kaggle Cord-19 Dataset; and Google Scholar] allowing superior search performance relative to other acknowledged scientific publications search engines (12, 13). The Risklick AI is able to find and process COVID-19 references more effectively in terms of precision, F1 score, and recall, compared to the baseline platforms. Additionally, this living evidence management tool processes data in an automated manner, allowing analysis to be performed in a short amount of time. In total, 34 publications providing risk analysis on different vaccine technologies were gathered on June 10, 2020 (Supplementary Figure 1). These publications were used to evaluate the different risks in the development of the different vaccine technologies against COVID-19. The Risklick AI is available at www.risklick.ch.
Selection of Variables
In this study, seven different types of vaccines were investigated: DNA and RNA based, inactivated virus, virus-like particle, live attenuated virus, protein subunit, and viral vector. The risks associated with vaccine production were then measured across seven categories: safety, cost of goods sold, manufacturing scalability, manufacturing process, shipping, and duration of immunity. Each category was defined in Supplementary Table 2.
Risk Analysis
Using the set of publications, the risk for each vaccine technology for all the individual categories of risk was measured by using the Risklick semi-automated interface allowing the Probability and Impact Matrix calculation (14, 15). The probability of the risk happening (or “likelihood”) was categorized as “remote, unlikely, possible, likely, or certain” for each vaccine technology. These likelihood marks graded risk from lowest risk (remote) to highest risk (certain). Each of the grades was attributed a numeric value from one to five. Then, the potential impact of such a risk happening for each vaccine technology was categorized as “insignificant, minor, moderate, major, or critical.” These impact marks graded impact from lowest (insignificant) to highest (critical). Each of the grades was attributed a numeric value rising from one to five. The risk score of all the individual risk categories in each vaccine technology was calculated with the likelihood multiplied by the impact score. Visualization of risks per technology was performed by using radar charts, where all the risks were represented in the percentage of maximum risk per category based on the scores from Table 1. Risks were first analyzed on July 28, 2020, and compared with the situation a year later on May 22, 2021.
Table 1. Risk score of the different categories of risk for each vaccine technology.
Validation
Verification and validation procedures were performed by two independent immunologists. All risk assessments, likelihood measurements, and impact estimations were analyzed and verified through non-automated input of the researchers.
Results
The COVID-19 pandemic revealed the necessity for accelerated vaccine development worldwide (3, 7). In this critical context, we performed a risk analysis on the different vaccine technologies proposed by the manufacturers against COVID-19 in the early months of 2020 by using the AI-based search engine Risklick (12, 13). We graded seven different risks across seven different vaccine technologies by using a Probability and Impact Matrix (as illustrated in Supplementary Figure 1, for the risks linked to safety for each vaccine). Then, the risk score of every risk category for each vaccine technology was calculated. Risk calculation was limited by the low number of available publications at the time. An ideal quantitative risk assessment would require more data to reduce the limitations and strengthen the conclusions of the analysis. This resulted in a table of risks for each individual vaccine technology (Table 1). In this table, we already noticed that each vaccine technology possessed strengths and weaknesses different from one another, indicating the potential critical issues for the development of future vaccines. To simplify the visualization of the risks, a resume of risks per technology was performed using radar charts (Figure 1). To allow comparison, all technologies of the vaccines were compared to virus-like particle (VLP). This choice was made in the perspective of VLP being the least probable vaccine technology to effectively reach the market against COVID-19 in the near future. There are currently only four VLP-based COVID-19 vaccines in clinical trials and even in the optimistic cases, production could not reach more than 100 million doses before 2022 (16). This was confirmed in analysis with VLP presenting a particularly high risk for the time to availability and manufacturing scalability.
Figure 1. The radar charts of the seven different vaccine technologies illustrating the major risks estimated by risk score, expressed in percentage. Risks represented are safety, cost of goods sold, manufacturing scalability, manufacturing process, time of availability, shipping, and duration of immunity. Each technology is represented in blue while being compared to virus-like particle (VLP) in orange. Risk associated to (A) DNA-Based, (B) RNA-based, (C) inactivated virus, (D) live attenuated virus, (E) protein subunit, and (F) viral vector vaccines are presented. The most important risks point toward 100, while less important risks point toward 0.
In June 2020, we already observed that the pharmaceutical companies providing vaccines would face different issues. Taken separately, DNA-based vaccines have the highest risk of providing only short-term immunity (Table 1, Figure 1). Shipping is a particular risk for the RNA-based vaccines and, to a lesser degree, viral vectors, which require extremely low temperatures. Inactivated viruses and live attenuated viruses raised concerns regarding their safety and the cost of goods sold. Protein subunits appear less risky across most categories, but the duration of immunity and time of availability still appeared as a considerable risk. Finally, the viral vectors presented serious risks connected to production, particularly for the categories of th manufacturing scalability and manufacturing process.
After 1 year, we compared our results to the contemporary situation (May 22, 2021). Presently, 18 different vaccines are approved in one or more countries (17). When selecting the vaccines with authorization in more than two countries, this number falls to 10 (three inactivated viruses, five virus vectors, and two RNA-based vectors).
First, we noticed that an extremely low number of the vaccines we classified as “high risky” managed to reach market authorization so far (these being inactivated virus, live attenuated virus, and VLP; Table 1). Moreover, based on the WHO database (18), the most widely represented technology in the clinical phase of the vaccine development is the protein subunit, which was also the least risky technology according to our analysis (Figure 1). However, none of the vaccines using this technology have reached market authorization to date. In fact, one of the major risks of the protein subunit compared to the competitors was its relatively important time to availability (Figure 1), which was much lower in the RNA-based vaccines and viral vectors. Hence, the major risk associated with the protein subunit vaccine might be the reason why vaccines based on such technology are slower to reach approval compared to vaccines based on the other technologies, despite the high number of clinical trials running using this technology at the time this article was written.
With respect to the other technologies, RNA-based vaccines were rapidly commercialized probably due to low cost and high scalability of production, which translate into an extremely low risk for the time to availability (Supplementary Figure 1). However, their high risk associated with shipping was verified when supply chains started suffering from storage temperature issues following market authorization (19). With respect to the viral vector technologies, the major risk was associated with manufacturing. This was demonstrated by the shortfall of the Oxford/AstraZeneca doses in Europe for 2021 (20) and Sputnik in Latin America (21). Hence, the major risk calculated in 2020 appeared to play a critical role in the deployment of the viral vector vaccines today as it did for the RNA vaccines. With respect to vaccine distribution, the early stage of vaccine distribution in 2021 highlighted difficulties with respect to the cold chain distribution process (22). This situation provoked the evident risks of supply chain failures and caused an important pressure on vaccine supply chain infrastructures (23, 24). Such risks will need to be addressed in the future, particularly from the perspective of mRNA vaccine distribution (25). Altogether, the risk analysis performed using the Risklick AI in early 2020 offered an accurate prediction of risks that would be faced by the different pharmaceutical companies providing vaccines in the year 2021.
Discussion
The COVID-19 outbreak resulted in one of the biggest waves of publications in the history of modern science (26, 27). This influx of COVID-19-related data made literature retrieval and monitoring one of the greatest challenges of the pandemic (28). The COVID-19 pandemic highlighted the need for vaccine developers and health authorities to provide general guidance for the faster development and preparation of vaccines while offering the highest safety and efficacy standards (1, 29). Shortly after the start of the COVID-19 pandemic, several pharmaceutical companies providing vaccines announced their objective to develop vaccines against COVID-19, using different technologies, all within a matter of months (2, 3).
The choice of the type of vaccine is crucial: carrier or vector, adjuvant, excipients, dosage form, and route of administration; all directly impact the potential efficacy of the vaccine against COVID-19, but also the logistics of manufacturing, storing, and distributing the vaccine might affect the availability of the product (5, 6, 9). Hence, this simple variation in the composition of the vaccines will directly impact the policy of mass vaccination and the access to vaccine doses worldwide, affecting the duration of the COVID-19 pandemic (8).
In this context, we performed a risk analysis on the different technologies used by the pharmaceutical companies providing vaccines against COVID-19 in the early months of 2020 by using the AI-based search engine Risklick (12, 13).
Our results demonstrate that the AI-based analysis of risks allows the identification of the vaccine category with the highest chances of success. Moreover, the Risklick AI allowed us to foresee the different issues pharmaceutical companies providing vaccines would face, notably during the production and distribution phases of their product.
Our results highlight how AI-based technologies will become essential in the development of various therapies in the future. Moreover, our results demonstrate that, by using in-depth risk analysis, the global performance of vaccine developers could be increased by anticipating major issues, allowing corrective, and preventive actions. Such an approach, if generalized, would represent an important gain of time and efficiency for every scientist and manufacturer involved in providing care and drugs. The adoption of detailed risk analysis by all the scientists and manufacturers involved in the COVID-19 pandemic could represent a game-changer that could positively impact the future of therapy development worldwide.
Data Availability Statement
The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author/s.
Author Contributions
PA designed the study. QH and NB wrote the manuscript. QH, PA, and LM performed the experimental work. DA, SF, NB, and DT developed the AI models. All authors had full access to the data, helped draft the report or critically revised the draft, contributed to data interpretation, reviewed, and approved the final version of the report.
Funding
This study was funded by the Innosuisse project funding number 41013.1 IP-ICT.
Conflict of Interest
QH, NB, LM, and PA are work for Risklick.
The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher's Note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Supplementary Material
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fdgth.2021.745674/full#supplementary-material
Supplementary Figure 1. The Probability and Impact Matrix for the safety of the six different vaccine technologies.
Supplementary Table 1. List of publications used for the semi-automated artificial intelligence (AI) risk calculation.
Supplementary Table 2. Definition of the variables used to assess the risk of vaccine technologies.
References
1. Thiel N, Selwyn C, Murphy G, Simpson S, Chakrabarti AC. Recommendations for acceleration of vaccine development and emergency use filings for COVID-19 leveraging lessons from the novel oral polio vaccine. npj Vaccines. (2021) 6:63. doi: 10.1038/s41541-021-00325-4
2. Izda V, Jeffries MA, Sawalha AH. COVID-19: a review of therapeutic strategies and vaccine candidates. Clin Immunol. (2021) 222:108634. doi: 10.1016/j.clim.2020.108634
3. Russell RL, Pelka P, Mark BL. Frontrunners in the race to develop a sars-cov-2 vaccine. Can J Microbiol. (2021) 67:189–212. doi: 10.1139/cjm-2020-0465
4. Amanat F, Krammer F. SARS-CoV-2 vaccines: status report. Immunity. (2020) 52:583–9. doi: 10.1016/j.immuni.2020.03.007
5. Wang J, Peng Y, Xu H, Cui Z, Williams RO. The COVID-19 vaccine race: challenges and opportunities in vaccine formulation. AAPS PharmSciTech. (2020) 21:225. doi: 10.1208/s12249-020-01744-7
6. Haynes BF, Corey L, Fernandes P, Gilbert PB, Hotez PJ, Rao S, et al. Prospects for a safe COVID-19 vaccine. Sci Transl Med. (2020) 12:eabe0948. doi: 10.1126/scitranslmed.abe0948
7. Mullard A. COVID-19 vaccine development pipeline gears up. Lancet. (2020) 395:1751–2. doi: 10.1016/S0140-6736(20)31252-6
8. Rossman H, Shilo S, Meir T, Gorfine M, Shalit U, Segal E. COVID-19 dynamics after a national immunization program in Israel. Nat Med. (2021) 27:1055–61. doi: 10.1038/s41591-021-01337-2
9. Singh JA, Upshur REG. The granting of emergency use designation to COVID-19 candidate vaccines: implications for COVID-19 vaccine trials. Lancet Infect Dis. (2021) 21:e103–9. doi: 10.1016/S1473-3099(20)30923-3
10. Rawat K, Kumari P, Saha L. COVID-19 vaccine: a recent update in pipeline vaccines, their design and development strategies. Eur J Pharmacol. (2021) 892:173751. doi: 10.1016/j.ejphar.2020.173751
11. Iserson K V. Sars-cov-2 (covid-19) vaccine development and production: an ethical way forward. Cambridge Q Healthc Ethics. (2020) 30:59–68. doi: 10.1017/S096318012000047X
12. Haas Q, Alvarez DV, Borissov N, Ferdowsi S, Von Meyenn L, Trelle S, et al. Utilizing artificial intelligence to manage COVID-19 scientific evidence torrent with risklick ai: a critical tool for pharmacology and therapy development. Pharmacology. (2021) 106:244–53. doi: 10.1159/000515908
13. Teodoro D, Ferdowsi S, Borissov N, Kashani E, Vicente Alvarez D, Copara J, Gouareb R, Naderi N, Amini P. Information Retrieval in an Infodemic: The Case of COVID-19 Publications. J Med Internet Res. (2021) 23:e30161. doi: 10.2196/30161
14. Dumbravă V, Iacob V-S. Using probability – impact matrix in analysis and risk assessment projects. J Knowl Manag Econ Inf Technol. (2013) 3:1–7.
15. Sohrab Ferdowsi, Nikolay Borissov, Julien Knafou, Poorya Amini, Douglas Teodoro. Classification of hierarchical text using geometric deep learning: the case of clinical trials corpus. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. (2021).
16. Nooraei S, Bahrulolum H, Hoseini ZS, Katalani C, Hajizade A, Easton AJ, et al. Virus-like particles: preparation, immunogenicity and their roles as nanovaccines and drug nanocarriers. J Nanobiotechnol. (2021) 19:1–27. doi: 10.1186/s12951-021-00806-7
17. Vaccines – COVID19 Vaccine Tracker. Available online at: https://covid19.trackvaccines.org/vaccines/ (accessed September 26, 2021)
18. COVID-19 Vaccine Tracker and Landscape. Available online at: https://www.who.int/publications/m/item/draft-landscape-of-covid-19-candidate-vaccines (accessed September 26, 2021)
19. Sagili Anthony DP, Sivakumar K, Venugopal P, Sriram DK, George M. Can mRNA vaccines turn the tables during the COVID-19 pandemic? Current status and challenges. Clin Drug Investig. (2021) 41:499–509. doi: 10.1007/s40261-021-01022-9
20. Update on EU Supply of COVID-19 Vaccine AstraZeneca. Available online at: https://www.astrazeneca.com/media-centre/articles/2021/update-on-eu-supply-of-covid-19-vaccine-astrazeneca.html (accessed September 26, 2021)
21. The Moscow Times. Argentina, Mexico Face Second-Dose Shortages of Russia's Sputnik V Vaccine – Reports - The Moscow Times. Available online at: https://www.themoscowtimes.com/2021/06/21/argentina-mexico-face-second-dose-shortages-of-russias-sputnik-v-vaccine-reports-a74279 (accessed September 26, 2021)
22. Carneiro DC, Sousa JD, Monteiro-Cunha JP. The COVID-19 vaccine development: a pandemic paradigm. Virus Res. (2021) 301:198454. doi: 10.1016/j.virusres.2021.198454
23. Radanliev P, De Roure D, Ani U, Carvalho G. The ethics of shared Covid-19 risks: an epistemological framework for ethical health technology assessment of risk in vaccine supply chain infrastructures. Heal Technol. (2021) 11, p.1083–1091. doi: 10.1007/s12553-021-00565-3
24. Grau S, Ferrández O, Martín-García E, Maldonado R. Accidental interruption of the cold chain for the preservation of the moderna COVID-19 vaccine. Vaccines. (2021) 9:512. doi: 10.3390/vaccines9050512
25. Crommelin DJA, Anchordoquy TJ, Volkin DB, Jiskoot W, Mastrobattista E. Addressing the cold reality of mRNA vaccine stability. J Pharm Sci. (2021) 110:997–1001. doi: 10.1016/j.xphs.2020.12.006
26. Palayew A, Norgaard O, Safreed-Harmon K, Andersen TH, Rasmussen LN, Lazarus J V. Pandemic publishing poses a new COVID-19 challenge. Nat Hum Behav. (2020) 4:666–9. doi: 10.1038/s41562-020-0911-0
27. Callaway E. Will the pandemic permanently alter scientific publishing? Nature. (2020) 582:167–8. doi: 10.1038/d41586-020-01520-4
28. Muhammad LJ, Islam MM, Usman SS, Ayon SI. Predictive data mining models for novel coronavirus (COVID-19) infected patients' recovery. SN Comput Sci. (2020) 1:206. doi: 10.1007/s42979-020-00216-w
Keywords: artificial intelligence, pharmacology, COVID-19, vaccine, risk analysis
Citation: Haas Q, Borisov N, Alvarez DV, Ferdowsi S, von Meyenn L, Teodoro D and Amini P (2021) Vaccine Development in the Time of COVID-19: The Relevance of the Risklick AI to Assist in Risk Assessment and Optimize Performance. Front. Digit. Health 3:745674. doi: 10.3389/fdgth.2021.745674
Received: 22 July 2021; Accepted: 24 September 2021;
Published: 02 November 2021.
Edited by:
Giovanni Ferrara, University of Alberta, CanadaReviewed by:
Petar Radanliev, University of Oxford, United KingdomFarid Rahimi, Australian National University, Australia
Copyright © 2021 Haas, Borisov, Alvarez, Ferdowsi, von Meyenn, Teodoro and Amini. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Poorya Amini, cG9vcnlhLmFtaW5pJiN4MDAwNDA7Y3R1LnVuaWJlLmNo