Unveiling the future of COVID-19 patient care: groundbreaking prediction models for severe outcomes or mortality in hospitalized cases

Background: Previous studies have identified COVID-19 risk factors, such as age and chronic health conditions, linked to severe outcomes and mortality. However, accurately predicting severe illness in COVID-19 patients remains challenging, lacking precise methods.

Objective: This study aimed to leverage clinical real-world data and multiple machine-learning algorithms to formulate innovative predictive models for assessing the risk of severe outcomes or mortality in hospitalized patients with COVID-19.

Methods: Data were obtained from the Taipei Medical University Clinical Research Database (TMUCRD) including electronic health records from three Taiwanese hospitals in Taiwan. This study included patients admitted to the hospitals who received an initial diagnosis of COVID-19 between January 1, 2021, and May 31, 2022. The primary outcome was defined as the composite of severe infection, including ventilator use, intubation, ICU admission, and mortality. Secondary outcomes consisted of individual indicators. The dataset encompassed demographic data, health status, COVID-19 specifics, comorbidities, medications, and laboratory results. Two modes (full mode and simplified mode) are used; the former includes all features, and the latter only includes the 30 most important features selected based on the algorithm used by the best model in full mode. Seven machine learning was employed algorithms the performance of the models was evaluated using metrics such as the area under the receiver operating characteristic curve (AUROC), accuracy, sensitivity, and specificity.

Results: The study encompassed 22,192 eligible in-patients diagnosed with COVID-19. In the full mode, the model using the light gradient boosting machine algorithm achieved the highest AUROC value (0.939), with an accuracy of 85.5%, a sensitivity of 0.897, and a specificity of 0.853. Age, vaccination status, neutrophil count, sodium levels, and platelet count were significant features. In the simplified mode, the extreme gradient boosting algorithm yielded an AUROC of 0.935, an accuracy of 89.9%, a sensitivity of 0.843, and a specificity of 0.902.

Conclusion: This study illustrates the feasibility of constructing precise predictive models for severe outcomes or mortality in COVID-19 patients by leveraging significant predictors and advanced machine learning. These findings can aid healthcare practitioners in proactively predicting and monitoring severe outcomes or mortality among hospitalized COVID-19 patients, improving treatment and resource allocation.

Introduction

The emergence of the coronavirus disease 2019 (COVID-19) outbreak in China during late 2019 has escalated into a worldwide health apprehension, primarily due to its rapid transmission and deleterious health implications (1). Its prevalent symptoms encompass fever, dry cough, and dyspnea (2). According to prior investigations, a distinct subset of afflicted individuals faces a heightened susceptibility to severe infection, with respiratory impairments such as dyspnea, elevated respiratory rate, and diminished oxygen saturation dominating the symptomatology. Individuals with advanced disease may also manifest respiratory failure, septic shock, or multi-organ dysfunction (3).

The swift propagation and extensive ramifications of this worldwide pandemic have imposed a significant strain on healthcare systems across diverse nations. This strain is particularly evident in the realms of clinical resource allocation and decision-making protocols. Numerous medical institutions have encountered unparalleled scarcities of essential supplies, among them mechanical ventilators, primarily stemming from the rapid surge in critically ill COVID-19 patients necessitating both airway assistance and mechanical ventilatory support. This predicament, confronting healthcare delivery systems, underscores the urgency of employing innovative and pioneering technologies to navigate acute and systemic challenges in healthcare provisioning. With the overarching aims of mitigating mortality and sustaining healthcare infrastructure, the primary objective entails averting severe outcomes and fatalities among patients.

The incorporation of artificial intelligence (AI) and machine learning (ML) within the healthcare domain, spanning tasks such as image analysis, clinical decision-making, and prognosis prediction, constitutes a burgeoning discipline with broad applications across diverse maladies (4). Within the context of COVID-19, artificial intelligence has demonstrated its pivotal role in both diagnostic and prognostic domains, encompassing prediction, detection, classification, screening, and diagnosis of COVID-19 infections (5, 6). Scoping reviews have underscored the potential of artificial intelligence as a weapon in the fight against COVID-19; nonetheless, many proposed methodologies are yet to secure clinical acceptance (7). Predictive models stand as extensively investigated tools within biotechnology, enriching clinical comprehension of the diagnostic and prognostic dimensions of various illnesses.

According to the Taiwan Centers for Disease Control, during the initial phase of the COVID-19 outbreak, a substantial proportion (42%) of the cases were primarily located in the northern region of Taiwan, probably due to the presence of the International airports in that area and May 2022 marked the onset of the first wave of the pandemic (8). The Taipei Medical University Clinical Research Database (TMUCRD) gathers data from multiple centers and sources of various data types. It systematically collects both structured and unstructured data from three affiliated hospitals: Taipei Medical University Hospital, Wanfang Hospital, and Shuangho Hospital (9–11). The National Health Insurance database in Taiwan has a gap of 2 years in the dissemination of data for research purposes. Therefore, in terms of finding recent breakthroughs in the field of COVID-19, TMUCRD could help enhance the understanding of factors influencing COVID-19 outcomes.

Based on the most accurate information available, no prediction model study of COVID-19 severe symptoms in Taiwan. This study aimed to predict severe outcomes, including the use of ventilators, intubation, admission to the intensive care unit (ICU), and mortality, among COVID-19 patients hospitalized in Taiwan. The primary objective of this study is to develop predictive models that can assist clinicians in identifying individuals who are most vulnerable to severe outcomes, including mortality. This focused identification provides healthcare practitioners with the tools to carry out prompt interventions.

Methods

Study design and data source

To create the dataset, this study utilized clinical data obtained from the Taipei Medical University Clinical Research Database (TMUCRD). TMUCRD consolidates extensive clinical data derived from three associated hospitals: Taipei Medical University Hospital, Wanfang Hospital, and Shuang-Ho Hospital. The database comprises structured and unstructured information. This study obtained approval from the Taipei Medical University Joint Institutional Review Board (TMU-JIRB) with grant number N202302020.

Population selection

This study included patients who were hospitalized and confirmed to have contracted COVID-19 within the period spanning from January 1, 2021, to May 31, 2022. The diagnosis of COVID-19 was established either through a positive outcome from a real-time reverse transcription polymerase chain reaction (RT-PCR) test or a positive outcome from a rapid antigen test.

The exclusion criteria encompassed newly registered patients who had not previously sought medical care at the three hospitals due to the lack of complete medical background information records, individuals under the age of 20, and patients with undisclosed gender information. As a result, a total of 22,192 patients were retained for inclusion in this study. The selection process for the study population is visually depicted in Figure 1.

FIGURE 1

Figure 1. Flowchart of cohort selection.

Outcome measurement

The index date is defined as the date of the first COVID diagnosis. The primary outcome was defined as a serious event, encompassing occurrences such as ventilator use, intubation, intensive care unit (ICU) admission, and mortality within 3 months of confirmed COVID-19 infection. Additionally, each of the aforementioned specific indicators was considered as a secondary outcome in this study. Data censoring occurred either at the date of death, loss to follow-up, or at the end of the study (May 31, 2022).

Features

Based on a literature review and consultation with clinicians, this study identified features associated with the above outcomes based on demographic information, health status, COVID-19-related details, comorbidities, long-term medication records, and laboratory test results. The selected features include: (1) demographic information: gender and age; (2) health status: body mass index (BMI) and Charlson Comorbidity Index (CCI) score; (3) COVID-19-related details: COVID-19 vaccine and Covid-19 medications; (4) comorbidities: myocardial infarction (MI), chronic kidney disease (CKD), congestive heart failure (CHF), peripheral vascular disease, cerebrovascular disease (CVA), cardiovascular disease (CVD), dementia, chronic obstructive pulmonary disease (COPD), rheumatic disease, peptic ulcer disease, liver disease, diabetes mellitus (DM), hemiplegia, renal disease, cancer, human immunodeficiency virus/ acquired immune deficiency syndrome (HIV/AIDS), hypertension, hyperlipidemia, hyperuricemia, depression or anxiety, anemia, Parkinson’s disease (PD), osteoporosis; (5) long-term medication records: benzodiazepine (BZD), non-steroidal anti-inflammatory drug (NSAID), aspirin, hypertension (HTN) drugs, DM drugs, statins, antihyperuricemic drugs, antihistamin, gastro-oesophageal reflux disease (GORD) drugs, steroids; and (6) laboratory test results: HbA1C, total cholesterol (TC), high-density lipoprotein (HDL), low-density lipoprotein (LDL), triglycerides (TG), Uric acid (UA), aspartate aminotransferase/AST (GOT), alanine transaminase/ALT (GPT), total protein, albumin, globubin, blood urea nitrogen (BUN), creatinine, red blood cells (RBC), hemoglobin (HGB), mean corpuscular hemoglobin (MCH), mean corpuscular hemoglobin concentration (MCHC), white blood cell (WBC), neutrophil, lymphocyte, platelet count (PLT), hematocrit (HCT), sodium (NA), potassium (K), troponin I, and troponin T.

The Charlson Comorbidity Index (CCI) score was computed, and comorbidity was determined using disease codes sourced from the ICD-9 or ICD-10 classification systems found in the medical records. Among the cohort members, individuals were categorized as having comorbidities if they had undergone a minimum of two outpatient visits or one hospitalization related to the specific disease before the index date. Evaluation of the COVID-19 vaccine status is based on the vaccination records within the year preceding the index date. Assessment of COVID-19 medications is grounded in the medication status during the 3 months following the index date. Long-term medication users in the cohort were characterized as patients who had received a prescription for one or more of the aforementioned drugs for a period of 28 days or longer in the year (365 days) prior to the index date. In cases where multiple test results were obtainable, priority was given to the latest laboratory test value within a one-year period before the index date. The technique of Multiple Imputation by Chained Equations (MICE) was employed to address the presence of missing continuous features (12).

Statistical analysis

In the realm of descriptive statistics, continuous data are elucidated through the utilization of the mean (standard deviation, S.D.) and median (minimum and maximum values). Conversely, categorical data are expounded upon by presenting the count of cases along with their corresponding percentages. Additionally, the count and proportion of missing values were computed. Statistical analyses were conducted employing R version 4.1.3 (R Project for Statistical Computing).

Algorithms used in this study

Seven machine learning algorithms were utilized to formulate personalized prediction models. The machine learning algorithms encompass Linear Discriminant Analysis (LDA), Logistic Regression (LR), Support Vector Machine (SVM), Random Forest (RF), Gradient Boosting Machine (GBM), Light GBM, and Extreme Gradient Boosting (XGBoost) (refer to Supplementary Appendix 1). Prediction models were developed in this study based on two modes and employing diverse algorithms: (1) Full mode: encompassing all selected features’ data; (2) Simplified mode: incorporating 30 crucial features chosen based on the algorithm used by the best model in full mode.

Model training and testing

The participant cohort was divided into training and testing datasets, with 80% of participants assigned to the training subset, and the remaining portion constituting the testing dataset. The cross-validation technique was also performed to access the over-fitting (13, 14).

Evaluation of model performance and interpretation

Performance assessment and comparison of all prediction models involved the calculation of metrics including the area under the receiver operating characteristic curve (AUROC), accuracy, sensitivity (recall), specificity, positive predictive value (PPV or precision), negative predictive value (NPV), and F1-score. The optimal model was determined by identifying the one with the highest AUROC through a comparative analysis of various models using testing results. Data processing was executed using MSSQL Server 2017, while model training and testing were carried out utilizing the Python programming language version 3.9 (15). The SHapley Additive exPlanations (SHAP) values were used to assess feature’s contribution (also known as its importance) to the most optimal model when interpreting the models (16).

Results

Baseline of patient characteristics

Table 1 shows basic characteristics of the study cohort, including patients’ demographic information, health status, COVID-19-related details, comorbidities, long-term medication records, and laboratory test results. In this study, 22,192 inhospitalized patients were included. Among the entire patient cohort, there were 12,452 female patients (56.1%), slightly outnumbering the 9,740 male patients (43.9%). The patients had a mean age of 49.3 (S.D. 17.4), with the majority falling below 65 years old (17,625, 79.4%), followed by those aged 65–85 (3,960, 17.8%), and those above 85 (607, 2.7%). Among the subset of patients with available BMI records (11,695), the patients’ average BMI was 24.4 (S.D. 4.51). The majority had a BMI greater than or equal to 24 (48.32%), while 45.44% had BMIs between 18.5 and 24, and 6.24% had BMIs below 18.5. The patients had an average CCI score of 0.53 (S.D. 1.52), with the majority achieving a CCI score of 0 (18,298, 82.5%). Following were patients with CCI scores ranging from 0 to 3 (2,115, 9.5%), while a smaller portion exhibited scores greater than 3 (1,779, 8.0%). A total of 5,820 individuals (26.2% of all patients) had a history of vaccine succession, while 558 individuals (2.5% of all patients) had received anti-COVID-19 virus drugs (Paxlovid or Molnupiravir). Complete basic patient characteristics are provided in Appendix 2.

TABLE 1

Table 1. Baseline of patient characteristics.

Full mode

Table 2 presents the performance evaluation of prediction models for overall severe outcome prediction, encompassing mortality, in the full mode. Upon analyzing the test outcomes, the Light GBM model exhibited the highest AUROC (0.939), surpassing other models including XGBoost (AUROC = 0.938), GBM (AUROC = 0.937), RF (AUROC = 0.936), LR (AUROC = 0.869), SVM (AUROC = 0.852), and LDA (AUROC = 0.852). The best-performing model (Light GBM) demonstrated accuracy, sensitivity, and specificity of 85.5%, 0.897, and 0.853, respectively. The cross-validation performance is provided in the Supplementary Appendices 6 and 8. In the cross-validation performance, the Light GBM had the consistent result with the external AUC at 0.924. Figure 2 illustrates the AUROC values of different models in the context of the full mode. The ROC curve delineating the performance of the prediction models for each specific outcome is provided in Supplementary Appendix 3(A). Figure 3 presents the feature importance for predicting severe outcomes or mortality using the optimal model within the full mode. The most significant features were age, vaccination before having PCR test, neutrophil count result, levels of sodium test and platelet count result.

TABLE 2

Table 2. Performance of prediction models under full mode.

FIGURE 2

Figure 2. ROC curve of performance of prediction models of severe outcomes or mortality under the full mode.

FIGURE 3

Figure 3. Shapley additive explanations chart of the feature importance for predicting severe outcomes or mortality by the best model under the full mode.

Simplified mode

The LGBM algorithm selected the 30 most crucial features from the entire set, which encompassed: sex type, age, BMI, CCI score, vaccination before having PCR test, COVID-19 medications, comorbidities including cardiovascular disease, COPD, renal disease, depression or anxiety, long-term medication such as NSAID, drugs for hypertension, drugs for GORD, aspirin, statin, antihyperuricemic, laboratory test results contain AST (GOT), ALT (GPT), creatinine, RBC, hemoglobin, MCH. MCHC, WBC, Neutrophil, PLT, HCT, NA and K. Table 3 displays the performance evaluation of prediction models for overall severe outcome prediction, inclusive of mortality, in the simplified mode. Based on the results of the tests, the XGBoost model achieved the highest AUROC (0.935) among the other models, namely RF (AUROC = 0.934), Light GBM (AUROC = 0.934), GBM (AUROC = 0.933), LR (AUROC = 0.863), SVM (AUROC = 0.846), and LDA (AUROC = 0.841). The optimal model (XGBoost) achieved accuracy, sensitivity, and specificity of 89.9%, 0.843, and 0.902, respectively. The XGBoost model demonstrates consistent performance when using the cross-validation strategy, with an external AUC of 0.934 The cross-validation performance of the prediction of individual indicators in the simple mode is shown in Supplementary Appendices 7 and 8. Figure 4 illustrates the AUROC values of different models within the context of the simplified mode. The ROC curve delineating the performance of the prediction models for each specific outcome is provided in Supplementary Appendix 3(B).

TABLE 3

Table 3. Performance of prediction models under simplified mode.

FIGURE 4

Figure 4. ROC curve of performance of prediction models of severe outcomes or mortality under the simplified mode. ROC, Receiver Operating Characteristic; MED, Medication; Lab, Laboratory result; LGBM, Light Gradient Boosting Machine; CCI, Charlson Comorbidity Index; GOT, Glutamic-oxaloacetic transaminase; GPT, Glutamic-pyruvic transaminase; MCH, Mean corpuscular hemoglobin; MCHC, Mean corpuscular hemoglobin concentration; COPD, Chronic Obstructive Pulmonary Disease; NSAID, Non-steroidal anti-inflammatory drugs; GORD, Gastro-oesophageal reflux disease.

The calibration plot showcasing the performance of prediction models for severe outcomes or mortality can be found in Supplementary Appendix 4. Additionally, the calibration plots illustrating the performance of prediction models for specific outcomes are furnished in Supplementary Appendix 5.

Discussion

Precise and personalized assessment of individuals at risk of developing severe COVID-19 outcomes holds the potential to enhance both the efficacy of clinical interventions and the judicious utilization of medical resources (17, 18). Several pivotal factors contribute to the heightened predictive capacity of machine learning (ML) models compared to conventional techniques. The considerable advantage of ML models lies in their capacity to generate predictions from vastly expanded datasets, a facet not to be understated. Moreover, ML models remain impervious to human emotions and subjective perspectives, thereby ensuring the objectivity and impartiality of the predictive process. Simultaneously, the innate adaptability inherent to ML models empowers them to swiftly acclimate and assimilate alterations, thereby amplifying their responsiveness to dynamic environments. Ultimately, ML models exhibit an aptitude for discerning intricate patterns of great complexity, often surpassing the capabilities of conventional methodologies. The choice of seven unique machine learning algorithms in this study is based on a comprehensive approach to developing personalized prediction models (19). The algorithms were chosen based on careful evaluation of their attributes and capabilities, ensuring they were in line with the project’s goals and the specific peculiarities of the dataset. The prediction models were developed by employing a range of algorithms, including traditional ones like LDA and LR, as well as basic methods like SVM. Additionally, this study utilize ensemble techniques that involve tree-based algorithms such as RF, GBM, Light GBM, and XGBoost (20, 21).

While prior investigations have constructed and validated predictive models with the goal of forecasting COVID-19 outcomes (22, 23), this study boasts several notable strengths. Firstly, it adeptly harnessed a more diverse and comprehensive dataset than its antecedents, encapsulating demographic particulars, COVID-19 vaccination statuses, COVID-19 drug utilization, comorbidities, long-term medication histories, and results from laboratory tests. Notably, this extends beyond the purview of earlier studies, which omitted the inclusion of long-term medication records and laboratory test outcomes (23–25). Furthermore, distinct from conventional algorithms, this study also employed advanced algorithms, a measure that facilitated the attainment of heightened precision in predictive models. Lastly, through a meticulous analysis of feature significance, this study procured a collection of the most pivotal predictors profoundly impacting model performance (6, 26, 27). The meticulous and personalized appraisal of patients susceptible to severe COVID-19 would undoubtedly amplify the efficacy of clinical interventions and streamline the judicious allocation of medical resources.

This study elucidates that the age of COVID-19 patients stands as the foremost predictor of severe outcome risk, aligning harmoniously with the conclusions drawn from diverse antecedent observational studies, which consistently affirm that elderly COVID-19 patients exhibit a heightened vulnerability to severe outcomes (27–29). Furthermore, this study’s findings expound upon the notion that pre-infection vaccination of COVID-19 patients equally serves as a pivotal predictor of serious events’ risk (including ventilator utilization, intubation, and mortality), as its primary function lies in averting the manifestation of numerous severe outcome risks. This alignment with prior research findings attests to the study’s robustness (30–32).

Presently, numerous national health authorities have issued declarations stipulating the utilization of antiviral agents against COVID-19, notably paxlovid (for individuals aged ≥12 years and weighing ≥40 kg) and molnupiravir (for individuals aged ≥18 years), as a crucial treatment avenue for at-risk patients (33). Zheng et al. conducted a meta-analysis, revealing Paxlovid’s efficacy and safety in managing high-risk COVID-19 patients (34). Debbiny et al.’s outcomes further underscored Paxlovid’s heightened efficacy within vulnerable demographics, encompassing elderly patients, those under immunosuppression, and individuals contending with underlying neurological or cardiovascular conditions (35). Concurrently, Benaicha et al.’s meta-analysis showcased the substantial reduction in all-cause mortality and hospitalization risk attributed to molnupiravir (36). Remarkably, this study’s findings reinforce the pivotal role of COVID-19 antiviral agents in predicting severe outcome risks. Post-COVID-19 infection, individuals incorporating COVID-19 antiviral medications within their treatment regimens evinced a substantial decline in the necessity for ventilator assistance within a three-month timeframe, vis-à-vis counterparts devoid of such treatment. The alignment of the predictive model with antecedent research outcomes underscores its congruence with established clinical practice and the prudent integration of prior findings.

The findings further highlight the significance of prolonged utilization of specific medications (such as benzodiazepines) as a salient affirmative predictor of severe outcome risk, a trend congruent with precedent observational investigations. This discovery bears noteworthy implications within clinical contexts (29). Benzodiazepines, encompassing medications frequently employed to address insomnia, anxiety, seizures, and alcohol withdrawal syndromes, interface with gamma-aminobutyric acid (GABA) receptors within the central nervous system, engendering a tranquilizing and pacifying impact upon the physiological framework. Notably, alongside the potential for immunosuppressive reactions entailing benzodiazepine administration, protracted usage might entail diminished respiratory function, exacerbating complexities among COVID-19 patients (37, 38).

Moreover, study’s investigation unveiled the substantial predictive potency of laboratory test outcomes, encompassing neutrophil count, white blood cell count, platelet count, MCH, and GOT, GPT, NA, and K levels. These variables assumed pivotal roles in the formulation of the predictive model, due to their influential role in disease progression. According to other systematic reviews, high blood White Blood Cell count (WBC), high blood aspartate aminotransferase (AST), high blood C-reactive protein (CRP), low blood platelet count, and a decrease in lymphocyte count may increase the possibilities of severe COVID-19 symptoms (39, 40). Hence, these variables assumed pivotal roles in the formulation of the predictive model, due to their influential role in disease progression.

Nonetheless, this study does encompass certain limitations. Primarily, it hinges upon electronic health records culled from diverse hospitals, constituting the primary wellspring of data. While these records amass a wealth of clinical intricacies, such as demographic particulars, disease management particulars, comprehensive medical histories incorporating comorbidities, prolonged medication use, and pivotal diagnostic outcomes, they regrettably omit several other data categories of import. Absent from this compilation are diverse facets of an individual’s lifestyle, spanning dietary habits, physical activity, tobacco and alcohol consumption, as well as socioeconomic indicators. In prospective endeavors, incorporation of this omitted information might yield alternative predictive models. In clinical practices, hospitals can adopt similar models to assist physicians in the prognostic process. However, a major obstacle is the limited availability and quality of data. The selection of these features was meticulously made, taking into account the available literature. While multiple features were employed in the study, the ones the study utilized are highly accessible and easily obtainable in the electronic health record (EHR) system. Therefore, our findings can be readily applied in future research. The issue of model interpretability is of utmost importance, as healthcare practitioners may struggle to comprehend complex machine learning algorithms. To improve the model’s interpretability, SHAP value ranking was additionally conducted in the findings.

Secondarily, it merits mention that the hospital-held electronic health records solely chronicle the specifics of a patient’s clinical visits, bypassing documentation of medical procedures and interventions executed within other healthcare institutions. Consequently, the clinical insights accessible for each patient might not have attained a truly all-encompassing status, potentially culminating in inaccuracies within the predictions of the predictive model.

Finally, a veritable acknowledgement is that the data origination in this study emanates solely from clinical archives of three hospitals within a singular Taiwanese system. While these hospitals have the largest number of COVID-19 patients in Taiwan, the study may not fully represent the entire population of Taiwan. Therefore, these models, which rely exclusively on hospital cases specific to Northern Taiwan, may have limitations in terms of the generalizability of their findings. Hence, for forthcoming research, it is prudent to foster inter-hospital collaboration and international partnership. Standardized case selection, research blueprinting, data structuring, processing methodologies, and analytical tools—when conjoined with predictive models engendered through multi-center federated learning—will furnish the substratum for the impending research trajectory.

Conclusion

This study has successfully developed an innovative and precise computer-aided risk prediction model designed to anticipate severe outcomes (including ventilator use, intubation, and intensive care unit admission) or mortality among COVID-19 patients. The outcomes of this research reveal that both the comprehensive and simplified models achieved an area under the curve (AUC) exceeding 0.9, accompanied by an accuracy rate surpassing 85%. The potential to apply timely medical interventions tailored to high-risk patients holds promise for preventing adverse outcomes and thereby ameliorating the disease’s impact on a substantial patient cohort. Although prediction model in this study performed well in the test set, one limitation of this study is the need to take into account the dataset’s representation. The future focus will be on externally validating the model. Collaboration with both domestic hospitals in Taiwan and hospitals in other countries, along with the utilization of the international database, is imperative. There is an expectation that further hospitals in southern Taiwan will be used to validate and enhance this model.

Data availability statement

The original contributions presented in the study are included in the article/Supplementary material, further inquiries can be directed to the corresponding author.

Ethics statement

The studies involving humans were approved by the Taipei Medical University–Joint Institutional Review Board (TMU-JIRB no. N202302020). The studies were conducted in accordance with the local legislation and institutional requirements. Written informed consent for participation was not required from the participants or the participants’ legal guardians/next of kin because all data were anonymized and de-identified before the analysis.

Author contributions

NH: Conceptualization, Formal analysis, Methodology, Writing – original draft, Writing – review & editing, Visualization. F-JT: Writing – review & editing. Y-HC: Conceptualization, Methodology, Writing – original draft, Visualization, Writing – review & editing. WB: Conceptualization, Writing – original draft, Visualization. PP: Data curation, Formal analysis, Methodology, Software, Writing – review & editing. P-AN: Data curation, Formal analysis, Methodology, Visualization, Writing – review & editing. DH: Writing – review & editing. CS-KL: Writing – review & editing. T-CL: Writing – review & editing. C-IC: Writing – review & editing. M-HH: Writing – review & editing. CYL: Writing – review & editing. C-WH: Writing – review & editing. H-CY: Writing – review & editing. JH: Conceptualization, Data curation, Methodology, Project administration, Resources, Supervision, Validation, Writing – review & editing.

Glossary

Funding

The author(s) declare financial support was received for the research, authorship, and/or publication of this article. This work was supported by National Science and Technology Council in Taiwan (grant nos. NSTC 112-2314-B-038-083 and NSTC 112-2813-C-038-047-B).

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fmed.2023.1289968/full#supplementary-material

References

1. Wang, D, Hu, B, Hu, C, Zhu, F, Liu, X, Zhang, J, et al. Clinical characteristics of 138 hospitalized patients with 2019 novel coronavirus-infected pneumonia in Wuhan, China. JAMA. (2020) 323:1061–9. doi: 10.1001/jama.2020.1585

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Wiersinga, WJ, Rhodes, A, Cheng, AC, Peacock, SJ, and Prescott, HC. Pathophysiology, transmission, diagnosis, and treatment of coronavirus disease 2019 (COVID-19): a review. JAMA. (2020) 324:782–93. doi: 10.1001/jama.2020.12839

CrossRef Full Text | Google Scholar

3. Wu, Z, and McGoogan, JM. Characteristics of and important lessons from the coronavirus disease 2019 (COVID-19) outbreak in China: summary of a report of 72 314 cases from the Chinese Center for Disease Control and Prevention. JAMA. (2020) 323:1239–42. doi: 10.1001/jama.2020.2648

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Cau, R, Faa, G, Nardi, V, Balestrieri, A, Puig, J, Suri, JS, et al. Long-COVID diagnosis: From diagnostic to advanced AI-driven models. Eur J Radiol. (2022) 148:110164. doi: 10.1016/j.ejrad.2022.110164

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Comito, C, and Pizzuti, C. Artificial intelligence for forecasting and diagnosing COVID-19 pandemic: a focused review. Artif Intell Med. (2022) 128:102286. doi: 10.1016/j.artmed.2022.102286

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Wynants, L, Van Calster, B, Collins, GS, Riley, RD, Heinze, G, Schuit, E, et al. Prediction models for diagnosis and prognosis of covid-19: systematic review and critical appraisal. BMJ. (2020):369. doi: 10.1136/bmj.m1328

CrossRef Full Text | Google Scholar

7. Abd-Alrazaq, A, Alajlani, M, Alhuwail, D, Schneider, J, Al-Kuwari, S, Shah, Z, et al. Artificial intelligence in the fight against COVID-19: scoping review. J Med Internet Res. (2020) 22:e20756. doi: 10.2196/20756

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Lai, C-C, Lee, P-I, and Hsueh, P-R. How Taiwan has responded to COVID-19 and how COVID-19 has affected Taiwan, 2020–2022. J Microbiol Immunol Infect. (2023) 56:433–41. doi: 10.1016/j.jmii.2023.04.001

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Hsu, JC, Nguyen, P-A, Phuc, PT, Lo, T-C, Hsu, M-H, Hsieh, M-S, et al. Development and validation of novel deep-learning models using multiple data types for lung cancer survival. Cancers. (2022) 14:5562. doi: 10.3390/cancers14225562

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Nguyen, QTN, Nguyen, PA, Wang, CJ, Phuc, PT, Lin, RK, Hung, CS, et al. Machine learning approaches for predicting 5-year breast cancer survival: a multicenter study. Cancer Sci. (2023) 114:4063–72. doi: 10.1111/cas.15917

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Chen, S-M, Phuc, PT, Nguyen, P-A, Burton, W, Lin, S-J, Lin, W-C, et al. A novel prediction model of the risk of pancreatic cancer among diabetes patients using multiple clinical data and machine learning. Cancer Med. (2023) 12:19987–99. doi: 10.1002/cam4.6547

CrossRef Full Text | Google Scholar

12. Azur, MJ, Stuart, EA, Frangakis, C, and Leaf, PJ. Multiple imputation by chained equations: what is it and how does it work? Int J Methods Psychiatr Res. (2011) 20:40–9. doi: 10.1002/mpr.329

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Jia, Z. Controlling the overfitting of heritability in genomic selection through cross validation. Sci Rep. (2017) 7:13678. doi: 10.1038/s41598-017-14070-z

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Brodeur, ZP, Herman, JD, and Steinschneider, S. Bootstrap aggregation and cross-validation methods to reduce overfitting in reservoir control policy search. Water Resour Res. (2020) 56:3–8. doi: 10.1029/2020WR027184

CrossRef Full Text | Google Scholar

15. Pedregosa, F, Varoquaux, G, Gramfort, A, Michel, V, Thirion, B, Grisel, O, et al. Scikit-learn: machine learning in Python. J Mach Learn Res. (2011) 12:2825–30.

Google Scholar

16. Ning, Y, Ong, MEH, Chakraborty, B, Goldstein, BA, Ting, DSW, Vaughan, R, et al. Shapley variable importance cloud for interpretable machine learning. Patterns. (2022) 3:100452. doi: 10.1016/j.patter.2022.100452

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Krumholz, HM. Big data and new knowledge in medicine: the thinking, training, and tools needed for a learning health system. Health Aff (Millwood). (2014) 33:1163–70. doi: 10.1377/hlthaff.2014.0053

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Goldstein, BA, Navar, AM, Pencina, MJ, and Ioannidis, JP. Opportunities and challenges in developing risk prediction models with electronic health records data: a systematic review. J Am Med Informat Associat. (2017) 24:198–208. doi: 10.1093/jamia/ocw042

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Hassan, CAU, Khan, MS, and Shah, MA. Comparison of machine learning algorithms in data classification. In Ma. X (Eds.), 2018 24th International Conference on Automation and Computing (ICAC) ; (2018); Newcastle Upon Tyne, UK: IEEE.

Google Scholar

20. Mahesh, B. Machine learning algorithms-a review. Int J Sci Res. (2020) 9:381–6. doi: 10.21275/ART20203995

CrossRef Full Text | Google Scholar

21. Kwekha-Rashid, AS, Abduljabbar, HN, and Alhayani, B. Coronavirus disease (COVID-19) cases analysis using machine-learning applications. Appl Nanosci. (2023) 13:2013–25. doi: 10.1007/s13204-021-01868-7

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Chen, H, Chen, R, Yang, H, Wang, J, Hou, Y, Hu, W, et al. Development and validation of a nomogram using on admission routine laboratory parameters to predict in-hospital survival of patients with COVID-19. J Med Virol. (2021) 93:2332–9. doi: 10.1002/jmv.26713

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Amiri, P, Montazeri, M, Ghasemian, F, Asadi, F, Niksaz, S, Sarafzadeh, F, et al. Prediction of mortality risk and duration of hospitalization of COVID-19 patients with chronic comorbidities based on machine learning algorithms. Digit Health. (2023) 9:205520762311704. doi: 10.1177/20552076231170493

CrossRef Full Text | Google Scholar

24. Wollenstein-Betech, S, Cassandras, CG, and Paschalidis, IC. Personalized predictive models for symptomatic COVID-19 patients using basic preconditions: hospitalizations, mortality, and the need for an ICU or ventilator. Int J Med Inform. (2020) 142:104258. doi: 10.1016/j.ijmedinf.2020.104258

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Das, AK, Mishra, S, and Saraswathy, GS. Predicting CoVID-19 community mortality risk using machine learning and development of an online prognostic tool. PeerJ. (2020) 8:e10083. doi: 10.7717/peerj.10083

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Chen, T, Wu, D, Chen, H, Yan, W, Yang, D, Chen, G, et al. Clinical characteristics of 113 deceased patients with coronavirus disease 2019: retrospective study. BMJ. (2020):368. doi: 10.1136/bmj.m1091

CrossRef Full Text | Google Scholar

27. Gong, J, Ou, J, Qiu, X, Jie, Y, Chen, Y, Yuan, L, et al. A tool for early prediction of severe coronavirus disease 2019 (COVID-19): a multicenter study using the risk nomogram in Wuhan and Guangdong, China. Clin Infect Dis. (2020) 71:833–40. doi: 10.1093/cid/ciaa443

PubMed Abstract | CrossRef Full Text | Google Scholar

28. The Novel Coronavirus Pneumonia Emergency Response Epidemiology Team. The epidemiological characteristics of an outbreak of 2019 novel coronavirus diseases (COVID-19)—China, 2020. China CDC Weekly. (2020) 2:113–22. doi: 10.46234/ccdcw2020.032

CrossRef Full Text | Google Scholar

29. Zhou, F, Yu, T, Du, R, Fan, G, Liu, Y, Liu, Z, et al. Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: a retrospective cohort study. Lancet. (2020) 395:1054–62. doi: 10.1016/S0140-6736(20)30566-3

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Huang, YZ, and Kuan, CC. Vaccination to reduce severe COVID-19 and mortality in COVID-19 patients: a systematic review and meta-analysis. Eur Rev Med Pharmacol Sci. (2022) 26:1770–6. doi: 10.26355/eurrev_202203_28248

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Efe, C, Tascilar, K, Gerussi, A, Bolis, F, Lammert, C, Ebik, B, et al. SARS-CoV-2 vaccination and risk of severe COVID-19 outcomes in patients with autoimmune hepatitis. J Autoimmun. (2022) 132:102906. doi: 10.1016/j.jaut.2022.102906

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Becerril-Gaitan, A, Vaca-Cartagena, BF, Ferrigno, AS, Mesa-Chavez, F, Barrientos-Gutierrez, T, Tagliamento, M, et al. Immunogenicity and risk of Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) infection after Coronavirus Disease 2019 (COVID-19) vaccination in patients with cancer: a systematic review and meta-analysis. Eur J Cancer. (2022) 160:243–60. doi: 10.1016/j.ejca.2021.10.014

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Saravolatz, LD, Depcinski, S, and Sharma, M. Molnupiravir and nirmatrelvir-ritonavir: oral coronavirus disease 2019 antiviral drugs. Clin Infect Dis. (2023) 76:165–71. doi: 10.1093/cid/ciac180

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Zheng, Q, Ma, P, Wang, M, Cheng, Y, Zhou, M, Ye, L, et al. Efficacy and safety of Paxlovid for COVID-19: a meta-analysis. J Infect. (2023) 86:66–117. doi: 10.1016/j.jinf.2022.09.027

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Najjar-Debbiny, R, Gronich, N, Weber, G, Khoury, J, Amar, M, Stein, N, et al. Effectiveness of Paxlovid in reducing severe coronavirus disease 2019 and mortality in high-risk patients. Clin Infect Dis. (2023) 76:e342–9. doi: 10.1093/cid/ciac443

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Benaicha, K, Khenhrani, RR, Veer, M, Devi, S, Shahbaz, U, Salah, QM, et al. Efficacy of Molnupiravir for the treatment of mild or moderate COVID-19 in adults: a meta-analysis. Cureus. (2023) 15:e38586. doi: 10.7759/cureus.38586

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Kasanagottu, K, and Herzig, SJ. Opioids, benzodiazepines, and COVID-19: a recipe for risk. J Hosp Med. (2022) 17:580–1. doi: 10.1002/jhm.12889

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Delaney, LD, Bicket, MC, Hu, HM, O'Malley, M, McLaughlin, E, Flanders, SA, et al. Opioid and benzodiazepine prescribing after COVID-19 hospitalization. J Hosp Med. (2022) 17:539–44. doi: 10.1002/jhm.12842

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Battaglini, D, Lopes-Pacheco, M, Castro-Faria-Neto, HC, Pelosi, P, and Rocco, PRM. Laboratory biomarkers for diagnosis and prognosis in COVID-19. Front Immunol. (2022) 13:857573. doi: 10.3389/fimmu.2022.857573

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Kazemi, E, Soldoozi Nejat, R, Ashkan, F, and Sheibani, H. The laboratory findings and different COVID-19 severities: a systematic review and meta-analysis. Ann Clin Microbiol Antimicrob. (2021) 20:17. doi: 10.1186/s12941-021-00420-3

PubMed Abstract | CrossRef Full Text | Google Scholar