Redefining COVID-19 Severity and Prognosis: The Role of Clinical and Immunobiotypes

Background: Most of the explanatory and prognostic models of COVID-19 lack of a comprehensive assessment of the wide COVID-19 spectrum of abnormalities. The aim of this study was to unveil novel biological features to explain COVID-19 severity and prognosis (death and disease progression).

Methods: A predictive model for COVID-19 severity in 121 patients was constructed by ordinal logistic regression calculating odds ratio (OR) with 95% confidence intervals (95% CI) for a set of clinical, immunological, metabolomic, and other biological traits. The accuracy and calibration of the model was tested with the area under the curve (AUC), Somer’s D, and calibration plot. Hazard ratios with 95% CI for adverse outcomes were calculated with a Cox proportional-hazards model.

Results: The explanatory variables for COVID-19 severity were the body mass index (BMI), hemoglobin, albumin, 3-Hydroxyisovaleric acid, CD8+ effector memory T cells, Th1 cells, low-density granulocytes, monocyte chemoattractant protein-1, plasma TRIM63, and circulating neutrophil extracellular traps. The model showed an outstanding performance with an optimism-adjusted AUC of 0.999, and Somer’s D of 0.999. The predictive variables for adverse outcomes in COVID-19 were severe and critical disease diagnosis, BMI, lactate dehydrogenase, Troponin I, neutrophil/lymphocyte ratio, serum levels of IP-10, malic acid, 3, 4 di-hydroxybutanoic acid, citric acid, myoinositol, and cystine.

Conclusions: Herein, we unveil novel immunological and metabolomic features associated with COVID-19 severity and prognosis. Our models encompass the interplay among innate and adaptive immunity, inflammation-induced muscle atrophy and hypoxia as the main drivers of COVID-19 severity.

Introduction

The novel beta coronavirus severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) emerged in December 2019 in Wuhan, China and is the cause of the coronavirus disease 2019 (COVID-19) (1), that was acknowledged by the World Health Organization as a pandemic in March 2020 (2), and has challenged and overwhelmed all the healthcare systems around the world (3).

Currently, the American continent is the epicenter of this pandemic, contributing with 51% of all new cases and 45% of all new deaths as reported in the first week of 2021 (4). Latin American (LA) countries, in particular, are being devastated by this disease, and information regarding the risk factors of COVID-19 severity in LA is quite scant. In Mexico, as of January 20, 2021, there have been approximately 1,600,000 confirmed cases and more than 142,000 deaths (5). Beyond the available number of beds with ventilators, concerns about the quality of care have also been raised, as suggested by the high mortality rate for intubated COVID-19 patients in Mexico, compared with other countries (6).

Current evidence suggests that besides the clinical risk factors related to COVID-19 severity, there are diverse immunological (7, 8), metabolic (9) and muscle (10) abnormalities that also play a key role in disease severity. Although diverse prognostic models for COVID-19 severity have been proposed, most of them only include limited immunological variables, with clinical, radiographic and laboratory features, without the incorporation of a comprehensive assessment of the wide COVID-19 spectrum of abnormalities. Besides, most prognostic models for adverse outcomes are based on Asian and European populations (11–14). Therefore, the aim of the present study was to create and validate a compound explanatory model including diverse clinical, immunological, metabolomic, and muscle atrophy variables to classify COVID-19 patients according to their disease severity and to predict adverse outcomes.

Material and Methods

We recruited a cohort of 121 patients with COVID-19 confirmed by a positive polymerase chain reaction (PCR) for SARS-CoV-2 in nasopharyngeal swab who consecutively attended to the emergency department of the Instituto Nacional de Ciencias Médicas y Nutrición Salvador Zubirán, a reference hospital for patients with COVID-19 in Mexico from March to August, 2020. This study was approved by the institutional Ethics and Research committees (REF: 3341) according to the Helsinki declaration and all patients signed a written informed consent prior to their inclusion.

At hospital admission, patients had an exhaustive medical approach including a complete medical history and a low radiation high-definition non-contrasted thoracic computed tomography (CT). Prior to the initiation of medical treatment, a blood sample was drawn for the assessment of the following laboratory tests: complete blood count, glucose, blood urea nitrogen (BUN), creatinine, liver function tests, ultra-sensitive C-reactive protein (CRP), lactate dehydrogenase (LDH), creatine phosphokinase (CPK), troponin-I, thromboplastin time (TP), activated partial thromboplastin time (aPTT), D-dimer, fibrinogen, and arterial blood gases. All plasma and serum samples were stored at -80°C until further analysis. We calculated the following pneumonia severity scores: Pneumonia Severity Index/Pneumonia Patient Outcomes Research Team (PSI/PORT), CURB-65 (confusion, urea, respiratory rate, blood pressure and age ≥65), National Early Warning Score (NEWS) 2, quick Sequential Organ Failure Assessment score (qSOFA), SMART-COP (systolic blood pressure, multilobar chest radiography involvement, albumin level, respiratory rate, tachycardia, confusion, oxygenation, and arterial pH) and MuLBSTA (multilobular involvement, absolute lymphocyte count, bacterial co-infection, smoking history, history of hypertension and age ≥60) (15). Patients were stratified according to their disease severity as following (16):

1. Mild/moderate disease: Fever, upper respiratory infection symptoms, with or without pneumonia.

2. Severe: Any of the following: respiratory failure, respiratory rate ≥ 30 breaths per minute, oxygen saturation at rest ≤ 93%, PaO₂/FiO₂ ≤ 300 mmHg.

3. Critical: Any of the following: requirement of invasive mechanical ventilation (IMV), shock, multiple organ failure.

Additionally, the following experimental procedures were performed:

Multiparametric Flow Cytometry Characterization of Peripheral Blood Mononuclear Cells (PBMCs)

We isolated peripheral blood mononuclear cells (PBMCs) by density gradients after centrifugation with Ficoll-Paque (GE Healthcare Life Sciences, Illinois, USA). After washing the cells twice with phosphate buffered saline (PBS), we stained them with the viability marker Zombie Aqua (Biolegend, California, USA). The cells were washed twice with 5% fetal bovine serum (FBS) in PBS and incubated during 30 min at room temperature with the FcX blocker (Biolegend, California, USA) and the following fluorochrome-coupled antibodies: CD19-BUV496 (Cat. 612938, BD Biosciences, Franklin Lakes, New Jersey, USA), CD3-APC/Fire-750 (Cat. 344840), CD4-Alexa fluor 488 (Cat. 317420), CD8-PE/Dazzle-594 (Cat. 344744), CD10-PE (Cat. 312204), CD11c-PE/Dazzle-594 (Cat. 337228), CD14-PerCP (Cat. 325632), CD15-FITC (Cat. 301904), CD16-Alexa fluor 700 (Cat. 302026), CD21-Alexa fluor 700 (Cat. 354918), CD24-BV421 (Cat. 311122), CD25-BV421 (Cat. 302630), CD27-APC-Cy7 (Cat. 356424), CD38-BV650 (Cat. 356620), CD45RA-PE (Cat. 304108), CD45RO-FITC (Cat. 304242), CD56-PE (Cat. 318306), CD62L-PE/Cy5 (Cat. 304808), CD127-BV650 (Cat. 351326), CD335-BV650 (Cat. 331927), CD355-APC (339108), CCR7-PE/Dazzle-594 (Cat. 353236), IgD-PerCP/Cy5.5 (Cat. 405710) (all from Biolegend, California, USA). For the assessment of the T helper and cytotoxic subsets, we stimulated the PBMCs with phorbol 12-myristate 13-acetate (PMA), ionomycin and monensin during 5 hrs at 37°C. The cells were fixed and permeabilized with the cytofix/cytoperm fixation/permeabilization kit (BD Biosciences, New Jersey, USA) according to the instructions of the manufacturer. Intracytoplasmic cytokines were detected with the following fluorochrome-coupled antibodies: IFN-γ-APC (Cat. 506510), IL-4-PE (Cat. 500810), IL-17-BV421 (Cat. 512322) (all from Biolegend, California, USA). One million events were acquired in a 4-laser LSR Fortessa flow cytometer (BD Biosciences, New Jersey, USA). We characterized the PBMCs subsets depicted in Supplementary Table 1.

The gating strategy for every cell subset is depicted in Supplementary Figures 1, 2. The absolute number (cells/mcl) of LDG, monocyte and lymphocyte subsets were calculated according to the amount of total leukocytes, monocytes and lymphocytes respectively, in a complete blood count drawn at the same day of the obtention of PBMCs.

Measurement of the Cytokine/Chemokine and Coagulation Profiles

The serum concentration of 32 cytokines and chemokines and 4 coagulation factors in plasma were measured using the MILLIPLEX Multi-Analyte Profiling (MAP) Human Cytokine/Chemokine Magnetic Bead Panel 29-plex kit (EMD Millipore, Darmstadt, Germany), the TGF-β Base Magnetic Luminex Performance Assay 3-plex kit (R&D Systems, Minneapolis, USA), and the ProcartaPlex Multiplex Immunoassay Human Coagulation Panel 4-plex kit (Thermo Fisher Scientific, Massachusetts, USA), on a 2-laser Bio-Plex 200 suspension array system coupled to a Bio-Plex Pro Wash Station (Bio-Rad, California, USA), according to the instructions of the manufacturers. Bead-fluorescence intensity readings for all the samples and standards were converted into the corresponding analyte concentrations using the Bio-Plex Manager software v6.2 (Bio-Rad, California, USA).

Analytes measured included: interleukin 1-α (IL-1α), IL-1β, IL-1RA, IL-2, IL-3, IL-4, IL-5, IL-6, IL-7, IL-8, IL-10, IL-12p40, IL-12p70, IL-13, IL-15, IL-17A, interferon α-2 (IFNα2), IFNγ, tumor necrosis factor-α (TNF-α), TNF-β, monocyte chemoattractant protein 1 (MCP-1)/CCL2, macrophage inflammatory protein 1-α (MIP-1α)/CCL3, macrophage inflammatory protein-1β (MIP-1β)/CCL4, interferon γ-induced protein (IP-10)/CXCL10, eotaxin-1/CCL11, epidermal growth factor (EGF), granulocyte colony-stimulating factor (G-CSF), granulocyte-macrophage colony-stimulating factor (GM-CSF), vascular endothelial growth factor (VEGF), transforming growth factor-β1 (TGF-β1), TGF-β2, TGF-β3, Factor IX, Protein C (Factor XIX), Protein S, and von Willebrand Factor (vWF). The serum levels of IL-18 were assessed by ELISA (MBL, Massachusetts, USA) according to the provider instructions.

Appraisal of the Metabolomic Signature

We performed untargeted metabolomics analysis of sera from all patients employing gas chromatography coupled to mass spectrometry (GC/MS). Our metabolomic method detected 46 metabolites with relative standard deviation (RSD) <30% in the quality control (QC) sample, that consists of equal volumes of all the samples included in the analysis, as previously described (17). The detailed procedure is described in the Supplementary Material.

Assessment of the Circulating Neutrophil Extracellular Traps (NETs)

The amount of plasma NETs was addressed by ELISA, as previously described (18). Briefly, high binding 96 well plates were coated overnight at 4°C with mouse anti human neutrophil elastase (NE) 1:2000 (Cat 481001, Calbiochem, Darmstadt, Germany) in coating buffer from the cell death detection ELISA kit (Roche, Basilea, Switzerland). We washed the plates three times with PBS/Tween20 and blocked the non-specific binding sites with 1% bovine serum albumin (BSA) in PBS for 6 hrs at room temperature for the detection of DNA-NE complexes as previously described (18). The plasma samples were diluted 1:10 in 1% BSA and incubated overnight at 4°C. After washing three times with PBS/Tween 20, we incubated the plates with the anti-human DNA-POD antibody from the cell death detection ELISA kit (Roche, Basilea, Switzerland). We washed the plate five times with PBS/Tween 20 and applied the TMB substrate (Thermofisher Scientific, Massachusetts, USA). The plate was read at 450 nm after applying stop solution and the optic density index (ODI) was calculated as previously described (18).

Evaluation of the Markers of Muscle Atrophy, Hypoxia, Anti-Viral Response, and Oxidative Stress

We assessed by ELISA, following the provider instructions, the plasma levels of TRIM63 (MyBioSource, California, USA) and atrogin-1 (MyBioSource, California, USA) as markers of muscle atrophy. The serum concentrations of HIF-1α (Thermofisher Scientific, Massachusetts, USA) and 8-hydroxy 2 deoxyguanosine (Abcam, Cambridge, UK) to address tissue hypoxia and oxidative stress, and the plasma levels of TRIM21 (MyBioSource, California, USA) as part of the antiviral innate immune response were measured by a commercial ELISA as well.

Noteworthy, all the samples were processed by investigators blinded to the COVID-19 severity and outcome of each included subject.

Statistics

Quantitative variables were expressed as medians and interquartile ranges (IQR). Differences among groups were assessed by the Kruskal-Wallis test. The sample size was not initially calculated because there was no available data about the size of the effect of the novel variables included in the COVID-19 severity predictive index. Nonetheless, the included sample size gave us enough power to detect statistically significant differences among groups. To address the factors associated with COVID-19 severity, an ordinal logistic regression model was developed including all the variables described above. First, we evaluated the correlation between variables and removed from the univariate analysis those with redundant statistical and biological information. Also, we discarded the clinical variables included in the definition of COVID-19 severity to avoid bias. We used a random forest trained algorithm on the observed values of a data matrix to predict the missing values (19). The outcome measured by the predictive model was COVID-19 severity, as mild/moderate, severe and critical (16).

To select the variables for the construction of the explanatory model for COVID-19 severity, we performed a univariate ordinal logistic regression. The statistically significant variables (P<0.05) were selected as candidate predictors. Finally, we chose the significant variables in the best fitting multivariate model using the minimum Akaike information criteria (AIC). Assumption of proportionality odds was verified with the Brant and Hosmer-Lemeshow tests.

Model performance and internal calibration were evaluated with conventional discrimination indexes: area under the curve (AUC, C-statistic), Somer’s D, Spearman’s rho, R², and optimism-corrected overfitting. Calibration plot and discrimination indexes were obtained by bootstrapping 1000 samples of the original data. Accuracy metrics [sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), positive likelihood ratio (+LR), and negative likelihood ratio (–LR)] were calculated by cross-validation and the goodness of fit of the model was assessed by Lipsitz test (20). Furthermore, we constructed a heat map by hierarchical clustering analysis with Ward’s method for visualization of the relationship between patient’s condition and the variables included in the index. Cox regression was used to assess the composite outcome of disease progression and death. Hazards ratios (HR) and 95% confidential interval (CI) were calculated. We selected variables in the best multivariate model with stepwise-selection (AIC minimum criteria).

The statistical analysis was performed with the R project software (21). The development and report of this predictive index was according to the TRIPOD (22) and STROBE (23) statements.

Results

Clinical Features of the Study Cohort

Eighty (66%) patients were men. The median age at recruitment was 48 (36–58) years. 34 (28%) patients were mild/moderate, 51 (42%) severe, and 36 (29%) critical. All patients completed the follow-up and were included in the analysis. 81 (66%) subjects had at least one comorbidity. 32 (26%) patients had diabetes, 31 (25%) hypertension, 48 (39%) obesity, 12 (9%) dyslipidemia, 8 (6%) cardiopathy, 4 (3%) chronic renal disease, 6 (4%) chronic liver disease, 4 (3%) chronic pulmonary disease, and 6 (4%) cancer. During the follow-up period, 22 (18%) patients died and 6 (5%) patients progressed. Patients with critical and severe COVID-19 had a higher body mass index (BMI) in comparison to mild disease (30.3 (27.45-34.12) vs 29.6 (27.35-31.68) vs 26.3 (24.9-29.1), P=0.0008, respectively). Conversely, mild/moderate patients had higher levels of hemoglobin (16.35 g/dL (16.35-16.35) vs 15.50 (14.2-16.35) vs 14.55 (11.55-16.12), P<0.0001) and albumin (4.55 g/dL (4.39-4.60) vs 3.80 (3.53-4.19) vs 3.12 (2.80-3.45), P<0.0001) in comparison to severe and critical subjects (Table 1).

TABLE 1

Table 1 Clinical features of patients with COVID-19 according to disease severity.

Multiparametric Flow Cytometry Analysis of PBMCs Subsets

In Table 2, we depict the median (IQR) of every cell subset according to disease severity. Patients with critical COVID-19 had a lower absolute number of Th1 cells (19.18 (6.17-78.71) vs 100.05 (52.23-240.89), P<0.0001), whilst mild/moderate patients showed higher amounts of CD8⁺ effector memory T cells in comparison to severe and critical COVID-19 (34.45 (17.28-41.53) vs 22.29 (5.16-28.58) vs 1.25 (0.31-2.41), P=0.0002). In the myeloid compartment, total LDGs augmented according to disease severity (42.15 (22.01-94.11) vs 181 (65.7-319.40) vs (1062 (334.00-3427.20), P<0.0001) (Table 2).

TABLE 2

Table 2 Peripheral blood cellular subsets of patients with COVID to 19 according to disease severity.

Serum Cytokine/Chemokine Profile

Other immunological variables, including serum cytokines/chemokines are summarized in Table 3. The serum levels of MCP-1 were higher in critical patients in comparison to mild/moderate subjects (592.23 (381.68-902.38) vs 473 (350-531.70), P=0.013).

TABLE 3

Table 3 Serum Cytokines and chemokines from COVID-19 according to disease severity.

Assessment of Circulating NETs, Muscle Atrophy, Hypoxia and Oxidative Stress Markers

Circulating NETs complexes were higher in severe and critical patients (0.83 (0.46-0.84) vs 1.3 (0.89-1.73) vs 1.11 (0.85-1.62), P=0.0004) in comparison to mild/moderate subjects as well (Table 4). Interestingly, the plasma levels of TRIM63 augmented according to the disease severity (Table 4).

TABLE 4

Table 4 Other biological markers in patients with COVID-19 according to disease severity.

Metabolomolic Signature

Metabolites that showed a fold change >0.5 (P adjusted <0.05) by clustering analysis according to disease severity are displayed in Supplementary Figure 3 and Supplementary Table 2. Differences among groups are depicted in Supplementary Table 3. Interestingly, the serum levels of 3-Hydroxyisovaleric acid increased according to disease severity [1.78 (1.42-2.16) vs 2.54 (1.82-3.20) vs 3.86 (2.63-4.85), P<0.0001].

We developed enrichment and pathway analysis using the 15 metabolites identified in the clustering analysis. The main pathways altered were those related to the metabolism of glutamate/glutamine and alanine-aspartate-glutamate, Krebs cycle, and arginine biosynthesis (Supplementary Figure 4).

Construction of the CLICOVID-19 SRS and the Adverse Outcomes Predictive Model

Variables included in the univariate analysis to construct the models for disease severity and prediction of adverse outcomes are depicted in Supplementary Table 4. The variables that individually predicted COVID-19 severity are shown in Table 5.

TABLE 5

Table 5 Variables associated to COVID-19 severity included in the selected model.

Missing data from hemoglobin and albumin from 19 patients were handled as described in Methods. There were not missing data from any of the other predictors.

The best supported model includes the BMI, hemoglobin, albumin, 3-Hydroxyisovaleric acid, absolute numbers of effector memory CD8+ T cells, Th1, LDGs and the serum/plasma concentrations of MCP-1, TRIM63, and NETs. The model was titled CLICOVID-19 SRS (CLinical and Immunological COVID-19 Severity Risk Score) and was defined as following:

$$\begin{array}{c}logit(Severit{y}_{level})=0.138BMI-0.835Hb-2.105Alb+0.623OHisovaleric\\ -0.0044EMCD8-0.011Th1+0.002LDG+0.004MCP\\ +0.003Trim63+2.383NETs\end{array}$$

A graphical representation of the effect of the explanatory variables included in the model on disease severity is shown in Figure 1.

FIGURE 1

Figure 1 Prediction probabilities of COVID-19 severity for the variables included in the model. The graphs represent the size of the effect of each explanatory variable in the disease severity.

The model predicts the probability for each level of disease severity depending on the values of the defining variables with an outstanding discriminative capacity with an optimism-corrected AUC (c-statistic) of 0.999 (Supplementary Table 5). As depicted in Figure 2, the index is able to separate the disease severity levels in clearly defined clusters.

FIGURE 2

Figure 2 Heat map from clustering analysis for COVID-19 severity according-the predictive variables. Hierarchical clustering analysis generated by Ward´s method relating mild/moderate (M, green), severe (S, yellow) and critical (C, red).

The goodness of fit evaluated by the Lipsitz test was 5.12, P=0.823, which indicates low discrepancy between observed and expected values. Besides, the model had a good calibration as shown in Supplementary Figure 6. The model showed a sensitivity of 0.807 (0.547-1), specificity of 0.896 (0.773-1), PPV of 0.854 (0.627-1), NPV of 0.910 (0.793-1), +LR of 7.740 (2.408-7.882), and -LR of 0.216 (0.1-0.586).

The median (IQR) of follow-up time according to disease severity was: severe COVID-19 (73 (19.73) days), critical COVID-19 (52.5 (7-73) days). All mild/moderate were followed-up for 73 days. The variables that independently predicted adverse outcomes in COVID-19 patients are shown in Table 6.

TABLE 6

Table 6 Multivariate Cox regression model to predict the risk of the development of adverse outcomes (progression and death) in patients with COVID-19.

Discussion

The main finding of the present work shows that a composite model which includes the BMI, hemoglobin, albumin, 3-Hydroxyisovaleric acid, absolute numbers of effector memory CD8⁺ T cells, Th1, LDGs and the serum/plasma concentrations of MCP-1, TRIM63, and NETs is a useful tool for the stratification of patients according to their disease severity. Our severity score has an outstanding discriminative capacity, showed good calibration, internal validity and unveils novel explanatory features for disease severity. Furthermore, our work unveils novel predictors of disease progression in COVID-19, including a distinctive metabolomic profile.

In COVID-19 obesity represents an independent risk factor for severe disease and admission to intensive care (24), which agrees with our study findings. This has been attributed to chronic inflammation mediated by adipose tissue and CD4⁺ lymphopenia with a biased phenotype towards Th17 and Th22 responses (24), which might explain the higher susceptibility for COVID-19 adverse outcomes.

According to our results, alterations in body composition are fundamental in the prognosis of COVID-19, since our study shows that muscle catabolism markers such as TRIM63 predict disease severity. ACE-2 is the receptor for SARS-CoV-2 and the activation of the renin angiotensin aldosterone system (RAS) favors muscle atrophy and fibrosis (25) through the transcription of the E3 ubiquitin ligase TRIM63 (25). Besides, it has been shown that critically ill patients with COVID-19 have necrosis, regeneration, atrophy and fatty infiltration of the muscle, which along with visceral adiposity, are risk factors for critical illness (26). To our knowledge, this is the first work that unveils the role of this ubiquitin ligase in COVID-19 severity.

The leucine catabolite 3-Hydroxyisovaleric acid has been found to be increased during hypoxic conditions (27). Agreeing with our results, raised serum levels of 3-Hydroxyisovaleric acid have been described in patients with severe COVID-19 and impaired lung function (27). This increase has been suggested to be secondary to an altered mitochondrial branched-chain amino acid (BCAA) metabolism related to hypoxia (27) and to the activation of the gluconeogenesis pathway. Indeed, COVID-19 severity and enhanced muscle catabolism have been associated with enhanced gluconeogenesis (27), which further supports our findings.

The cytokine storm is another acknowledged mechanism of tissue damage in COVID-19 and is driven by diverse cytokines and chemokines including MCP-1 and IP-10 (8), that are key mediators of lung damage through the chemotaxis of myeloid cells to the target tissues in viral infections (28). According to our results, IP-10 has been described as a key cytokine related to COVID-19 progression and its monitoring has been proposed as a promising tool for therapeutic decisions (29). MCP-1 is a chemokine involved in this cytokine storm and has been found to be elevated in sera of COVID-19 patients with respiratory failure (30). MCP-1 correlates with D-dimer and induces the synthesis of tissue factor which promotes thrombosis (31). Additionally, MCP-1 favors a Th2 phenotype and blocks the production of IFN-γ secreting cells (31), which agrees with our findings of diminished numbers of Th1 cells in severe COVID-19. In this regard CD8⁺ and CD4⁺ T cells from severe COVID-19 patients are known to have a lower cytotoxic activity and decreased production of IFN-γ respectively, which correlates with age and inflammatory parameters (32). Regarding this cytotoxic response, previous studies have shown that COVID-19 patients have decreased levels of CD8⁺ effector memory T cells (CD27-, CD45RA-) (33), which is corroborated by our findings. In animal models of lung viral infections, it has been shown that effector memory CD8⁺ T cells are key to an effective initial and subsequent immune response, which indicates that an adequate number of effector memory CD8⁺ T cells is essential in the anti-SARS-CoV-2 response. In fact, this cell subset is enriched in mild convalescent patients (34).

An abnormal antiviral T cell response allows the persistence of SARS-CoV-2 with the consequent over-activation of the innate immune response (35). In patients with severe COVID-19, a profound dysregulation of the myeloid compartment and emergency myelopoiesis have been demonstrated (36). Neutrophils are the key mediators of tissue damage in lung viral infections. In agreement with our results, previous studies have shown an increase in a cell subset similar to low-density granulocytes in COVID-19, particularly in critically ill subjects (37), which could be explained by the emergency myelopoiesis. According to our findings, NETs have been described as a key pathogenic factor in severe and critical COVID-19 (38).

Lower levels of hemoglobin and albumin are surrogate markers for the hyper-inflammatory state that drives COVID-19 severity. Anemia is found in approximately 31.3% of patients and is much more frequent in critical COVID-19 (39). Hypoalbuminemia (<3.5 g/dL) is detected in 74% of patients with severe COVID-19 and correlates with thrombotic complications (40). Both parameters were confirmed to be risk factors for COVID-19 severity in our study. Regarding disease progression, our study highlights the importance of LDH, troponin I (41) and neutrophil to lymphocyte ratio (42) as markers of severe inflammation and hypoxia-induced tissue damage and confirms their role as risk factors for adverse outcomes in COVID-19.

Our work confirms the role of increased levels of cystine, 3,4 dihydroxibutanoic and depletion of citric acid as independent risk factors for adverse outcomes in COVID-19 patients (43). On the other hand, malate levels increment could be related to the malate shuttle and malate dehydrogenase activities as a way to contend with increased NADH levels due to electron transport chain inhibition and increased glycolysis (44). Malate levels increment could be related to the malate shuttle and malate dehydrogenase activities as a way to contend with increased NADH levels due to electron transport chain inhibition and increased glycolisis (44). To our knowledge this is the first study to unveil the role of lower levels of myo inositol as a marker of disease progression in COVID-19. Depletion of myo inositol has been related to adverse outcomes in neonatal ARDS (45) and its supplementation has been associated with lower ROS levels in animal models (46), outcomes suggesting the potential beneficial effect of its supplementation.

Most of the COVID-19 severity explanatory models are not optimal since they are based exclusively on clinical variables assessed solely in hospitalized patients, with retrospective designs and without the inclusion of Latin American population. This is of great relevance, since the American continent is currently the epicenter of this pandemic and Mexico is one of the most affected countries in terms of confirmed cases, but mainly in the high lethality rate, which has been suggested to be related to the increased prevalence of metabolic comorbidities (47). Although this study is one of the few to include Latin-American patients, we acknowledge that our results should be confirmed in independent cohorts in order to be representative of this population. Our explanatory model shows a good calibration, internal validity and the highest discriminatory capacity in an indirect comparison with most of the published severity indexes (48–50). It also showed a well goodness of fit with lower discrepancy between observed and expected values, which highlights its utility.

To our knowledge, this is the first model that includes clinical features along with biological variables that represent the wide spectrum of the pivotal pathogenic phenomena related to disease severity in COVID-19. Under the current digital evolution driven by this pandemic, we decided to construct an online risk digital calculator based on the predictive index, which is publicly available (https://rai-unam-mx.shinyapps.io/CLICOVID-19-SRS/).

Our study has several limitations, including the sample size related to the high number of biological variables that were assessed, which was a barrier to include a larger amount of patients in whom all these variables could be measured. Nonetheless, the sample size allowed us to find differences among the severity groups. Additionally, this was a unicentric study, and the features included in our predictive index might only be available in tertiary care centers with access to flow cytometry and gas chromatography coupled to mass spectrometry (GC/MS). Another limitation of the study is the lack of information regarding the HLA genetic background of the patients and the SARS-CoV-2 viral load or genotype, which has been suggested to be contributors to COVID-19 severity (51). Our findings must be confirmed by the external validation in an independent cohort and in different settings. Moreover, to enhance the application of this predictive index, we are also developing a rapid reactive test to address the biological variables included in the index from a whole blood sample.

In summary, the CLICOVID-19 SRS showed the best performance to accurately stratify the COVID-19 severity in both ambulatory and hospitalized patients in an indirect comparison with other predictive indexes. A representation of the results of this study in Figure 3. After external validation in different settings, the CLICOVID-19 SRS could be a useful tool to optimize the healthcare resources allocated to manage COVID-19 by identifying subjects that could be safely managed as outpatients and those in whom hospital admission and intensive care is imperative.

FIGURE 3

Figure 3 Diverse clinical and biological traits are able to accurately predict COVID-19 severity. Severe/critical disease is characterized by increased BMI, LDGs, circulating NETs, plasma TRIM63, and serum 3-Hydroxyisovaleric acid, as well as diminished levels of Hb, albumin, and lymphopenia of Th1 and CD8⁺ effector memory T cells.

Data Availability Statement

The original contributions presented in the study are included in the article/Supplementary Material. Further inquiries can be directed to the corresponding author.

Ethics Statement

The studies involving human participants were reviewed and approved by Research and Ethics committees. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

JT-R, DG-M, and AP-F participated in the planning, conducting and reporting of the work. JM-M, NM-D, JP-F, SR-R, VS-H, AA-A, MN-A, GJ-V, and TH-G participated in the conducting of the work. AK-B participated in the creation of the on-line calculator and related app. AP-L and LL participated in the planning and reporting of the work. DG-M is the guarantor and responsible for the overall content of the paper. All authors contributed to the article and approved the submitted version.

Funding

This study was funded by a research grant provided by Consejo Nacional de Ciencia y Tecnología (CONACYT, grant number 313252, F0005-2020-01), as well as a research grant from the Coordinación de Investigación Científica-UNAM, granted to JM-M. This grant covered the costs for the acquisition of reagents and appropriate equipment for some of the experimental procedures.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

The authors would like to acknowledge the technical support provided by the Red de Apoyo a la Investigación (RAI). The icons of the graphic abstract were done using icons taken from BioRender App in https://biorender.com/.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fimmu.2021.689966/full#supplementary-material

References

1. Zhu N, Zhang D, Wang W, Li X, Yang B, Song J, et al. A Novel Coronavirus From Patients With Pneumonia in China, 2019. N Engl J Med (2020) 382(8):727–33. doi: 10.1056/NEJMoa2001017

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Cucinotta D, Vanelli M. WHO Declares COVID-19 a Pandemic. Acta BioMed (2020) 91(1):157–60. doi: 10.23750/abm.v91i1.9397

PubMed Abstract | CrossRef Full Text | Google Scholar

3. National Health Comission. Diagnosis and Treatment Protocol for Novel Coronavirus Pneumonia (Trial Version 7). Chin Med J (Engl) (2020) 133(9):1087–95. doi: 10.1097/CM9.0000000000000819

PubMed Abstract | CrossRef Full Text | Google Scholar

4. WHO. COVID-19 Weekly Epidemiological Update. Available at: https://www.who.int/publications/m/item/weekly-epidemiological-update-19-january-2021.

Google Scholar

5. Johns Hopkins Coronavirus Resource Center (2021). Available at: https://coronavirus.jhu.edumap.html (Accessed Jan 21). COVID-19 Map.

Google Scholar

6. Namendys-Silva SA, Gutierrez-Villasenor A, Romero-Gonzalez JP. Hospital Mortality in Mechanically Ventilated COVID-19 Patients in Mexico. Intensive Care Med (2020) 46(11):2086–8. doi: 10.1007/s00134-020-06256-3

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Nile SH, Nile A, Qiu J, Li L, Jia X, Kai G. COVID-19: Pathogenesis, Cytokine Storm and Therapeutic Potential of Interferons. Cytokine Growth Factor Rev (2020) 53:66–70. doi: 10.1016/j.cytogfr.2020.05.002

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Fajgenbaum DC, June CH. Cytokine Storm. N Engl J Med (2020) 383(23):2255–73. doi: 10.1056/NEJMra2026131

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Shen B, Yi X, Sun Y, Bi X, Du J, Zhang C, et al. Proteomic and Metabolomic Characterization of COVID-19 Patient Sera. Cell (2020) 182(1):59–72.e15. doi: 10.1016/j.cell.2020.05.032

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Manzano GS, Woods JK, Amato AA. Covid-19-Associated Myopathy Caused by Type I Interferonopathy. N Engl J Med (2020) 383(24):2389–90. doi: 10.1056/NEJMc2031085

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Izcovich A, Ragusa MA, Tortosa F, Lavena Marzio MA, Agnoletti C, Bengolea A, et al. Prognostic Factors for Severity and Mortality in Patients Infected With COVID-19: A Systematic Review. PloS One (2020) 15(11):e0241955. doi: 10.1371/journal.pone.0241955

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Dong YM, Sun J, Li YX, Chen Q, Liu QQ, Sun Z, et al. Development and Validation of a Nomogram for Assessing Survival in Patients With COVID-19 Pneumonia. Clin Infect Dis (2020) 72(4):652–60. doi: 10.1093/cid/ciaa963

CrossRef Full Text | Google Scholar

13. Kaeuffer C, Ruch Y, Fabacher T, Hinschberger O, Mootien J, Eyriey M, et al. The BAS(2)IC Score: A Useful Tool to Identify Patients at High Risk of Early Progression to Severe Coronavirus Disease 2019. Open Forum Infect Dis (2020) 7(10):ofaa405. doi: 10.1093/ofid/ofaa405

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Li J, Chen Y, Chen S, Wang S, Zhang D, Wang J, et al. Derivation and Validation of a Prognostic Model for Predicting in-Hospital Mortality in Patients Admitted With COVID-19 in Wuhan, China: The PLANS (Platelet Lymphocyte Age Neutrophil Sex) Model. BMC Infect Dis (2020) 20(1):959. doi: 10.1186/s12879-020-05688-y

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Fan G, Tu C, Zhou F, Liu Z, Wang Y, Song B, et al. Comparison of Severity Scores for COVID-19 Patients With Pneumonia: A Retrospective Study. Eur Respir J (2020) 56(3):2002113. doi: 10.1183/13993003.02113-2020

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Liu W, Tao ZW, Lei W, Ming-Li Y, Kui L, Ling Z, et al. Analysis of Factors Associated With Disease Outcomes in Hospitalized Patients With 2019 Novel Coronavirus Disease. Chin Med J (Engl) (2020) 133(9):1032–38. doi: 10.1097/CM9.0000000000000775

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Fiehn O. Metabolomics by Gas Chromatography-Mass Spectrometry: Combined Targeted and Untargeted Profiling. Curr Protoc Mol Biol (2016) 114:21.33.1–21.33.11. doi: 10.1002/0471142727.mb3004s114

CrossRef Full Text | Google Scholar

18. Lood C, Blanco LP, Purmalek MM, Carmona-Rivera C, De Ravin SS, Smith CK, et al. Neutrophil Extracellular Traps Enriched in Oxidized Mitochondrial DNA are Interferogenic and Contribute to Lupus-Like Disease. Nat Med (2016) 22(2):146–53. doi: 10.1038/nm.4027

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Stekhoven DJ, Buhlmann P. MissForest–non-Parametric Missing Value Imputation for Mixed-Type Data. Bioinformatics (2012) 28(1):112–8. doi: 10.1093/bioinformatics/btr597

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Lipsitz SR, Buoncristiani JF. A Robust Goodness-of-Fit Test Statistic With Application to Ordinal Regression Models. Stat Med (1994) 13(2):143–52. doi: 10.1002/sim.4780130205

PubMed Abstract | CrossRef Full Text | Google Scholar

21. R Core Team. A Language and Environment for Statistical Computing R Foundation for Statistical Computing. Vienna, Austria (2021). Available at: http://www.Rproject.org/.

Google Scholar

22. Zamanipoor Najafabadi AH, Ramspek CL, Dekker FW, Heus P, Hooft L, Moons KGM, et al. TRIPOD Statement: A Preliminary Pre-Post Analysis of Reporting and Methods of Prediction Models. BMJ Open (2020) 10(9):e041537. doi: 10.1136/bmjopen-2020-041537

PubMed Abstract | CrossRef Full Text | Google Scholar

23. von Elm E, Altman DG, Egger M, Pocock SJ, Gotzsche PC, Vandenbroucke JP, et al. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement: Guidelines for Reporting Observational Studies. Lancet (2007) 370(9596):1453–7. doi: 10.1016/S0140-6736(07)61602-X

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Korakas E, Ikonomidis I, Kousathana F, Balampanis K, Kountouri A, Raptis A, et al. Obesity and COVID-19: Immune and Metabolic Derangement as a Possible Link to Adverse Clinical Outcomes. Am J Physiol Endocrinol Metab (2020) 319(1):E105–E9. doi: 10.1152/ajpendo.00198.2020

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Gonzalez A, Orozco-Aguilar J, Achiardi O, Simon F, Cabello-Verrugio C. SARS-CoV-2/Renin-Angiotensin System: Deciphering the Clues for a Couple With Potentially Harmful Effects on Skeletal Muscle. Int J Mol Sci (2020) 21(21):7904. doi: 10.3390/ijms21217904

CrossRef Full Text | Google Scholar

26. Cabanes-Martinez L, Villadoniga M, Gonzalez-Rodriguez L, Araque L, Diaz-Cid A, Ruz-Caracuel I, et al. Neuromuscular Involvement in COVID-19 Critically Ill Patients. Clin Neurophysiol (2020) 131(12):2809–16. doi: 10.1016/j.clinph.2020.09.017

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Barberis E, Timo S, Amede E, Vanella VV, Puricelli C, Cappellano G, et al. Large-Scale Plasma Analysis Revealed New Mechanisms and Molecules Associated With the Host Response to SARS-CoV-2. Int J Mol Sci (2020) 21(22):8623. doi: 10.3390/ijms21228623

CrossRef Full Text | Google Scholar

28. Wang J, Jiang M, Chen X, Montaner LJ. Cytokine Storm and Leukocyte Changes in Mild Versus Severe SARS-CoV-2 Infection: Review of 3939 COVID-19 Patients in China and Emerging Pathogenesis and Therapy Concepts. J Leukoc Biol (2020) 108(1):17–41. doi: 10.1002/JLB.3COVR0520-272R

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Lev S, Gottesman T, Sahaf Levin G, Lederfein D, Berkov E, Diker D, et al. Observational Cohort Study of IP-10’s Potential as a Biomarker to Aid in Inflammation Regulation Within a Clinical Decision Support Protocol for Patients With Severe COVID-19. PloS One (2021) 16(1):e0245296. doi: 10.1371/journal.pone.0245296

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Jontvedt Jorgensen M, Holter JC, Christensen EE, Schjalm C, Tonby K, Pischke SE, et al. Increased Interleukin-6 and Macrophage Chemoattractant Protein-1 are Associated With Respiratory Failure in COVID-19. Sci Rep (2020) 10(1):21697. doi: 10.21203/rs.3.rs-39162/v1

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Deshmane SL, Kremlev S, Amini S, Sawaya BE. Monocyte Chemoattractant Protein-1 (MCP-1): An Overview. J Interferon Cytokine Res: Off J Int Soc Interferon Cytokine Res (2009) 29(6):313–26. doi: 10.1089/jir.2008.0027

CrossRef Full Text | Google Scholar

32. Gutierrez-Bautista JF, Rodriguez-Nicolas A, Rosales-Castillo A, Jimenez P, Garrido F, Anderson P, et al. Negative Clinical Evolution in COVID-19 Patients Is Frequently Accompanied With an Increased Proportion of Undifferentiated Th Cells and a Strong Underrepresentation of the Th1 Subset. Front Immunol (2020) 11:596553. doi: 10.3389/fimmu.2020.596553

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Habel JR, Nguyen THO, van de Sandt CE, Juno JA, Chaurasia P, Wragg K, et al. Suboptimal SARS-CoV-2-Specific CD8(+) T Cell Response Associated With the Prominent HLA-A*02:01 Phenotype. Proc Natl Acad Sci U.S.A. (2020) 117(39):24384–91. doi: 10.1073/pnas.2015486117

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Rha MS, Jeong HW, Ko JH, Choi SJ, Seo IH, Lee JS, et al. PD-1-Expressing SARS-CoV-2-Specific CD8(+) T Cells Are Not Exhausted, But Functional in Patients With COVID-19. Immunity (2021) 54(1):44–52.e3. doi: 10.1016/j.immuni.2020.12.002

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Peng Y, Mentzer AJ, Liu G, Yao X, Yin Z, Dong D, et al. Broad and Strong Memory CD4(+) and CD8(+) T Cells Induced by SARS-CoV-2 in UK Convalescent Individuals Following COVID-19. Nat Immunol (2020) 21(11):1336–45. doi: 10.1038/s41590-020-0782-6

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Schulte-Schrepping J, Reusch N, Paclik D, Bassler K, Schlickeiser S, Zhang B, et al. Severe COVID-19 Is Marked by a Dysregulated Myeloid Cell Compartment. Cell (2020) 182(6):1419–40.e23. doi: 10.1016/j.cell.2020.08.001

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Leppkes M, Knopf J, Naschberger E, Lindemann A, Singh J, Herrmann I, et al. Vascular Occlusion by Neutrophil Extracellular Traps in COVID-19. EBioMedicine (2020) 58:102925. doi: 10.1016/j.ebiom.2020.102925

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Barnes BJ, Adrover JM, Baxter-Stoltzfus A, Borczuk A, Cools-Lartigue J, Crawford JM, et al. Targeting Potential Drivers of COVID-19: Neutrophil Extracellular Traps. J Exp Med (2020) 217(6):e20200652. doi: 10.1084/jem.20200652

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Liao D, Zhou F, Luo L, Xu M, Wang H, Xia J, et al. Haematological Characteristics and Risk Factors in the Classification and Prognosis Evaluation of COVID-19: A Retrospective Cohort Study. Lancet Haematol (2020) 7(9):e671–e8. doi: 10.1016/S2352-3026(20)30217-9

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Violi F, Ceccarelli G, Cangemi R, Alessandri F, D’Ettorre G, Oliva A, et al. Hypoalbuminemia, Coagulopathy, and Vascular Disease in COVID-19. Circ Res (2020) 127(3):400–1. doi: 10.1161/CIRCRESAHA.120.317173

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Zheng Z, Peng F, Xu B, Zhao J, Liu H, Peng J, et al. Risk Factors of Critical & Mortal COVID-19 Cases: A Systematic Literature Review and Meta-Analysis. J Infect (2020) 81(2):e16–25. doi: 10.1016/j.jinf.2020.04.021

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Feng X, Li S, Sun Q, Zhu J, Chen B, Xiong M, et al. Immune-Inflammatory Parameters in COVID-19 Cases: A Systematic Review and Meta-Analysis. Front Med (Lausanne) (2020) 7:301. doi: 10.3389/fmed.2020.00301

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Shi D, Yan R, Lv L, Jiang H, Lu Y, Sheng J, et al. The Serum Metabolome of COVID-19 Patients Is Distinctive and Predictive. Metabolism (2021) 118:154739. doi: 10.1016/j.metabol.2021.154739

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Hanse EA, Ruan C, Kachman M, Wang D, Lowman XH, Kelekar A. Cytosolic Malate Dehydrogenase Activity Helps Support Glycolysis in Actively Proliferating Cells and Cancer. Oncogene (2017) 36(27):3915–24. doi: 10.1038/onc.2017.36

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Hallman M, Saugstad OD, Porreco RP, Epstein BL, Gluck L. Role of Myoinositol in Regulation of Surfactant Phospholipids in the Newborn. Early Hum Dev (1985) 10(3-4):245–54. doi: 10.1016/0378-3782(85)90055-6

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Nascimento NR, Lessa LM, Kerntopf MR, Sousa CM, Alves RS, Queiroz MG, et al. Inositols Prevent and Reverse Endothelial Dysfunction in Diabetic Rat and Rabbit Vasculature Metabolically and by Scavenging Superoxide. Proc Natl Acad Sci USA (2006) 103(1):218–23. doi: 10.1073/pnas.0509779103

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Bello-Chavolla OY, Bahena-Lopez JP, Antonio-Villa NE, Vargas-Vazquez A, Gonzalez-Diaz A, Marquez-Salinas A, et al. Predicting Mortality Due to SARS-CoV-2: A Mechanistic Score Relating Obesity and Diabetes to COVID-19 Outcomes in Mexico. J Clin Endocrinol Metab (2020) 105(8):dgaa346. doi: 10.1210/clinem/dgaa346

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Clift AK, Coupland CAC, Keogh RH, Diaz-Ordaz K, Williamson E, Harrison EM, et al. Living Risk Prediction Algorithm (QCOVID) for Risk of Hospital Admission and Mortality From Coronavirus 19 in Adults: National Derivation and Validation Cohort Study. BMJ (2020) 371:m3731. doi: 10.1136/bmj.m3731

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Knight SR, Ho A, Pius R, Buchan I, Carson G, Drake TM, et al. Risk Stratification of Patients Admitted to Hospital With Covid-19 Using the ISARIC WHO Clinical Characterisation Protocol: Development and Validation of the 4C Mortality Score. BMJ (2020) 370:m3339. doi: 10.1136/bmj.m3339

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Liang W, Liang H, Ou L, Chen B, Chen A, Li C, et al. Development and Validation of a Clinical Risk Score to Predict the Occurrence of Critical Illness in Hospitalized Patients With COVID-19. JAMA Intern Med (2020) 180(8):1081–9. doi: 10.1001/jamainternmed.2020.2033

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Romero-Lopez JP, Carnalla-Cortes M, Pacheco-Olvera DL, Ocampo-Godinez JM, Oliva-Ramirez J, Moreno-Manjon J, et al. A Bioinformatic Prediction of Antigen Presentation From SARS-CoV-2 Spike Protein Revealed a Theoretical Correlation of HLA-DRB1*01 With COVID-19 Fatality in Mexican Population: An Ecological Approach. J Med Virol (2021) 93(4):2029–38. doi: 10.1002/jmv.26561

PubMed Abstract | CrossRef Full Text | Google Scholar