Predicting survival in patients with SARS-CoV-2 based on cytokines and soluble immune checkpoint regulators

Background: Coronavirus disease 2019 (COVID-19) has been widespread for over four years and has progressed to an endemic stage. Accordingly, the evaluation of host immunity in infected patients and the development of markers for prognostic prediction in the early stages have been emphasized. Soluble immune checkpoints (sICs), which regulate T cell activity, have been reported as promising biomarkers of viral infections.

Methods: In this study, quantitative values of 17 sICs and 16 cytokines (CKs) were measured using the Luminex multiplex assay. A total of 148 serum samples from 100 patients with COVID-19 were collected and the levels were compared between survivors vs. non-survivors and pneumonic vs. non-pneumonic conditions groups. The impact of these markers on overall survival were analyzed using a machine learning algorithm.

Results: sICs, including sCD27, sCD40, herpes virus entry mediator (sHVEM), T-cell immunoglobulin and mucin-domain containing-3 (sTIM-3), and Toll-like receptor 2 (sTLR-2) and CKs, including chemokine CC motif ligand 2 (CCL2), interleukin-6 (IL-6), IL-8, IL-10, IL-13, granulocyte-macrophage colony-stimulating factor (GM-CSF), and tumor necrosis factor-α (TNF- α), were statistically significantly increased in the non-survivors compared to those of in the survivors. IL-6 showed the highest area under the receiver-operating curve (0.844, 95% CI = 0.751–0.913) to discriminate non-survival, with a sensitivity of 78.9% and specificity of 82.4%. In Kaplan-Meier analysis, patients with procalcitonin over 0.25 ng/mL, C-reactive protein (CRP) over 41.0 mg/dL, neutrophil-to-lymphocyte ratio over 18.97, sCD27 over 3828.8 pg/mL, sCD40 over 1283.6 pg/mL, and IL-6 over 21.6 pg/mL showed poor survival (log-rank test). In the decision tree analysis, IL-6, sTIM-3, and sCD40 levels had a strong impact on survival. Moreover, IL-6, CD40, and CRP levels were important to predict the probability of 90-d mortality using the SHapley Additive exPlanations method.

Conclusion: sICs and CKs, especially IL-6, sCD27, sCD40, and sTIM-3 are expected to be useful in predicting patient outcomes when used in combination with existing markers.

1 Introduction

As the SARS-CoV-2 pandemic has been prevalent for more than four years, it is progressing to the endemic stage, where SARS-CoV-2 variants occur and the number of re-infected patients increases (Markov et al., 2023; Nesteruk, 2023; Nguyen et al., 2023). To respond to this ongoing endemic stage of coronavirus disease 2019 (COVID-19), control of the infection from a long-term perspective has been emphasized and the importance of host immunity in each patient has increased. In addition, laboratory tests are important not only for the detection of SARS-CoV-2 infection but also for the screening and prognosis prediction of severe patients (Carabelli et al., 2023; Lim, 2023; Puhach et al., 2023). Many studies have been conducted on infection markers in SARS-CoV-2 patients (Kermali et al., 2020). C-reactive protein (CRP), white cell count, and cytokines (CK) such as interleukin-6 (IL-6), IL-8, and IL-10 have been investigated as factors associated with the diagnosis or severity of COVID-19 (Han et al., 2020; Kermali et al., 2020). CRP is a plasma protein generated in the liver in response to inflammatory mediators and is associated with disease severity during SARS-CoV-2 infection. Regarding white blood cells (WBC), in patients with severe COVID-19, there is a pattern where the neutrophil count is dominant and the lymphocyte count decreases, leading to the expectation that the neutrophil-to-lymphocyte (N-to-L) ratio could serve as a potential biomarker for the early detection of severe features. In particular, IL-6, which functions as a pleiotropic CK with both pro- and anti-inflammatory functions, displays hyperinflammation beyond homeostatic levels, inducing CK release syndromes, such as a CK storm (REMAP-CAP Investigators, 2021). The elevation of IL-6 in both serum and alveoli serves as an indicator of hyperinflammation and is utilized as an unfavorable prognostic factor (REMAP-CAP Investigators, 2021). In addition, soluble immune checkpoints (sICs) are emerging factors for understanding SARS-CoV-2 infection and progression (Cai et al., 2020; Sun et al., 2022). Previous studies have reported the importance of sICs as potential biomarkers of viral infections (Cai et al., 2020; Aghbash et al., 2021; Morrell et al., 2022). sICs are expressed on immune cells to regulate T cell activity and primarily act as immune evasion mechanisms (Marin-Acevedo et al., 2018; Zahavi and Weiner, 2019; Cai et al., 2020; Joseph et al., 2021; Sun et al., 2022; Yu et al., 2023). Specifically, in SARS-CoV-2 infection, up-regulation of T-cells has been reported in several studies; sICs are implicated in mediating T-cell exhaustion and inducing T-cell lymphopenia, critical factors contributing to the pathogenesis of severe COVID-19 (Cai et al., 2020). However, few studies have quantitatively evaluated the effectiveness of sICs as prognostic markers in clinical practice for patients with SARS-CoV-2 infection (Gambichler et al., 2020; Gatto et al., 2020; Awadasseid et al., 2021). Additional research is required to concurrently assess various biomarkers, determine markers related to host immunity, and predict prognosis in the early stages of SARS-CoV-2 infection.

In a previous study, the authors compared the quantitative values of sICs and CKs in SARS-CoV-2 infected patients in the survivor and non-survivor groups (Lee et al., 2022). However, this study has some limitations. The number of target patients was low, and some specimens fell below the limit of detection (LoD), making it challenging to measure CK levels accurately. In the current study, we aimed to enhance the robustness of our findings by implementing several methodological enhancements. Firstly, we doubled the total number of patients with SARS-CoV-2 and focused on samples collected within one week of SARS-CoV-2 diagnosis. The focus on the first week was paramount as it represented a critical period for patient management. Furthermore, we took measures to ensure adequate sample volume from all patients, thereby reducing instances of the samples falling below the LoD. Additionally, our study enrolled patients during the period when the Omicron variant was prevalent. The variant exhibited distinct clinical characteristics, including different mortality rates, compared to the Wuhan strain variant in the previous study. Secondly, infection markers, including WBC, N-to-L ratios, and CRP (Ponti et al., 2020; Bedel and Korkut, 2021; Agarwal, 2022), were added and analyzed in an integrated manner along with sICs and CK. Furthermore, co-infection with fungi, bacteria, and viruses in patients with SARS-CoV-2 infection has a poor prognosis, and the microbial community interacts with immune responses related to prognosis (Jeong et al., 2022). Jeong et al. reported that SARS-CoV-2 infection is related to multidrug resistance (Jeong et al., 2022). Therefore, this study sought to examine the relationship between sICs and multidrug-resistant bacteria. Finally, we utilized a machine learning (ML) algorithm to efficiently evaluate the importance of various types of biomarkers in prognosis prediction.

In particular, this study emphasizes the utilization of sICs and CKs in clinical practice and seeks to identify a process that can be applied to actual patient management. As it is not always possible to measure all sICs and CK in actual medical practice, their importance must be evaluated and a process must be established to select these markers and to best utilize them. In this study, we implemented ML algorithms and comprehensively analyzed the prognostic prediction abilities of various clinical biomarkers, including sICs, CK, and infection biomarkers.

2 Materials and methods

2.1 Characteristics of study population

From May 2022 to September 2022, 148 specimens were collected from individuals (N = 100) diagnosed with SARS-COV-2 and had a history of hospitalization. The inclusion criteria were as follows: (i) serum sample within one week after admission, (ii) hospitalization, and (iii) detection of SARS-CoV-2 RNA from nasopharyngeal/throat swabs using real-time reverse transcription-polymerase chain reaction (RT-PCR). The CFX96 real-time PCR (Bio-Rad, Hercules, CA, USA) was used for detection of SARS-CoV-2 RNA. Pediatric patients and those with insufficient serum sample volumes were excluded. Among the 148 collected specimens, 100 were drawn during the first week after admission. These specimens were used for comparing non-survivors and survivors, as well as for evaluating differences between pneumonic and non-pneumonic conditions, conducting survival analysis, and ML analysis, including decision tree analysis. Additionally, values from the second week were measured in 48 patients, comprising of 42 survivors and 6 non-survivors. The trend of sICs and CK values between the first and second weeks was analyzed. This study was approved by the Institutional Review Board of Kangnam Sacred Heart Hospital at Hallym University. The requirement for informed consent was waived to maintain personal information anonymity (HKS. 2020-08-004-003).

2.2 Assessments of sIC regulators, CKs, and laboratory tests

Residual serum samples collected from the chemistry tests were aliquoted into microtubes and stored at -70°C for sIC and CK evaluation. The quantitative values of serum sICs and CKs were measured using a Luminex 200 Bio-Plex instrument (Bio-Rad, Hercules, CA, USA), following the manufacturer’s protocols. Measurements of sICs and CK were conducted using the methods described in a previous study (Jeong et al., 2022; Lee et al., 2022). Briefly, using a color-coded multiplexing method, Luminex 200 Bio-Plex simultaneously detects analytes through capture antibodies on beads, followed by fluorescence-based measurements for high-throughput immune profiling. In this study, we included 17 types of sICs and 16 types of CK that were previously measured in the literature (Lee et al., 2022). Laboratory data obtained on the same date as the measurement date for sIC and CK were retrospectively collected from medical records. It included procalcitonin (PCT), CRP, total WBC, absolute neutrophil, and lymphocyte counts. In addition, multidrug-resistant bacteria such as methicillin-resistant S.aureus (MRSA), carbapenem-resistant Acinetobacter baumannii (CRAB), and carbapenem-resistant Enterobacteriaceae (CRE) were investigated.

2.3 Statistical analysis

The Mann-Whitney U test was used to evaluate the statistically significant differences in survivors vs. non-survivors and pneumonic vs. non-pneumonic conditions groups. The area under the receiver-operating characteristic curve (AUC) of receiver operating characteristic (ROC) was used to evaluate the performance of each biomarker. The cutoff for the continuum value was calculated using the Youden index, and its sensitivity and specificity were evaluated. Pearson’s correlation coefficient was used to assess the linear relationship between various sICs, CKs, and infection markers (e.g., CRP, PCT). Cumulative overall survival curves for each sIC, CK, and laboratory biomarker were calculated using the Kaplan-Meier analysis and examined using the log-rank test. Cluster analysis was performed to comprehensively evaluate the factors affecting the prognosis of SARS-CoV-2 infected patients. K-means clustering algorithms were selected for clustering and the optimal number of clusters (K) was determined. A prognostic analysis was performed for each cluster. Paired samples from the 1^st and 2^nd weeks were compared using paired t-tests. Differences were considered statistically significant at P < 0.05. All statistical analyses were performed using MedCalc software version 12.0 (MedCalc Software, Mariakerke, Belgium).

2.4 ML

ML, based on a decision-tree algorithm, was used for survival and prognosis prediction. A decision tree is a data classification technique that is constructed in various ways as branch-like fragments. It consists of a root node, leaf node, and decision (internal) node, which indicate the node at the top, the class to be assigned to patients, and the tree corresponding to the features, respectively. The sICs, CK, and laboratory test data were all included as parameters and selected for survival and prognosis, prioritizing items that were statistically significant in the Mann-Whitney U analysis. Before training, patient identification information was removed from the raw data for preprocessing, and categorical variables were encoded using one-hot encoding to facilitate processing by the decision tree algorithm. The decision to split the dataset into 80% training and 20% testing was based on standard ML practices to balance learning capability and validation accuracy. Google Colaboratory, a Colabcloud-based service, was utilized (Hoyos-Rivera et al., 2006) to perform the ML operations. Training and prediction were performed using a decision tree classifier, the structure of the tree was visualized by drawing a plot, and information corresponding to the node was obtained. The decision tree classifier was configured with a maximum depth of 10 to prevent overfitting. The significance of each variable was analyzed through seaborn barplot and the accuracy and AUC of ROC curve were examined. In addition, to evaluate significance of the features for survival prediction and interpret the ML results, we utilized the SHapley Additive exPlanations (SHAP). SHAP values were calculated to quantify the contribution of each feature to the prediction outcome, providing a clearer picture of model decisions. This allowed us to identify which features were most influential and how they interacted to affect survival predictions. The code utilized in this study can be accessed online (https://github.com/Nurilee822/sICsandCK_COVID19).

3 Results

3.1 Patients

Of the 100 patients enrolled in the study, 19 died and 81 survived. Among the 100 patients, 49 were diagnosed with pneumonia and the remaining 51 patients showed mild symptoms. The average age was 82.0 and 73.0 years for non-survivors and survivors, respectively, and 80.0 and 72.0 for pneumonic and non-pneumonic conditions patients, respectively (Table 1). Fourteen patients had cancer, 50 had hypertension, 32 had diabetes mellitus (DM), 16 had cardiovascular disease, 36 had other diseases, and 20 had no relevant medical history. CRAB, CRE, and MRSA infections were present in six, one, and six patients, respectively. No statistically significant differences were exhibited in age, sex, underlying disease, and bacterial infections, such as CRAB, CRE, and MRSA between the non-survivors and survivors. In patients with pneumonia, no statistically significant disparity was observed in age or sex between the pneumonic and non-pneumonic conditions patients. Among the underlying diseases, cardiovascular diseases and CRAB bacterial infections were statistically noticeably increased in the pneumonic patients compared to those in the non-pneumonic conditions patients. WBC count, N-to-L ratio, PCT, and CRP were statistically significantly increased in the non-survivors and pneumonic patient groups compared to the survivors and non-pneumonic conditions patients.

Table 1

Table 1. Demographics and characteristics of patients with SARS-CoV-2 infection.

3.2 sIC regulators and CKs

Among sICs, soluble clusters of differentiation (sCD)27, sCD40, soluble herpes virus entry mediator (sHVEM), soluble T-cell immunoglobulin and mucin-domain containing-3 (sTIM-3), and soluble Toll-like receptor 2 (sTLR-2) were significantly increased in the non-survivors compared to those in the survivors. Soluble B-lymphocyte and T-lymphocyte attenuator (sBTLA) and soluble lymphocyte-activation gene 3 (sLAG-3) showed significantly lower values in the non-survivors compared to those in the survivors. Among CKs, chemokine CC motif ligand 2 (CCL2), granulocyte-macrophage colony-stimulating factor (GM-CSF), interleukin 6 (IL-6), IL-8, IL-10, IL-13, and tumor necrosis factor-α (TNF-α) were significantly increased in the non-survivors compared to those in the survivors. In the case of pneumonia, sBTLA and sLAG-3 among sICs were significantly lowered in the pneumonic patients compared to those in the non-pneumonic conditions patients, and sCD40 and sTIM-3 were statistically substantially increased in the pneumonic patients than those in the non-pneumonic conditions patients. CK, GM-CSF, and IL-6 levels were higher in pneumonic patients than in non-pneumonic conditions patients (Table 2, Figure 1). Figure 1 includes the top three markers (existing inflammatory markers, sICs, and CK) used in this study. These markers are ranked based on their P-value, with inclusion only when the P-value was significant at <0.05.

Table 2

Table 2. Levels of soluble type immune checkpoint regulators (sICs) and cytokines (CKs) in patients with SARS-CoV-2 infection .

Figure 1

Figure 1. Comparison of clinical markers, sICs, CKs in mild symptom (N=48), pneumonia (N=33), and death (N=19) in patients infected with SARS-CoV-2 by Mann-Whitney U test.

In correlation analysis showed that CRP has a positive correlation with sCD40 (r² = 0.414, P<0.0001), sHVEM (r² = 0.359, P=0.0002), TIM-3 (r² = 0.446, P<0.0004), while showing a significant negative correlation with sLAG-3 (r²=-0.367, P=0.0002). PCT also shows a statistically significant positive correlation with sCD40 (r² = 0.370, P=0.0014), sHVEM (r² = 0.318, P=0.0064), and sTIM-3 (r² = 0.397, P=0.0006). In case of CK, 11 CKs including CCL2, CCL4, GM-CSF, IL-1, IL-4, IL-6, IL-9, IL-10, IL-12, and TNF-a showed moderate correlation with CRP, and/or PCT (Supplementary Table 1).

3.3 Survival analysis

The AUC, cutoff value, sensitivity, and specificity of each indicator to discriminate non-survivors in patients with COVID-19 infection are presented in Table 3. The area under the curve for each indicator ranged from 0.652 to 0.844. IL-6 showed the highest AUC value (0.844, 95% CI = 0.751 – 0.913), with a sensitivity of 78.9% and specificity of 82.4% at a cut off 21.59 set by the Youden index (Table 3). When Kaplan-Meier 90-d survival analysis was performed, patients with factors as follows showed poor prognosis with statistical significance (Figure 2, Supplementary Figure 1): PCT over 0.25 ng/mL, CRP over 41.0 mg/dL, WBC over 6.32 × 10⁹/L, N-to-L ratio over 18.97, sBTLA below 153.86 pg/mL, sCD27 over 3828.8 pg/mL, sCD40 over 1283.61 pg/mL, sTIM-3 over 10230.28 pg/mL, sLAG-3 under 43475.3 pg/mL, sTLR-2 over 857.67 pg/mL, CCL-2 over 297.51 pg/mL, GM-CSF over 5.28 pg/mL, IL-6 over 21.59 pg/mL, IL-10 over 69.1 pg/mL, IL-13 over 22.05 pg/mL, and TNF-α over 7.65 pg/mL. Figure 2 includes items that demonstrated significance in the Kaplan-Meier analysis. The most important features identified in this analysis were further scrutinized in the decision tree analysis.

Table 3

Table 3. Area under the Receiver Operating Characteristic (ROC) curve, 95% confidence interval, cut-off point, sensitivity, specificity, p-value to discriminate non-survival in patients with COVID-19 infection.

Figure 2

Figure 2. Kaplan-Meier curves of soluble immune checkpoints (sICs) and cytokines for the overall survival in patients infected with SARS-CoV-2. The green and blue lines represent patient groups divided based on the Youden index for each item. The green line represents patients with an unfavorable prognosis in the Kaplan-Meier analysis, while the blue line indicates a patient group showing a relatively favorable prognosis. Increased level of sICs, including sCD27, sCD40, and sTIM-3, were significantly associated with poor prognosis in the log-rank test. Decreased level of sBTLA was correlated with poor prognosis. In case of cytokines, increased levels of IL-6 and IL-13 were associated with a poor prognosis.

K-means clustering was conducted and the optimal number was determined to be three. Among clusters 1, 2, and 3, cluster 2 exhibited the poorest prognosis, whereas cluster 3 showed the most favorable prognosis. Cluster 2 showed statistically significant final cluster centers with PCT 14.7 ng/mL, CRP 161.52 mg/dL, N-to-L ratio 43.6, sCD27 37248.92 pg/mL, sTIM-3 17070.16 pg/mL, sCD40 6520.67 pg/mL, GM-CSF 14.74 pg/mL, IL-6 1232.37 pg/mL, and TNF-α 17.32 pg/mL and these values corresponded to the highest among the three clusters (Supplementary Figure 2).

3.4 ML application for predicting prognosis

The enrolled individuals diagnosed with SARS-CoV-2 were analyzed utilizing decision tree analysis. Decision tree analysis was performed based on sICs and CKs, with significantly different levels between the survivors and non-survivors. The decision tree and feature importance are shown in Figure 3. According to these rules, IL-6 was the root node with the largest information gain when applying the built-in sklearn analysis. In the second layer of the leaf nodes, the sTIM-3 and sCD40 values were the classification standards. The conditions where IL-6 exceeded 22.68 and both sCD40 surpassed 1508.725 and sBTLA was below 315.745, were classified as non-survivors. Additionally, when the value of IL-6 was 22.68 or below, it was classified as survivors based on sTIM-3 being below 13015.83 and sCD27 below 927.965. The survival prediction AUC of the decision tree was 0.803, with an accuracy of 0.9. When the significance of each variable in the decision tree was analyzed using a Seaborn bar plot, IL-6, sTIM-3, sCD40, and sBTLA had a high impact on the model (Figure 3). Furthermore, we used the SHAP method to identify the most important features of the enrolled patients and visually explain how these variables affected the mortality rate. The top 15 most important features of this study and the SHAP values for each biomarker in the model output are presented in Figure 4. IL-6, sCD40, and CRP are the top three important biomarkers associated with a higher predicted 90-d mortality in patients diagnosed with SARS-CoV-2. In addition, sTLR-2, GM-CSF, TIM-3, CD27, CCL-2, TNF-α, PCT, N-to-L ratio, sBTLA, IL-8, IL-10, and IL-13 have been selected as significant top 15 features (Figure 4).

Figure 3

Figure 3. Decision tree analysis for predicting the survival in patients infected with SARS-CoV-2. (A) In each node, gini represented the quantified value of purity in the node, and a sample is the number of patients. The color demonstrate represents the class that encompasses the majority of samples within each node, and orange color for survivors and blue for non-survivors. In the prediction processing, at the root node, the patients were divided into two groups, with IL-6 value ≤22.68 or not. Then, the divided patients were classified by the second layer leaf node with sTIM-3 or sCD40. (B) The survival prediction AUC of the decision tree was 0.803 and (C) IL-6, sTIM-3, sCD40, sBTLA, and IL-13 were shown to be most important features among sICs and CKs.

Figure 4

Figure 4. The top 15 risk factors evaluated by the average absolute SHAP value. The SHAP value (x-axis) is a unified index that responds to the influence of a mortality in the model, and the feature ranking (y-axis) indicates the importance of the predictive model. All patients’ attributes for the outcome are plotted in rows, as colored dots, where red and blue dots represent the high risk values and low risk values, respectively.

3.5 Comparing paired samples between 1^st and 2^nd weeks with SARS-CoV-2 infection

Among all patients, the sICs and CK values in the 1^st and 2^nd weeks were compared pairwise in 48 patients (Table 4; Figure 5). Among the sICs, the sCD27, sCD28, sCD40, sHVEM, sLAG-3, and sTIM-3 levels were significantly higher in the 2^nd week than those in the initial week. Among the CKs, CCL2, CCL3, CCL4, IL-8, and IL-13 showed statistically significant increases in the 2^nd week in comparison with those in the 1^st week. On the other hand, IFN-α significantly decreased in the 2^nd week than that in the 1^st week. IL-6 did not show a significant statistical variation in values between the 1^st and 2^nd weeks, and it displayed increased values in the initial week (12.4 pg/mL, 95% CI: 5.01–32.0) as well as in the 2^nd week (15.4 pg/mL, 95% CI: 4.95–62.7). In addition, IL-10 and GM-CSF levels did not show statistical significance between the 1^st and 2^nd week. All sIC and CK data exhibiting a statistically significant difference between the values in the two weeks are included in the supplemental data (Supplementary Figure 3).

Table 4

Table 4. Comparison of differences in median values according to measurement time of paired samples using Wilcoxon-rank test.

Figure 5

Figure 5. Comparison of sIC and CK levels between the first and second weeks after SARS-CoV-2 infection by paired Wilcoxon-rank test. sCD27 showed median level of 4912.4 pg/mL in first week and increased to 6685.6 pg/mL in the second week (P = 0.0048). sCD40 showed median level of 986.7 pg/mL and 1181.6 pg/mL, in first and second weeks respectively (P = 0.0042). sHVEM (P = 0.0074), CCL2 (P = 0.0007), and IL-13 (P = 0.0008) also displayed significantly increased values at second week compared to those in the first week. IFN-α showed a significantly lower value in the second week (0.86 pg/mL) compared to that in the first week (4.40 pg/mL, P = 0.0105).

4 Discussion

In this study, several sICs and CKs were measured among those with SARS-CoV-2 diagnosed patients and the values in the survivors and non-survivors, and pneumonic and non-pneumonic conditions groups were compared. In addition, the clinical value of sICs and CKs was analyzed as additional markers useful for predicting prognosis along with existing clinical indicators. In addition to basic statistical analysis, ML analysis was used to screen factors of high importance. Assessing the significance of different biomarkers in predicting prognosis and assigning priority orders has considerable clinical relevance. During the extended period of SARS-CoV-2 infection, various studies have analyzed the prognostic prediction of several infection factors, including CRP, PCT, and CKs. However, few studies have comprehensively analyzed these markers and explored their relationships. Hence, in this study, the simultaneous measurement of various infection markers, sICs, and CKs was conducted, and the significance of each marker in predicting the prognosis of individuals affected by SARS-CoV-2 was analyzed.

WBC, N-to-L, PCT, and CRP, which have previously been noted to be significant in predicting the prognosis of patients diagnosed to SARS-CoV-2 (Ponti et al., 2020; Bedel and Korkut, 2021; Agarwal, 2022), were significantly higher in the non-survivors and pneumonic patients compared to those in the survivors and non-pneumonic conditions patients in this study. Among the sICs, sCD27, sCD40, sHVEM, and sTIM-3 showed considerably elevated values in the non-survivors than those in the survivors and sLAG-3 showed markedly reduced values in the non-survivors compared to those in the survivors. In case of CKs, CCL-2, GM-CSF, IL-6, IL-8, IL-10, IL-13, and TNF-alpha were statistically noticeably raised in the non-survivors compared to those in the survivors. All of these factors, except sLAG-3, showed worse prognosis in patients with elevated values than in those with decreased values in the Kaplan-Meier analysis. In the case of sLAG-3, the prognosis was worse in patients with lower values than in those with higher values. sICs play an important role in immune-mediated infection control by inducing immune responses that activate effector functions across diverse immune cells during viral infections (Bullock, 2017; Remedios et al., 2019; Deng et al., 2021; Joseph et al., 2021). Among the sICs, CD27 and CD40, which were identified as factors with high prognostic power through ML analysis in this study, play important roles in promoting the survival and effector functions of natural killer/T cells in viral infection, IFN-I response, and inhibition of its signaling (Yao et al., 2016; Bullock, 2017; Remedios et al., 2019; Deng et al., 2021). CD27 supports the initial activation and proliferation of T-cells, effectively enhancing their cytolytic and cytokine functions of cells (Bullock, 2017). The increase in sCD27 has been reported to play a crucial role in augmenting the appropriate immune cell response while inhibiting T cell proliferation (Penatzer et al., 2022). CD40, expressed on CD4+ T cells, also plays a significant role in stimulating T cells and promoting the formation of germinal centers and antibody class switching. Additionally, CD40 is associated with various cellular immune processes, such as T and B cell activation and apoptosis. It has been documented to function as a proinflammatory mediator in inflammatory diseases and may serve as an indicator for sepsis mortality (de Freitas E Silva and von Stebut, 2021; Liu et al., 2022). While we measured CD40, we could not include CD40L due to the unavailability of a commercial kit. CD40L, expressed on activated T cells, interacts with CD40 on various cell types, playing a crucial role in immune responses. The soluble form of CD40L (sCD40L) can exacerbate inflammation and contribute to the cytokine storm seen in severe SARS-CoV-2 infection (Hashem et al., 2014; Soong et al., 2014). Without CD40L data, our understanding of the CD40-CD40L interaction and its effects on immune activation is incomplete. Future studies should measure both CD40 and CD40L to fully elucidate their roles in immune modulation and severe COVID-19 outcomes. Including these measurements will provide deeper insights into the pathophysiology of the disease and inform better therapeutic strategies. Furthermore, a recent study investigating immune checkpoint molecules and the expression of CD39 demonstrated that the co-expression of TIM-3 and CD39 was a significant predictor of severe COVID-19 (Gambichler et al., 2024). These cellular expressions could serve as important prognostic markers.

Additionally, TIM-3 and LAG-3 are indicators of CD8+ T cell exhaustion and are associated with the overexpression of immune inhibitory factors due to T cell exhaustion during SARS-CoV-2 infection (Wherry and Kurachi, 2015; Diao et al., 2020). Through this study, which expanded the number of patients infected with SARS-CoV-2, we were able to confirm that sICs, such as CD27, CD40, and TIM-3, have survival prediction significance. These results are expected to provide supporting evidence for the potential use of sICs as diagnostic and prognostic factors for viral infections, including those caused by SARS-CoV-2. One important consideration is that the sample collection period for this study coincided with the subsequently emerging omicron variant, which has higher transmissibility than the delta variant, is associated with lower clinical severity (Shrestha et al., 2022; Wolf et al., 2023). SARS-CoV-2 infection evolves depending on the timing of infection, exhibiting different host innate immune responses for each variant with distinct characteristics (Wolf et al., 2023). The values of sICs and CKs analyzed in this study may also show differences in quantification compared to those of patients collected during different infection periods. Therefore, further research is required to longitudinally compare the sICs and CKs values for each time period. Furthermore, the 14 and 50 patients included in this paper have cancer and hypertension respectively, as an underlying disease. We compared the sICs values based on cancer and/or hypertension status to exclude the impact of their sICs values on the analysis of COVID-19. Statistical analysis using the Mann-Whitney test showed no significant differences in any of the sICs. Although the clinical utility of sICs is currently limited due to constraints such as testing costs and turnaround time, it is anticipated that the use of sICs in clinical practice for viral infections as disease progression and/or prediction markers will expand in the future.

In this study, ML was used to more clearly and comprehensively investigate the clinical significance and prognostic impact of these markers. To the best of our knowledge, only a few studies have discriminated the significance of sICs and CKs using ML. Various artificial intelligence algorithms, including ML, are expected to be used in various ways as decision-making tools in the medical field (Wang K. et al., 2021; Wang L. et al., 2021; Feng et al., 2022; Hu et al., 2022; Shehab et al., 2022; Sorayaie Azar et al., 2022). In this study, we utilized K-means clustering to handle high-dimensional data, simplifying the analysis and further stratifying patient cohorts. This approach indirectly enables the identification of the most informative markers, as the centroids of the clusters represent the average characteristics of the data points within each cluster (Shehab et al., 2022). Moreover, the ML algorithms DT and SHAP were used as auxiliary prediction tools for patients diagnosed with SARS-CoV-2 infection. DT is a supervised learning method that can predict and classify large datasets using simple logic (Lundberg et al., 2020; Sorayaie Azar et al., 2022; International BR, 2023). SHAP also provides consistent interpretability, helps elucidate the decision-making process, and determines the significance of each feature in the prediction (The Lancet Respiratory M, 2018; Watson et al., 2019). ML analysis has the advantage of not only accurately classifying and predicting specimens through learning from observations and logic, but also enabling decision-making, excluding subjective factors. In this study, IL-6 showed the highest importance in predicting mortality using both the DT and SHAP methods. This aligns with the outcomes of the ROC curve analysis, which showed that IL-6 had the highest AUC value among all factors, including existing infection markers such as CRP and PCT. It has been reported that IL-6 induces disease, is related to CK storm, and is helpful in predicting survival prognosis (Copaescu et al., 2020; Gubernatorova et al., 2020; Majidpoor and Mortezaee, 2022). The results of this study, which included a larger number of individuals with SARS-CoV-2 infection than previous studies and utilized both ML and performance analyses, further support the value of IL-6 in clinical practice. Additionally, through DT and SHAP, it was observed that sCD27, sCD40, and sTIM-3 among sICs and IL-13 and TNF-alpha among CKs were highly important in survival prediction. In the future, for the prediction of mortality from SARS-CoV-2 infection, the use of sICs and CKs, in addition to existing infection markers, could be helpful. IL-6 is expected to be the most effective marker for survival prediction.

In addition, in this study, it was possible to perform a paired t-test analysis on the difference in values between the 1^st and 2^nd weeks in the expanded patient group and discriminate sICs and/or CKs that significantly increased or decreased in the second week. sICs, such as sCD27, sCD40, aTIM-3, and sLAG-3, and CKs, such as IL-13, which were found to have a high correlation with prognosis in this study, showed significant differences over time. In contrast, IL-6 levels did not show a significant difference between the first and second weeks, and increased to high levels even in the early stages of infection. IL-6 has been previously reported as an early increase marker (Waje-Andreassen et al., 2005), and is widely recognized for its pivotal role in severe inflammatory responses, including the cytokine storm (Grebenciucova and VanHaerents, 2023). Additionally, sCD27 and sCD40 have been shown to play significant roles in immune responses in conditions such as sepsis and viral infections (Lee et al., 2022), while sTIM-3 has been linked to T cell exhaustion, particularly in chronic viral infections and cancers (Paranga et al., 2024). However, despite the substantial research on their clinical relevance, there are limited reports on the temporal dynamics of IL-6 elevation during SARS-CoV-2 infection. The results of this study suggest that IL-6 is a valuable prognostic marker that increases significantly even in the early stages of infection.

The limitations of this study were as follows. First, it was affected by a selection bias owing to its retrospective nature. If the predictive model is validated in a multicenter cohort in future follow-up studies, the predictive ability of the ML algorithm is expected to become more reliable. Second, as a comparison with healthy controls was not possible for the measurement of sICs and CKs, a more objective evaluation would be possible if additional comparisons with healthy controls were conducted in future research. In addition, most non-survivors in this study (16 of 19) were classified as pneumonia-related deaths and the analysis of the prognosis among patients with pneumonia was limited compared to that of deceased patients. Furthermore, the overall number of non-survivors is relatively low compared to the number of survivors. This disparity may be partially attributed to the clinical manifestations associated with the Omicron variant. It is necessary to conduct follow-up studies involving more non-survivor patients. Third, we aimed to elucidate the relationship between co-infection with viruses, bacteria, and fungi in COVID-19 and sIC. However, it was challenging to determine statistical significance because of the limited number of patients infected with multidrug-resistant bacteria. We intend to continue this research in follow-up studies.

In conclusion, this study simultaneously measured various sIC and CK levels along with previous infection markers in hospitalized patients with SARS-CoV-2 infection and designed a process that could be used in actual clinical practice by utilizing the ML algorithm. sICs and CKs are expected to be more actively used in the future for the prognostic prediction of viral infections, including SARS-CoV-2. This study is expected to be helpful as a basis for the appropriate selection of factors for use as markers in medical practice among the numerous types of sICs and CKs in patients with severe form of SARS-CoV-2. Further research is essential, particularly increasing the sample size to enhance the study’s robustness and statistical power. Larger cohorts will improve reliability and allow precise evaluation of the clinical utility of various sICs and CKs, advancing prognostic predictions for viral infections in clinical practice.

Data availability statement

The datasets presented in this study can be found in online repositories. The names of the repository/repositories and accession number(s) can be found below: Harvard Dataverse https://doi.org/10.7910/DVN/12D49O.

Ethics statement

The studies involving humans were approved by Institutional Review Board of Hallym University Kangnam Sacred Heart Hospital. The studies were conducted in accordance with the local legislation and institutional requirements. The ethics committee/institutional review board waived the requirement of written informed consent for participation from the participants or the participants’ legal guardians/next of kin because Patients’ personal information remained anonymous.

Author contributions

NL: Conceptualization, Formal Analysis, Writing – original draft. KJ: Methodology, Writing – review & editing. M-JP: Methodology, Writing – review & editing. WS: Supervision, Writing – review & editing. SJ: Conceptualization, Data curation, Funding acquisition, Writing – review & editing.

Funding

The author(s) declare that financial support was received for the research, authorship, and/or publication of this article. This research was supported by Hallym University Research Fund 2022 (HURF-2022-37).

Acknowledgments

The authors appreciate medical technicians for collecting samples to perform the experiments. We would like to thank Editage (www.editage.co.kr) for English language editing.

Conflict of interest

The authors declare that this study was conducted in the absence of any commercial or financial relationships that could be construed as potential conflicts of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fcimb.2024.1397297/full#supplementary-material

Abbreviations

BTLA, B- and T-lymphocyte attenuator; CD, cluster of differentiation; CTLA-4, cytotoxic T-lymphocyte-associated protein 4; GITR, glucocorticoid-induced TNFR-related protein; GITRL, ligand for receptor TNFRSF18/AITR/GITR; HVEM, herpes virus entry mediator; ICOS, inducible T-cell costimulator; LAG-3, lymphocyte-activation gene 3; PD-1, programmed cell death protein 1; PD-L1, programmed death-ligand 1; TIM-3, T-cell immunoglobulin and mucin-domain containing-3; TLR-2, Toll-like receptor 2; CCL, chemokine CC motif ligand; CXCL, C-X-C motif chemokine ligand; GM-CSF, Granulocyte-macrophage colony-stimulating factor; IFN, Interferon; IL, Interleukin; TNF, tumor necrosis factor.

References

Agarwal, S. (2022). Neutrophil-lymphocyte ratio predicting case severity in SARS-CoV-2 infection: a review. Cureus 14, e29760. doi: 10.7759/cureus.29760

PubMed Abstract | Crossref Full Text | Google Scholar

Aghbash, P. S., Eslami, N., Shamekh, A., Entezari-Maleki, T., Baghi, H. B. (2021). SARS-CoV-2 infection: the role of PD-1/PD-L1 and CTLA-4 axis. Life Sci. 270, 119124. doi: 10.1016/j.lfs.2021.119124

PubMed Abstract | Crossref Full Text | Google Scholar

Awadasseid, A., Yin, Q., Wu, Y., Zhang, W. (2021). Potential protective role of the anti-PD-1 blockade against SARS-CoV-2 infection. BioMed. Pharmacother. 142, 111957. doi: 10.1016/j.biopha.2021.111957

PubMed Abstract | Crossref Full Text | Google Scholar

Bedel, C., Korkut, M. (2021). Neutrophil-to-lymphocyte ratio and COVID-19. Shock 56, 874. doi: 10.1097/shk.0000000000001712

PubMed Abstract | Crossref Full Text | Google Scholar

Bullock, T. N. (2017). Stimulating CD27 to quantitatively and qualitatively shape adaptive immunity to cancer. Curr. Opin. Immunol. 45, 82–88. doi: 10.1016/j.coi.2017.02.001

PubMed Abstract | Crossref Full Text | Google Scholar

Cai, H., Liu, G., Zhong, J., Zheng, K., Xiao, H., Li, C., et al. (2020). Immune checkpoints in viral infections. Viruses 12, 1051. doi: 10.3390/v12091051

PubMed Abstract | Crossref Full Text | Google Scholar

Carabelli, A. M., Peacock, T. P., Thorne, L. G., Harvey, W. T., Hughes, J., Peacock, S. J., et al. (2023). SARS-CoV-2 variant biology: immune escape, transmission and fitness. Nat. Rev. Microbiol. 21, 162–177. doi: 10.1038/s41579-022-00841-7

PubMed Abstract | Crossref Full Text | Google Scholar

Copaescu, A., Smibert, O., Gibson, A., Phillips, E. J., Trubiano, J. A. (2020). The role of IL-6 and other mediators in the cytokine storm associated with SARS-CoV-2 infection. J. Allergy Clin. Immunol. 146, 518–34.e1. doi: 10.1016/j.jaci.2020.07.001

PubMed Abstract | Crossref Full Text | Google Scholar

de Freitas E Silva, R., von Stebut, E. (2021). Unraveling the role of immune checkpoints in leishmaniasis. Front. Immunol. 12. doi: 10.3389/fimmu.2021.620144

PubMed Abstract | Crossref Full Text | Google Scholar

Deng, Y., Chatterjee, B., Zens, K., Zdimerova, H., Müller, A., Schuhmachers, P., et al. (2021). CD27 is required for protective lytic EBV antigen-specific CD8+ T-cell expansion. Blood 137, 3225–3236. doi: 10.1182/blood.2020009482

PubMed Abstract | Crossref Full Text | Google Scholar

Diao, B., Wang, C., Tan, Y., Chen, X., Liu, Y., Ning, L., et al. (2020). Reduction and functional exhaustion of T cells in patients with coronavirus disease 2019 (COVID-19). Front. Immunol. 11. doi: 10.3389/fimmu.2020.00827

PubMed Abstract | Crossref Full Text | Google Scholar

Feng, Y., Wang, Z., Cui, R., Xiao, M., Gao, H., Bai, H., et al. (2022). Clinical analysis and artificial intelligence survival prediction of serous ovarian cancer based on preoperative circulating leukocytes. J. Ovarian Res. 15, 64. doi: 10.1186/s13048-022-00994-2

PubMed Abstract | Crossref Full Text | Google Scholar

Gambichler, T., Reuther, J., Scheel, C. H., Becker, J. C. (2020). On the use of immune checkpoint inhibitors in patients with viral infections including COVID-19. J. Immunother. Cancer 8, 1145. doi: 10.1136/jitc-2020-001145

PubMed Abstract | Crossref Full Text | Google Scholar

Gambichler, T., Rüth, J., Goesmann, S., Höxtermann, S., Skrygan, M., Susok, L., et al. (2024). A prospective study investigating immune checkpoint molecule and CD39 expression on peripheral blood cells for the prognostication of COVID-19 severity and mortality. Viruses 16, 810. doi: 10.3390/v16050810

PubMed Abstract | Crossref Full Text | Google Scholar

Gatto, L., Franceschi, E., Nunno, V. D., Brandes, A. A. (2020). Potential protective and therapeutic role of immune checkpoint inhibitors against viral infections and COVID-19. Immunotherapy 12, 1111–1114. doi: 10.2217/imt-2020-0109

PubMed Abstract | Crossref Full Text | Google Scholar

Grebenciucova, E., VanHaerents, S. (2023). Interleukin 6: at the interface of human health and disease. Front. Immunol. 14. doi: 10.3389/fimmu.2023.1255533

PubMed Abstract | Crossref Full Text | Google Scholar

Gubernatorova, E. O., Gorshkova, E. A., Polinova, A. I., Drutskaya, M. S. (2020). IL-6: relevance for immunopathology of SARS-CoV-2. Cytokine Growth Factor. Rev. 53, 13–24. doi: 10.1016/j.cytogfr.2020.05.009

PubMed Abstract | Crossref Full Text | Google Scholar

Han, H., Ma, Q., Li, C., Liu, R., Zhao, L., Wang, W., et al. (2020). Profiling serum cytokines in COVID-19 patients reveals IL-6 and IL-10 are disease severity predictors. Emerg. Microbes Infect. 9, 1123–1130. doi: 10.1080/22221751.2020.1770129

PubMed Abstract | Crossref Full Text | Google Scholar

Hashem, A. M., Gravel, C., Chen, Z., Yi, Y., Tocchi, M., Jaentschke, B., et al. (2014). CD40 ligand preferentially modulates immune response and enhances protection against influenza virus. J. Immunol. 193, 722–734. doi: 10.4049/jimmunol.1300093

PubMed Abstract | Crossref Full Text | Google Scholar

Hoyos-Rivera, G. J., Gomes, R., Willrich, R., Courtiat, J. P. (2006). Colab: a new paradigm and tool for collaboratively browsing the web. IEEE Trans. Syst. Man. Cybern. Part A. Syst. Hum. 36, 1074–1085. doi: 10.1109/TSMCA.2006.883173

Crossref Full Text | Google Scholar

Hu, C., Li, L., Huang, W., Wu, T., Xu, Q., Liu, J., et al. (2022). Interpretable machine learning for early prediction of prognosis in sepsis: a discovery and validation study. Infect. Dis. Ther. 11, 1117–1132. doi: 10.1007/s40121-022-00628-6

PubMed Abstract | Crossref Full Text | Google Scholar

International BR (2023). Retracted: processing decision tree data using internet of things (IoT) and artificial intelligence technologies with special reference to medical application. BioMed. Res. Int. 2023, 9762317. doi: 10.1155/2023/9762317

PubMed Abstract | Crossref Full Text | Google Scholar

Jeong, S., Lee, N., Park, Y., Kim, J., Jeon, K., Park, M. J., et al. (2022). Prevalence and clinical impact of coinfection in patients with coronavirus disease 2019 in Korea. Viruses 14, 446. doi: 10.3390/v14020446

PubMed Abstract | Crossref Full Text | Google Scholar

Joseph, J., Rahmani, B., Cole, Y., Puttagunta, N., Lin, E., Khan, Z. K., et al. (2021). Can soluble immune checkpoint molecules on exosomes mediate inflammation? J. Neuroimmune. Pharmacol. 17, 381–397. doi: 10.1007/s11481-021-10018-3

PubMed Abstract | Crossref Full Text | Google Scholar

Kermali, M., Khalsa, R. K., Pillai, K., Ismail, Z., Harky, A. (2020). The role of biomarkers in diagnosis of COVID-19 - a systematic review. Life Sci. 254, 117788. doi: 10.1016/j.lfs.2020.117788

PubMed Abstract | Crossref Full Text | Google Scholar

Lee, N., Jeong, S., Jeon, K., Park, M. J., Song, W. (2022). Prognostic impacts of soluble immune checkpoint regulators and cytokines in patients with SARS-CoV-2 infection. Front. Immunol. 13. doi: 10.3389/fimmu.2022.903419

PubMed Abstract | Crossref Full Text | Google Scholar

Lim, S. P. (2023). Targeting SARS-CoV-2 and host cell receptor interactions. Antiviral Res. 210, 105514. doi: 10.1016/j.antiviral.2022.105514

PubMed Abstract | Crossref Full Text | Google Scholar

Liu, Y. C., Shou, S. T., Chai, Y. F. (2022). Immune checkpoints in sepsis: new hopes and challenges. Int. Rev. Immunol. 41, 207–216. doi: 10.1080/08830185.2021.1884247

PubMed Abstract | Crossref Full Text | Google Scholar

Lundberg, S. M., Erion, G., Chen, H., DeGrave, A., Prutkin, J. M., Nair, B., et al. (2020). From local explanations to global understanding with explainable AI for trees. Nat. Mach. Intell. 2, 56–67. doi: 10.1038/s42256-019-0138-9

PubMed Abstract | Crossref Full Text | Google Scholar

Majidpoor, J., Mortezaee, K. (2022). Interleukin-6 in SARS-CoV-2 induced disease: interactions and therapeutic applications. BioMed. Pharmacother. 145, 112419. doi: 10.1016/j.biopha.2021.112419

PubMed Abstract | Crossref Full Text | Google Scholar

Marin-Acevedo, J. A., Dholaria, B., Soyano, A. E., Knutson, K. L., Chumsri, S., Lou, Y. (2018). Next generation of immune checkpoint therapy in cancer: new developments and challenges. J. Hematol. Oncol. 11, 39. doi: 10.1186/s13045-018-0582-8

PubMed Abstract | Crossref Full Text | Google Scholar

Markov, P. V., Ghafari, M., Beer, M., Lythgoe, K., Simmonds, P., Stilianakis, N. I., et al. (2023). The evolution of SARS-coV-2. Nat. Rev. Microbiol. 21, 361–379. doi: 10.1038/s41579-023-00878-2

PubMed Abstract | Crossref Full Text | Google Scholar

Morrell, E. D., Bhatraju, P. K., Sathe, N. A., Lawson, J., Mabrey, L., Holton, S. E., et al. (2022). Chemokines, soluble PD-L1, and immune cell hyporesponsiveness are distinct features of SARS-CoV-2 critical illness. Am. J. Physiol. Lung Cell Mol. Physiol. 323, L14–L26,. doi: 10.1152/ajplung.00049.2022

PubMed Abstract | Crossref Full Text | Google Scholar

Nesteruk, I. (2023). Endemic characteristics of SARS-CoV-2 infection. Sci. Rep. 13, 14841. doi: 10.1038/s41598-023-41841-8

PubMed Abstract | Crossref Full Text | Google Scholar

Nguyen, N. N., Nguyen, Y. N., Hoang, V. T., Million, M., Gautret, P. (2023). Sars-Cov-2 reinfection and severity of the disease: a systematic review and meta-analysis. Viruses 15, 967. doi: 10.3390/v15040967

PubMed Abstract | Crossref Full Text | Google Scholar

Paranga, T. G., Pavel-Tanasa, M., Constantinescu, D., Iftimi, E., Plesca, C. E., Miftode, I. L., et al. (2024). Distinct soluble immune checkpoint profiles characterize COVID-19 severity, mortality and SARS-CoV-2 variant infections. Front. Immunol. 15. doi: 10.3389/fimmu.2024.1464480

PubMed Abstract | Crossref Full Text | Google Scholar

Penatzer, J. A., Alexander, R., Simon, S., Wolfe, A., Breuer, J., Hensley, J., et al. (2022). Early detection of soluble CD27, BTLA, and TIM-3 predicts the development of nosocomial infection in pediatric burn patients. Front. Immunol. 13. doi: 10.3389/fimmu.2022.940835

PubMed Abstract | Crossref Full Text | Google Scholar

Ponti, G., Maccaferri, M., Ruini, C., Tomasi, A., Ozben, T. (2020). Biomarkers associated with COVID-19 disease progression. Crit. Rev. Clin. Lab. Sci. 57, 389–399. doi: 10.1080/10408363.2020.1770685

PubMed Abstract | Crossref Full Text | Google Scholar

Puhach, O., Meyer, B., Eckerle, I. (2023). SARS-CoV-2 viral load and shedding kinetics. Nat. Rev. Microbiol. 21, 147–161. doi: 10.1038/s41579-022-00822-w

PubMed Abstract | Crossref Full Text | Google Scholar

REMAP-CAP Investigators (2021). Interleukin-6 receptor antagonists in critically ill patients with COVID-19. N. Engl. J. Med. 384, 1491–1502. doi: 10.1056/NEJMoa2100433

PubMed Abstract | Crossref Full Text | Google Scholar

Remedios, K. A., Meyer, L., Zirak, B., Pauli, M. L., Truong, H.-A., Boda, D., et al. (2019). CD27 promotes CD4(+) effector T cell survival in response to tissue self-antigen. J. Immunol. 203, 639–646. doi: 10.4049/jimmunol.1900288

PubMed Abstract | Crossref Full Text | Google Scholar

Shehab, M., Abualigah, L., Shambour, Q., Abu-Hashem, M. A., Shambour, M. K. Y., Alsalibi, A. I., et al. (2022). Machine learning in medical applications: a review of state-of-the-art methods. Comput. Biol. Med. 145, 105458. doi: 10.1016/j.compbiomed.2022.105458

PubMed Abstract | Crossref Full Text | Google Scholar

Shrestha, L. B., Foster, C., Rawlinson, W., Tedla, N., Bull, R. A. (2022). Evolution of the SARS-CoV-2 omicron variants BA.1 to BA.5: implications for immune escape and transmission. Rev. Med. Virol. 32, e2381. doi: 10.1002/rmv.2381

PubMed Abstract | Crossref Full Text | Google Scholar

Soong, R. S., Song, L., Trieu, J., Lee, S. Y., He, L., Tsai, Y. C., et al. (2014). Direct T cell activation via CD40 ligand generates high avidity CD8+ T cells capable of breaking immunological tolerance for the control of tumors. PloS One 9, e93162. doi: 10.1371/journal.pone.0093162

PubMed Abstract | Crossref Full Text | Google Scholar

Sorayaie Azar, A., Babaei Rikan, S., Naemi, A., Bagherzadeh Mohasefi, J., Pirnejad, H., Bagherzadeh Mohasefi, M., et al. (2022). Application of machine learning techniques for predicting survival in ovarian cancer. BMC Med. Inform Decis. Mak. 22, 345. doi: 10.1186/s12911-022-02087-y

PubMed Abstract | Crossref Full Text | Google Scholar

Sun, X., Gao, C., Zhao, K., Yang, Y., Rassadkina, Y., Fajnzylber, J., et al. (2022). Immune-profiling of SARS-CoV-2 viremic patients reveals dysregulated innate immune responses. Front. Immunol. 13. doi: 10.3389/fimmu.2022.984553

PubMed Abstract | Crossref Full Text | Google Scholar

The Lancet Respiratory M (2018). Opening the black box of machine learning. Lancet Respir. Med. 6, 801. doi: 10.1016/s2213-2600(18)30425-9

PubMed Abstract | Crossref Full Text | Google Scholar

Waje-Andreassen, U., Kråkenes, J., Ulvestad, E., Thomassen, L., Myhr, K. M., Aarseth, J., et al. (2005). IL-6: an early marker for outcome in acute ischemic stroke. Acta Neurol. Scand. 111, 360–365. doi: 10.1111/j.1600-0404.2005.00416.x

PubMed Abstract | Crossref Full Text | Google Scholar

Wang, K., Tian, J., Zheng, C., Yang, H., Ren, J., Liu, Y., et al. (2021). Interpretable prediction of 3-year all-cause mortality in patients with heart failure caused by coronary heart disease based on machine learning and SHAP. Comput. Biol. Med. 137, 104813. doi: 10.1016/j.compbiomed.2021.104813

PubMed Abstract | Crossref Full Text | Google Scholar

Wang, L., Zhu, L., Jiang, J., Wang, L., Ni, W. (2021). Decision tree analysis for evaluating disease activity in patients with rheumatoid arthritis. J. Int. Med. Res. 49, 3000605211053232. doi: 10.1177/03000605211053232

PubMed Abstract | Crossref Full Text | Google Scholar

Watson, D. S., Krutzinna, J., Bruce, I. N., Griffiths, C. E., McInnes, I. B., Barnes, M. R., et al. (2019). Clinical applications of machine learning algorithms: beyond the black box. BMJ 364, l886. doi: 10.1136/bmj.l886

PubMed Abstract | Crossref Full Text | Google Scholar

Wherry, E. J., Kurachi, M. (2015). Molecular and cellular insights into T cell exhaustion. Nat. Rev. Immunol. 15, 486–499. doi: 10.1038/nri3862

PubMed Abstract | Crossref Full Text | Google Scholar

Wolf, J. M., Wolf, L. M., Bello, G. L., Maccari, J. G., Nasi, L. A. (2023). Molecular evolution of SARS-coV-2 from December 2019 to August 2022. J. Med. Virol. 95, e28366. doi: 10.1002/jmv.28366

PubMed Abstract | Crossref Full Text | Google Scholar

Yao, X., Wu, J., Lin, M., Sun, W., He, X., Gowda, C., et al. (2016). Increased CD40 expression enhances early STING-mediated type I interferon response and host survival in a rodent malaria model. PloS Pathog. 12, e1005930. doi: 10.1371/journal.ppat.1005930

PubMed Abstract | Crossref Full Text | Google Scholar

Yu, S., Hu, H., Ai, Q., Bai, R., Ma, K., Zhou, M., et al. (2023). SARS-CoV-2 spike-mediated entry and its regulation by host innate immunity. Viruses 15, 639. doi: 10.3390/v15030639

PubMed Abstract | Crossref Full Text | Google Scholar

Zahavi, D. J., Weiner, L. M. (2019). Targeting multiple receptors to increase checkpoint blockade efficacy. Int. J. Mol. Sci. 20, 158. doi: 10.3390/ijms20010158

PubMed Abstract | Crossref Full Text | Google Scholar