Comparison between traditional logistic regression and machine learning for predicting mortality in adult sepsis patients

Background: Sepsis is a life-threatening disease associated with a high mortality rate, emphasizing the need for the exploration of novel models to predict the prognosis of this patient population. This study compared the performance of traditional logistic regression and machine learning models in predicting adult sepsis mortality.

Objective: To develop an optimum model for predicting the mortality of adult sepsis patients based on comparing traditional logistic regression and machine learning methodology.

Methods: Retrospective analysis was conducted on 606 adult sepsis inpatients at our medical center between January 2020 and December 2022, who were randomly divided into training and validation sets in a 7:3 ratio. Traditional logistic regression and machine learning methods were employed to assess the predictive ability of mortality in adult sepsis. Univariate analysis identified independent risk factors for the logistic regression model, while Least Absolute Shrinkage and Selection Operator (LASSO) regression facilitated variable shrinkage and selection for the machine learning model. Among various machine learning models, which included Bagged Tree, Boost Tree, Decision Tree, LightGBM, Naïve Bayes, Nearest Neighbors, Support Vector Machine (SVM), and Random Forest (RF), the one with the maximum area under the curve (AUC) was chosen for model construction. Model validation and comparison with the Sequential Organ Failure Assessment (SOFA) and the Acute Physiology and Chronic Health Evaluation (APACHE) scores were performed using receiver operating characteristic (ROC) curves, calibration curves, and decision curve analysis (DCA) curves in the validation set.

Results: Univariate analysis was employed to assess 17 variables, namely gender, history of coronary heart disease (CHD), systolic pressure, white blood cell (WBC), neutrophil count (NEUT), lymphocyte count (LYMP), lactic acid, neutrophil-to-lymphocyte ratio (NLR), red blood cell distribution width (RDW), interleukin-6 (IL-6), prothrombin time (PT), international normalized ratio (INR), fibrinogen (FBI), D-dimer, aspartate aminotransferase (AST), total bilirubin (Tbil), and lung infection. Significant differences (p < 0.05) between the survival and non-survival groups were observed for these variables. Utilizing stepwise regression with the “backward” method, independent risk factors, including systolic pressure, lactic acid, NLR, RDW, IL-6, PT, and Tbil, were identified. These factors were then incorporated into a logistic regression model, chosen based on the minimum Akaike Information Criterion (AIC) value (98.65). Machine learning techniques were also applied, and the RF model, demonstrating the maximum Area Under the Curve (AUC) of 0.999, was selected. LASSO regression, employing the lambda.1SE criteria, identified systolic pressure, lactic acid, NEUT, RDW, IL6, INR, and Tbil as variables for constructing the RF model, validated through ten-fold cross-validation. For model validation and comparison with traditional logistic models, SOFA, and APACHE scoring.

Conclusion: Based on deep machine learning principles, the RF model demonstrates advantages over traditional logistic regression models in predicting adult sepsis prognosis. The RF model holds significant potential for clinical surveillance and interventions to enhance outcomes for sepsis patients.

Introduction

Sepsis represents a critical condition marked by organ dysfunction resulting from an imbalanced host response to infection, leading to high mortality rates and substantial healthcare costs (1, 2). Despite the establishment of the initial consensus definitions (Sepsis-1) in 1991, the global incidence of sepsis continues to rise, making the true epidemiology of sepsis a subject of ongoing concern. Further exploration of high-risk factors associated with sepsis-related mortality is essential (3). In clinical settings, the evaluation of sepsis severity and the identification of risk factors for mortality often rely on scoring systems such as Sequential Organ Failure Assessment (SOFA), Quick Sequential Organ Failure Assessment (qSOFA), and Acute Physiology and Chronic Health Evaluation (APACHE) (4–6). However, these scoring systems involve numerous parameters, posing challenges for clinical practitioners. Consequently, there has been a growing interest in exploring the effectiveness of biomarkers and clinical prediction models in predicting the prognosis of sepsis patients (7–9).

Over the past few years, linear regression models have dominated the clinical landscape for predicting sepsis mortality (10, 11). However, their limitations, including the inability to handle non-linearity among variables, sensitivity to outlier values, and the need to meet the linear regression hypothesis, constrain their utility with non-linear and imbalanced datasets (12). Machine learning (ML) is a subfield of artificial intelligence (AI) that focuses on developing systems capable of learning from data or improving performance. Specifically, machine learning is a technique that enables computers to create models by training algorithms using datasets (13). Previous studies had indicated that ML models play a crucial role in predicting the prognosis of sepsis patients. ML models had demonstrated superior predictive accuracy compared to traditional statistical methods. By leveraging complex algorithms, these models can identify non-linear relationships and interactions between variables that may be overlooked by simpler models, leading to more precise predictions of sepsis mortality (14, 15). On the other hand, the key strengths of ML models is their ability to handle high-dimensional data effectively. They can incorporate a vast array of clinical variables, which allows for a more comprehensive understanding of the patient’s condition and the factors contributing to mortality risk (16, 17). Consequently, these above benefits position ML as an essential tool in the prediction of sepsis mortality, aiding in the improvement of clinical decision-making and patient outcomes.

Based on the methodological review mentioned above, we employed both traditional generalized linear regression and ML models to assess their predictive capabilities in adult sepsis’s mortality during their hospitalization duration. Notably, we conducted internal validation for both models and compared their performance with SOFA and APACHE scores in terms of discrimination, calibration, and clinical practicality. This comprehensive analysis offers profound insights into mortality risk adjustment for observational adult sepsis datasets, contributing valuable information to the understanding of predictive models and their applicability in clinical settings. This study closely complied with TRIPOD guidelines (18) and the PROBAST risk of bias tool (19).

Methods

Source of clinical data

The clinical data for this cross-sectional study were obtained from the electronic medical records of Huadu District People’s Hospital of Guangzhou, Southern Medical University. The study focused on adult patients diagnosed with sepsis during hospitalization from January 2020 to December 2022, adhering to the Sepsis-3 definition (20, 21). Exclusion criteria included patients under 18 years old, those with malignant tumors, individuals with immunosuppression, those who died or withdrew treatment within 24 h of admission, and cases where clinical data could not be extracted. Following these criteria, 606 cases of adult sepsis were included in the study. Due to it directly reflects the sepsis patient’s survival and is a key performance indicator during the hospitalization duration, we define the mortality as the outcome of this study.

Variables extraction

Variable extraction involved retrieving general information (gender, age, and body mass index), medical history [hypertension, diabetes, and coronary heart disease (CHD)], clinical signs (temperature, heart rate, systolic pressure, and infection site), laboratory examination results [white blood cell count (WBC), platelet count, neutrophil (NEUT) and lymphocyte (LYMP) counts, neutrophil-to-lymphocyte ratio (NLR), red cell distribution width (RDW), C-reactive protein, procalcitonin, lactic acid, prothrombin time (PT), international normalized ratio (INR), fibrinogen (FIB), D-dimer, creatinine, alanine transaminase (ALT), aspartate transaminase (AST), total bilirubin (Tbil), and interleukin-6 (IL-6)], etiologic detection (Gram-positive bacteria, Gram-negative bacteria, or fungal), and severity scores of sepsis (SOFA and APACHE) from the electronic medical record system. All data were extracted within the first 24 h of patient admission. For missing values, multiple imputation was performed using the “mice” package in R software.

Model construction of logistic regression

The study divided the 606 adult sepsis cases randomly into a training set (n = 435) and a validation set (n = 171) at a ratio of 7:3. Based on whether the patient died or not between 24 h after admission and discharge, participants were categorized into a survival group (421) and a non-survival group (185). For traditional logistic regression model construction, univariate analysis identified significant risk factors (p < 0.05), which were then included in binary logistic regression. The stepwise regression with the “backward” method was employed to achieve the optimal model with the least AIC value.

Machine learning model selection and construction

For ML model selection, eight integrated algorithms, including Bagged Tree, Boost Tree, Decision Tree, LightGBM, Naïve Bayes, Nearest Neighbors, Support SVM, and RF, were considered. In the “tidymodels” framework of R software, workflow sets were used to compare these models, perform resampling, and tune parameters. Because of its ability to provide a comprehensive measure of a model’s performance across all classification thresholds, we select AUC as an optimum index in order to offer more nuanced view of model performance. The ML model with the highest AUC value was chosen for model construction. In order to perform variable shrinkage and selection, which may avoid the overfitting of the ML model, we utilized LASSO regression with ten-fold cross-validation. The count of variables in the ultimate model was ascertained based on the specific location of lambda.1SE, a coefficient that signifies the ideal equilibrium between model intricacy and forecasting accuracy.

Models validation and comparison

Models were validated and compared using discrimination, calibration, clinical benefit, and generalization. Discrimination was assessed by calculating the AUC of the ROC, while calibration was evaluated using calibration curves and the Hosmer-Lemeshow test. Decision curve analysis (DCA) curves were employed to assess the clinical benefit of the models. To estimate generalization, logistic regression, and ML models were compared with SOFA and APACHE scores using discrimination, calibration, and DCA for both the training and validation sets. The research design flowchart is depicted in Figure 1.

Figure 1

Figure 1. Flowchart illustrating the research design.

Variables importance

During traditional logistic regression, the importance of variables is determined by assessing the absolute value of each regression coefficient from the covariate. A larger absolute value indicates a more significant and important predictor. Variable importance is a key characteristic of ML models. In ML models, if changing the value of a variable leads to false prediction results, it implies that the variable is sensitive to classification outcomes and holds greater importance. The calculation of variable importance in the ML model such as RF involves determining the importance of each single decision tree, and by considering the number of trees set in the RF, the average of these values yields the overall variable importance of the RF model (22, 23).

Ethics statement

Data extraction and collection for this study were approved by the Ethics Committee of Huadu District People’s Hospital of Guangzhou (Registration Number: 2023088). Due to the retrospective nature of the study, the Ethics Committee of Huadu District People’s Hospital of Guangzhou waived the need of obtaining informed consent. And we had confirmed that the method of this research was performed in accordance with the regulation of Ethics Committee of Huadu District People’s Hospital of Guangzhou.

Statistical analysis

R version 4.1.3 was used for data analysis and the creation of statistical figures. Missing values in this cross-sectional study were addressed through multiple imputations using the “mice” package. The study population of adult sepsis patients was divided into training and validation sets using the “caret” package. Descriptive statistics, including mean ± standard deviation for continuous data with normal distribution and median (upper and lower quartiles) for non-normally distributed data, were employed to characterize average values. For univariate analysis, the Chi-square test was used to analyze differences in categorical data, while t-tests and Mann–Whitney U tests were employed for normally and non-normally distributed continuous data, respectively.

For logistic regression model construction, the “glm” function was used to conduct univariate analysis and binary logistic regression. The final model was determined by stepwise regression with the least AIC value using the “backward” method. For machine learning models, the Least Absolute Shrinkage and Selection Operator (LASSO) regression was utilized for variable shrinkage and selection, and the “glmnet” package was employed with parameter tuning using lambda.1SE under ten-fold cross-validation to remove irrelevant variables. The framework of “tidymodels” facilitated model selection, construction, workflow settings, validation, and comparison of predictive capabilities. Random forest was selected as the machine learning model based on the highest AUC and accuracy values, and the “randomForest” package was used to fit the model. To optimize the out-of-bag (OOB) error and improve predictive efficacy, the “tuneRF” function was employed.

Discrimination of the models was investigated using ROC and AUC with the “pROC” package. Calibration was assessed using the “calibration” function from the “rms” package, and the Hosmer-Lemeshow test was performed. Net benefits, reflecting model clinical practicality, were calculated and compared using DCA with the “ggDCA” package.

Results

Baseline analysis and data splitting

The study included a total of 606 patients diagnosed with sepsis, categorized into a survival group (n = 421) and a non-survival group (n = 185) based on hospital stay duration. To facilitate model construction and validation, a random allocation resulted in a training set (n = 435) and a validation set (n = 171) at a 7:3 ratio. Details of the baseline analysis and data splitting are presented in Table 1.

Table 1

Table 1. Baseline analysis and data splitting.

Logistic regression model construction

We initially conducted univariate analysis for risk factor selection in the training set for the logistic regression model. The results of the univariate analysis revealed that with significant differences (p < 0.05) between the survival and non-survival groups for including variables, gender, CHD, systolic pressure, WBC, NEUT, LYMP, lactic acid, NLR, RDW, IL6, PT, INR, FBI, D-dimer, AST, Tbil, and lung infection were brought into multiple variable regression (Table 2). Based on these 17 variables, we utilized logistic step regression (backward step method) to optimize the model according to the Akaike Information Criterion (AIC). The results showed that with the least AIC value (98.65), the logistic model (OR: 1.012, 95% CI: 2.218–3.216) included systolic pressure, lactic acid, NLR, RDW, IL6, PT, and Tbil as the final determinants (Supplementary Table S1).

Table 2

Table 2. Univariate analysis of risk factors between the survival group and non-survival group.

Machine learning model and variables selection

As depicted in Supplementary Figure S1, a comprehensive comparison of various machine learning models revealed that RF stood out with high accuracy and an AUC value of 0.99. Therefore, RF was chosen as the preferred methodology for model construction. Regarding variable selection, LASSO regression, employing ten-fold cross-validation with lambda.1SE criteria, identified systolic pressure, lactic acid, NEUT, RDW, IL6, INR, and Tbil as the chosen variables for RF model construction (Figure 2). During the RF modeling process, we initially set 500 decision trees for preliminary model calculation in the training set. To determine the optimal parameter for mortality prediction in sepsis patients, we utilized the OOB error as a measure of the model’s performance index. The results demonstrated that when the iteration reached 141 decision trees, the error rates of both OOB and model classification showed a noticeable decrease, reaching a stable state. This observation illustrated that the RF model achieved the most stable and optimal situation (Figure 3).

Figure 2

Figure 2. Variable shrinkage and selection by LASSO regression. (A) Shrinkage pathway of LASSO regression. (B) Based on ten-fold cross-validation, seven variables, including systolic pressure, lactic acid, NEUT, RDW, IL6, INR, and Tbil, were chosen using the lambda.1SE criteria.

Figure 3

Figure 3. Error rate chart of RF model. As the iteration reached 141 decision trees, the error rates of both out-of-bag (OOB) and model classification showed a noticeable decrease, eventually reaching a steady state.

Model validation and multi-models comparison

To assess the predictive efficacy of traditional logistic and RF models, we conducted assessments of discrimination, calibration, and clinical net benefits. Additionally, we compared the performance of logistic regression and RF models with SOFA (4) and APACHE (6) to explore clinical practicality. Discrimination results indicated that the among the predictive models of RF, logistic, SOFA, and APACHE, the AUCs and their corresponding 95% confidence intervals (CIs) were significantly larger (P_{Delong’s test} < 0.05) in both training and validation sets compared to other three models (Figures 4A,B). For model calibration, we observed that calibration curves of RF were notably closer to the ideal reference line compared to other models in both training and validation sets, which indicated that comparing to other models, the RF model associated with better fitting goodness and predictive ability (Figures 5A,B). Results of clinical practicality, as indicated by the Area Under Decision Curve (AUDC), showed that in the training set (Figure 6A) and validation set (Figure 6B), comparing with other three models, the AUDCs of RF model were with the highest values. These findings illustrated that the RF model yielded optimal clinical net benefit for predicting mortality in adult sepsis patients.

Figure 4

Figure 4. Comparison of discriminative ability among RF, logistic regression, SOFA, and APACHE scoring system. (A) Training set; (B) validation set. The blue solid ROC curves with the largest AUC values both in training set and validation set represented that RF associated with the best discrimination among the four models. AUC, area under curve; SOFA, sequential organ failure assessment scoring; APACHE, acute physiology and chronic health evaluation scoring.

Figure 5

Figure 5. Comparison of calibration curves among RF, logistic regression, SOFA, and APACHE scoring system. (A) Training set; (B) validation set. The blue solid calibration curves which were notably closer to the ideal reference line both in training set and validation set represented that RF associated with the best goodness-of-fit and accuracy of prediction among the four models. SOFA, sequential organ failure assessment scoring; APACHE, acute physiology and chronic health evaluation scoring. The left x-axis represents the observed probability; the right x-axis represents the sample size, y-axis represents the predicted probability.

Figure 6

Figure 6. Comparison of decision curve analysis among RF, logistic regression, SOFA, and APACHE scoring system. (A) Training set; (B) validation set. With the highest value of AUDC and net benefit both in training set and validation set, RF was considered as the optimum model which associated with the best clinical practicality. SOFA, sequential organ failure assessment scoring; APACHE, acute physiology and chronic evaluation scoring. AUDC, area under DCA curve.

Variables importance of logistic and RF models

The variable importance calculations from both the logistic regression and RF models are presented in Supplementary Figure S2. In predicting mortality in the adult sepsis cohort, the logistic regression model identified systolic pressure, lactic acid, IL6, and NLR as the most important variables, followed by Tbil, PT, and RDW. Consistently, the RF model also highlighted systolic pressure, lactic acid, IL6, and NLR as the most crucial variables for predicting mortality. However, the variables with relatively less importance in the RF model were RDW, NEUT, and Tbil, in contrast to the logistic regression model.

Discussion

In this study, we investigated the risk factors predicting the mortality of adult patients with sepsis, employing both the traditional logistic regression approach and the RF approach. Overall, both models yielded similar results, with only slight differences in the included variables, with the inclusion of PT as a risk factor in the logistic regression model, while NEUT was included in the RF model. To assess the predictive capabilities of these models for adult sepsis prognosis, we conducted comprehensive validations, considering discrimination, calibration, and clinical benefits. Among the three above criterion of model assessment, calibration is a critical aspect of evaluating the performance of clinical prediction models. It refers to the degree to which the predicted probabilities of an event match the actual observed outcomes. A well-calibrated model is one where the predicted probabilities are reliable indicators of the likelihood of the event occurring in practice. This is particularly important in clinical settings, where accurate predictions can guide treatment decisions and patient management. Additionally, we compared the models with the widely used SOFA and APACHE scoring systems based on these criteria. The results of model validation and comparison demonstrated that the RF model exhibited significant superiority over the logistic regression model, as well as over the SOFA and APACHE scoring systems, in predicting mortality in adult sepsis patients.

Application of biomarkers in adult sepsis prediction

Sepsis represents an aberrant inflammatory response triggered by pathogenic microorganism infection. There is an increasing consensus suggesting that the immune system’s activation in the early stages and its subsequent inhibition in the later stages can both contribute to alterations in circulating levels of inflammatory mediators (24–26). While the exact mechanisms of sepsis remain incompletely understood, studies have highlighted the crucial role of biomarkers in sepsis diagnosis and prognosis prediction, significantly impacting the risk of mortality (27–29). Our study exhibited that besides systolic pressure, biomarkers such as lactic acid, RDW, NLR, IL6, NEUT, and Tbil were incorporated into the traditional logistic and RF models we constructed. A closer examination through variable importance analysis revealed that lactic acid, NLR, and IL6 played pivotal roles in determining the significance of variables in both models.

Lactic acid, a metabolic byproduct of anaerobic glucose fermentation, poses a threat to the human body when present at elevated levels. High concentrations of lactic acid not only inhibit the activity of various essential enzymes but also mitigate the sensitivity of endothelial cells to vasoactive drugs (30). Furthermore, in the context of microbial infection or sepsis, lactic acid assumes a critical role in suppressing immune cells, potentially leading to immune suppression and severe consequences for the individual (31). Elevated levels of lactic acid in patients with sepsis are associated with poor outcomes, as they reflect inadequate perfusion and oxygen delivery to tissues. Studies have shown that high lactate levels correlate with increased mortality rates in septic patients, making it a valuable prognostic marker. Over the years, numerous studies have underscored the association between elevated lactic acid levels and increased mortality rates in sepsis (4, 32, 33).

The NLR serves as a biomarker calculated by the ratio of neutrophil to lymphocyte counts, encompassing both the innate immune response, primarily mediated by neutrophils, and adaptive immunity, supported by lymphocytes (34). Neutrophils act as the frontline defenders against pathogen invasion through processes like chemotaxis and phagocytosis. Upon activation by pathogens, various cytokines, granular proteins, and reactive oxygen species (ROS) are produced and released by neutrophils (35). While this activation is crucial for pathogen resistance, excessive activation leading to increased production of ROS and cytokines may damage vascular endothelial cells through different mechanisms, resulting in tissue hypoperfusion and life-threatening organ failure (36). Consequently, an elevated neutrophil count, or a decreased lymphocyte count, contributes to an increased NLR, serving as a predictor of disease severity and poor prognosis in various conditions such as severe trauma (37), stroke (38), malignant tumor (39, 40) and sepsis (41, 42). Previous studies on NLR in predicting sepsis prognosis have demonstrated its independent association with high in-hospital mortality rates, showcasing significant advantages over conventional scores like SOFA or APACHE (43, 44). In summary, NLR stands out as a valuable biomarker for predicting mortality in sepsis patients.

Pro-inflammatory cytokines play a critical role in sepsis pathogenesis. IL-6, a member of the 4-helical cytokine family, activates signaling pathways by binding to an 80-kDa cytokine receptor (IL-6R). IL-6 plays a pivotal role in the immune response to infection, and it is released by various cells, including macrophages and T cells, in response to inflammatory stimuli. During sepsis, IL-6 is produced in response to pathogenic stimuli, and IL-6R is generated by neutrophils. Consequently, the IL-6/IL-6R complex triggers the phosphorylation and redistribution of VE-cadherin, leading to vascular endothelial damage and leakage (45). Excessive vascular endothelial damage and leakage in sepsis patients can result in blood pressure decline, hemodynamic collapse, irreversible septic shock, and even death. Clinical predictive models have consistently shown that IL-6 holds favorable predictive value for sepsis severity and prognosis. Elevated levels of IL-6 suggest severe illness and poor prognosis (46, 47). Moreover, studies have indicated that immunotherapeutic blockade of IL6 could reduce the mortality rate in sepsis (48).

Application of advanced statistical methods to complement common approaches

The RF algorithm possesses numerous statistical and computational advantages. This algorithm employs integrated learning, wherein its fundamental component is typically a decision tree, placing it within the broader category of integrated learning methods (49, 50). The terminology “random” and “forest” in RF signifies the amalgamation of classifiers, where each tree functions as an individual classifier. Notably, RF operates with hundreds of trees in parallel, collectively forming a forest. RF consolidates the results of all classification votes, designating the category with the highest votes as the final output, aligning with the Bagging concept and reflecting the core idea of RF (51). In contrast to the traditional logistic regression algorithm, RF demonstrates several distinct advantages: (1) RF employs an integrated algorithm with exceptionally high accuracy; (2) The randomness in model construction reduces susceptibility to overfitting; (3) It can handle discrete, continuous, or high-dimensional data without requiring data normalization; (4) The OOB feature allows obtaining unbiased estimates of true errors during model generation without losing training data. In the present study, the RF model demonstrated its superiority in predicting the prognosis of adult sepsis, exhibiting better discrimination, calibration, and clinical decision-making compared to traditional statistical methods (52, 53). Although RF model improves prediction accuracy by integrating multiple decision trees, but this also makes their decision-making process relatively complex and difficult to explain. Each decision tree is trained based on a randomly selected subset of features, which increases the model’s diversity but also makes it challenging to interpret. So as to address these limitations, we can solve these problems by conducting feature importance analysis, visualizing individual decision trees, employing local explainability methods, and integrating doctors’ experiences and expertise, it is possible to address the limitations of interpretability to a certain extent.

However, the present study has limitations that should be acknowledged. Firstly, being retrospective and cross-sectional, it relies on some laboratory results reflecting the patient’s condition at specific time points, which may not be generalizable to the entire population. We expect to validate the current research and strengthen the impact of this study through prospective research. Secondly, we explicitly state that our initial predictor selection was based on univariate analysis, which may not capture the full complexity of the relationships between the predictors and the outcome variable. Thirdly, despite RF’s significant advantages in predicting sepsis mortality compared to traditional regression, its interpretability limitation remains noteworthy. Finally, due to this is a single center study and without testing on an independent dataset, the model’s accuracy could be artificially inflated, reducing its generalizability. Therefore, integrating RF with traditional regression approaches could enhance the predictive capabilities of healthcare research in the future.

Conclusion

In conclusion, logistic regression and RF models were developed to predict mortality in adult sepsis patients, with both models identifying consistent risk factors. The RF model outperformed traditional regression and the SOFA and APACHE scoring systems, highlighting its superiority in mortality prediction.

Data availability statement

The original contributions presented in the study are included in the article/Supplementary material, further inquiries can be directed to the corresponding authors.

Ethics statement

The studies involving humans were approved by Ethics Committee of Huadu District People’s Hospital of Guangzhou. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.

Author contributions

HW: Conceptualization, Formal analysis, Writing – original draft. BL: Data curation, Writing – original draft. TJ: Data curation, Writing – original draft. KM: Formal analysis, Software, Writing – original draft. YL: Formal analysis, Writing – review & editing. SZ: Supervision, Writing – review & editing.

Funding

The author(s) declare that financial support was received for the research, authorship, and/or publication of this article. This study was supported by the Construction of Major Subject of Huadu District People’s Hospital of Guangzhou (Grant no. YNZDXK202201, 2022–2025) and Huadu District Basic and Applied Basic Research Joint Funded Project (Grant no. 23HDQYLH06).

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fmed.2024.1496869/full#supplementary-material

References

1. Singer, M, Deutschman, CS, Seymour, CW, Shankar-Hari, M, Annane, D, Bauer, M, et al. The third international consensus definitions for Sepsis and septic shock (Sepsis-3). JAMA. (2016) 315:801–10. doi: 10.1001/jama.2016.0287

PubMed Abstract | Crossref Full Text | Google Scholar

2. Esposito, S, De Simone, G, Boccia, G, De Caro, F, and Pagliano, P. Sepsis and septic shock: new definitions, new diagnostic and therapeutic approaches. J Glob Antimicrob Resist. (2017) 10:204–12. doi: 10.1016/j.jgar.2017.06.013

PubMed Abstract | Crossref Full Text | Google Scholar

3. Chiu, C, and Legrand, M. Epidemiology of sepsis and septic shock. Curr Opin Anaesthesiol. (2021) 34:71–6. doi: 10.1097/ACO.0000000000000958

PubMed Abstract | Crossref Full Text | Google Scholar

4. Liu, Z, Meng, Z, Li, Y, Zhao, J, Wu, S, Gou, S, et al. Prognostic accuracy of the serum lactate level, the SOFA score and the qSOFA score for mortality among adults with Sepsis. Scand J Trauma Resusc Emerg Med. (2019) 27:51. doi: 10.1186/s13049-019-0609-3

PubMed Abstract | Crossref Full Text | Google Scholar

5. Raith, EP, Udy, AA, Bailey, M, McGloughlin, S, MacIsaac, C, Bellomo, R, et al. Prognostic accuracy of the SOFA score, SIRS criteria, and qSOFA score for in-hospital mortality among adults with suspected infection admitted to the intensive care unit. JAMA. (2017) 317:290–300. doi: 10.1001/jama.2016.20328

Crossref Full Text | Google Scholar

6. Liu, X, Shen, Y, Li, Z, Fei, A, Wang, H, Ge, Q, et al. Prognostic significance of APACHE II score and plasma suPAR in Chinese patients with sepsis: a prospective observational study. BMC Anesthesiol. (2016) 16:46. doi: 10.1186/s12871-016-0212-3

PubMed Abstract | Crossref Full Text | Google Scholar

7. Barichello, T, Generoso, JS, Singer, M, and Dal-Pizzol, F. Biomarkers for sepsis: more than just fever and leukocytosis-a narrative review. Crit Care. (2022) 26:14. doi: 10.1186/s13054-021-03862-5

PubMed Abstract | Crossref Full Text | Google Scholar

8. Pierrakos, C, Velissaris, D, Bisdorff, M, Marshall, JC, and Vincent, J-L. Biomarkers of sepsis: time for a reappraisal. Crit Care. (2020) 24:287. doi: 10.1186/s13054-020-02993-5

PubMed Abstract | Crossref Full Text | Google Scholar

9. Lee, JH, Kim, S-H, Jang, JH, Park, JH, Jo, KM, No, T-H, et al. Clinical usefulness of biomarkers for diagnosis and prediction of prognosis in sepsis and septic shock. Medicine. (2022) 101:e31895. doi: 10.1097/MD.0000000000031895

PubMed Abstract | Crossref Full Text | Google Scholar

10. Xu, C, Zheng, L, Jiang, Y, and Jin, L. A prediction model for predicting the risk of acute respiratory distress syndrome in sepsis patients: a retrospective cohort study. BMC Pulm Med. (2023) 23:78. doi: 10.1186/s12890-023-02365-z

PubMed Abstract | Crossref Full Text | Google Scholar

11. Bhavani, SV, Semler, M, Qian, ET, Verhoef, PA, Robichaux, C, Churpek, MM, et al. Development and validation of novel sepsis subphenotypes using trajectories of vital signs. Intensive Care Med. (2022) 48:1582–92. doi: 10.1007/s00134-022-06890-z

PubMed Abstract | Crossref Full Text | Google Scholar

12. Burzykowski, T, Geubbelmans, M, Rousseau, A-J, and Valkenborg, D. Generalized linear models. Am J Orthod Dentofacial Orthop. (2023) 164:604–6. doi: 10.1016/j.ajodo.2023.07.005

PubMed Abstract | Crossref Full Text | Google Scholar

13. Deo, RC. Machine learning in medicine. Circulation. (2015) 132:1920–30. doi: 10.1161/CIRCULATIONAHA.115.001593

PubMed Abstract | Crossref Full Text | Google Scholar

14. García-Gallo, JE, Fonseca-Ruiz, NJ, Celi, LA, and Duitama-Muñoz, JF. A machine learning-based model for 1-year mortality prediction in patients admitted to an intensive care unit with a diagnosis of sepsis. Med Intensiva. (2020) 44:160–70. doi: 10.1016/j.medin.2018.07.016

PubMed Abstract | Crossref Full Text | Google Scholar

15. Agnello, L, Vidali, M, Padoan, A, Lucis, R, Mancini, A, Guerranti, R, et al. Machine learning algorithms in sepsis. Clin Chim Acta. (2024) 553:117738. doi: 10.1016/j.cca.2023.117738

PubMed Abstract | Crossref Full Text | Google Scholar

16. Moor, M, Rieck, B, Horn, M, Jutzeler, CR, and Borgwardt, K. Early prediction of Sepsis in the ICU using machine learning: a systematic review. Front Med. (2021) 8:607952. doi: 10.3389/fmed.2021.607952

PubMed Abstract | Crossref Full Text | Google Scholar

17. Islam, MM, Nasrin, T, Walther, BA, Wu, C-C, Yang, H-C, and Li, Y-C. Prediction of sepsis patients using machine learning approach: a meta-analysis. Comput Methods Prog Biomed. (2019) 170:1–9. doi: 10.1016/j.cmpb.2018.12.027

PubMed Abstract | Crossref Full Text | Google Scholar

18. Collins, GS, Dhiman, P, Andaur Navarro, CL, Ma, J, Hooft, L, Reitsma, JB, et al. Protocol for development of a reporting guideline (TRIPOD-AI) and risk of bias tool (PROBAST-AI) for diagnostic and prognostic prediction model studies based on artificial intelligence. BMJ Open. (2021) 11:e048008. doi: 10.1136/bmjopen-2020-048008

PubMed Abstract | Crossref Full Text | Google Scholar

19. Kaiser, I, Mathes, S, Pfahlberg, AB, Uter, W, Berking, C, Heppt, MV, et al. Using the prediction model risk of Bias assessment tool (PROBAST) to evaluate melanoma prediction studies. Cancers. (2022) 14:33. doi: 10.3390/cancers14123033

PubMed Abstract | Crossref Full Text | Google Scholar

20. Fleischmann-Struzek, C, Mellhammar, L, Rose, N, Cassini, A, Rudd, KE, Schlattmann, P, et al. Incidence and mortality of hospital- and ICU-treated sepsis: results from an updated and expanded systematic review and meta-analysis. Intensive Care Med. (2020) 46:1552–62. doi: 10.1007/s00134-020-06151-x

PubMed Abstract | Crossref Full Text | Google Scholar

21. Font, MD, Thyagarajan, B, and Khanna, AK. Sepsis and septic shock – basics of diagnosis, pathophysiology and clinical decision making. Med Clin North Am. (2020) 104:573–85. doi: 10.1016/j.mcna.2020.02.011

PubMed Abstract | Crossref Full Text | Google Scholar

22. Liu, Y, Zhou, S, Wei, H, and An, S. A comparative study of forest methods for time-to-event data: variable selection and predictive performance. BMC Med Res Methodol. (2021) 21:193. doi: 10.1186/s12874-021-01386-8

PubMed Abstract | Crossref Full Text | Google Scholar

23. Giannini, HM, Ginestra, JC, Chivers, C, Draugelis, M, Hanish, A, Schweickert, WD, et al. A machine learning algorithm to predict severe Sepsis and septic shock: development, implementation, and impact on clinical practice. Crit Care Med. (2019) 47:1485–92. doi: 10.1097/CCM.0000000000003891

PubMed Abstract | Crossref Full Text | Google Scholar

24. Cantey, JB, and Lee, JH. Biomarkers for the diagnosis of neonatal Sepsis. Clin Perinatol. (2021) 48:215–27. doi: 10.1016/j.clp.2021.03.012

PubMed Abstract | Crossref Full Text | Google Scholar

25. Grondman, I, Pirvu, A, Riza, A, Ioana, M, and Netea, MG. Biomarkers of inflammation and the etiology of sepsis. Biochem Soc Trans. (2020) 48:1–14. doi: 10.1042/BST20190029

PubMed Abstract | Crossref Full Text | Google Scholar

26. Opal, SM, and Wittebole, X. Biomarkers of infection and Sepsis. Crit Care Clin. (2020) 36:11–22. doi: 10.1016/j.ccc.2019.08.002

PubMed Abstract | Crossref Full Text | Google Scholar

27. Zhou, G, Liu, J, Zhang, H, Wang, X, and Liu, D. Elevated endothelial dysfunction-related biomarker levels indicate the severity and predict sepsis incidence. Sci Rep. (2022) 12:21935. doi: 10.1038/s41598-022-26623-y

PubMed Abstract | Crossref Full Text | Google Scholar

28. Makkar, N, Soneja, M, Arora, U, Sood, R, Biswas, S, Jadon, RS, et al. Prognostic utility of biomarker levels and clinical severity scoring in sepsis: a comparative study. J Investig Med. (2022) 70:1399–405. doi: 10.1136/jim-2021-002276

PubMed Abstract | Crossref Full Text | Google Scholar

29. Li, X, Li, T, Dong, G, Wei, Y, Xu, Z, and Yang, J. Clinical value of serum Interleukin-18 in neonatal Sepsis diagnosis and mortality prediction. J Inflamm Res. (2022) 15:6923–30. doi: 10.2147/JIR.S393506

PubMed Abstract | Crossref Full Text | Google Scholar

30. Rezar, R, Mamandipoor, B, Seelmaier, C, Jung, C, Lichtenauer, M, Hoppe, UC, et al. Hyperlactatemia and altered lactate kinetics are associated with excess mortality in sepsis: a multicenter retrospective observational study. Wien Klin Wochenschr. (2023) 135:80–8. doi: 10.1007/s00508-022-02130-y

PubMed Abstract | Crossref Full Text | Google Scholar

31. Luo, Y, Li, L, Chen, X, Gou, H, Yan, K, and Xu, Y. Effects of lactate in immunosuppression and inflammation: Progress and prospects. Int Rev Immunol. (2022) 41:19–29. doi: 10.1080/08830185.2021.1974856

PubMed Abstract | Crossref Full Text | Google Scholar

32. Jagan, N, Morrow, LE, Walters, RW, Plambeck, RW, Patel, TM, Moore, DR, et al. Sympathetic stimulation increases serum lactate concentrations in patients admitted with sepsis: implications for resuscitation strategies. Ann Intensive Care. (2021) 11:24. doi: 10.1186/s13613-021-00805-9

PubMed Abstract | Crossref Full Text | Google Scholar

33. Tongyoo, S, Sutthipool, K, Viarasilpa, T, and Permpikul, C. Serum lactate levels in cirrhosis and non-cirrhosis patients with septic shock. Acute Crit Care. (2022) 37:108–17. doi: 10.4266/acc.2021.00332

PubMed Abstract | Crossref Full Text | Google Scholar

34. Buonacera, A, Stancanelli, B, Colaci, M, and Malatino, L. Neutrophil to lymphocyte ratio: An emerging marker of the relationships between the immune system and diseases. Int J Mol Sci. (2022) 23:636. doi: 10.3390/ijms23073636

PubMed Abstract | Crossref Full Text | Google Scholar

35. Mortaz, E, Alipoor, SD, Adcock, IM, Mumby, S, and Koenderman, L. Update on neutrophil function in severe inflammation. Front Immunol. (2018) 9:2171. doi: 10.3389/fimmu.2018.02171

PubMed Abstract | Crossref Full Text | Google Scholar

36. Zhang, H, Wang, Y, Qu, M, Li, W, Wu, D, Cata, JP, et al. Neutrophil, neutrophil extracellular traps and endothelial cell dysfunction in sepsis. Clin Transl Med. (2023) 13:e1170. doi: 10.1002/ctm2.1170

PubMed Abstract | Crossref Full Text | Google Scholar

37. Sabouri, E, Majdi, A, Jangjui, P, Rahigh Aghsan, S, and Naseri Alavi, SA. Neutrophil-to-lymphocyte ratio and traumatic brain injury: a review study. World Neurosurg. (2020) 140:142–7. doi: 10.1016/j.wneu.2020.04.185

PubMed Abstract | Crossref Full Text | Google Scholar

38. Lattanzi, S, Brigo, F, Trinka, E, Cagnetti, C, Di Napoli, M, and Silvestrini, M. Neutrophil-to-lymphocyte ratio in acute cerebral hemorrhage: a system review. Transl Stroke Res. (2019) 10:137–45. doi: 10.1007/s12975-018-0649-4

PubMed Abstract | Crossref Full Text | Google Scholar

39. Lin, N, Li, J, Yao, X, Zhang, X, Liu, G, Zhang, Z, et al. Prognostic value of neutrophil-to-lymphocyte ratio in colorectal cancer liver metastasis: a meta-analysis of results from multivariate analysis. Int J Surg. (2022) 107:106959. doi: 10.1016/j.ijsu.2022.106959

PubMed Abstract | Crossref Full Text | Google Scholar

40. Corbeau, I, Jacot, W, and Guiu, S. Neutrophil to lymphocyte ratio as prognostic and predictive factor in breast Cancer patients: a systematic review. Cancers. (2020) 12:958. doi: 10.3390/cancers12040958

PubMed Abstract | Crossref Full Text | Google Scholar

41. Bu, X, Zhang, L, Chen, P, and Wu, X. Relation of neutrophil-to-lymphocyte ratio to acute kidney injury in patients with sepsis and septic shock: a retrospective study. Int Immunopharmacol. (2019) 70:372–7. doi: 10.1016/j.intimp.2019.02.043

PubMed Abstract | Crossref Full Text | Google Scholar

42. Liu, S, Li, Y, She, F, Zhao, X, and Yao, Y. Predictive value of immune cell counts and neutrophil-to-lymphocyte ratio for 28-day mortality in patients with sepsis caused by intra-abdominal infection. Burns Trauma. (2021) 9:tkaa040. doi: 10.1093/burnst/tkaa040

PubMed Abstract | Crossref Full Text | Google Scholar

43. Shi, Y, Yang, C, Chen, L, Cheng, M, and Xie, W. Predictive value of neutrophil-to-lymphocyte and platelet ratio in in-hospital mortality in septic patients. Heliyon. (2022) 8:e11498. doi: 10.1016/j.heliyon.2022.e11498

PubMed Abstract | Crossref Full Text | Google Scholar

44. Drăgoescu, AN, Pădureanu, V, Stănculescu, AD, Chiuțu, LC, Tomescu, P, Geormăneanu, C, et al. Neutrophil to lymphocyte ratio (NLR)-a useful tool for the prognosis of Sepsis in the ICU. Biomedicines. (2021) 10:75. doi: 10.3390/biomedicines10010075

PubMed Abstract | Crossref Full Text | Google Scholar

45. Rose-John, S, Jenkins, BJ, Garbers, C, Moll, JM, and Scheller, J. Targeting IL-6 trans-signalling: past, present and future prospects. Nat Rev Immunol. (2023) 23:666–81. doi: 10.1038/s41577-023-00856-y

PubMed Abstract | Crossref Full Text | Google Scholar

46. Ríos-Toro, J-J, Márquez-Coello, M, García-Álvarez, J-M, Martín-Aspas, A, Rivera-Fernández, R, Sáez de Benito, A, et al. Soluble membrane receptors, interleukin 6, procalcitonin and C reactive protein as prognostic markers in patients with severe sepsis and septic shock. PLoS One. (2017) 12:e0175254. doi: 10.1371/journal.pone.0175254

PubMed Abstract | Crossref Full Text | Google Scholar

47. Song, W, Tian, F, Wang, Y, Sun, Q, Guo, F, Zhao, G, et al. Predictive value of C-reactive protein, procalcitonin, and interleukin-6 on 30-day mortality in patients with bloodstream infections. Med Clin. (2023) 160:540–6. doi: 10.1016/j.medcli.2023.01.022

PubMed Abstract | Crossref Full Text | Google Scholar

48. Barrett, D. IL-6 blockade in cytokine storm syndromes. Adv Exp Med Biol. (2024) 1448:565–72. doi: 10.1007/978-3-031-59815-9_37

PubMed Abstract | Crossref Full Text | Google Scholar

49. Hu, J, and Szymczak, S. A review on longitudinal data analysis with random forest. Brief Bioinform. (2023) 24:bbad002. doi: 10.1093/bib/bbad002

PubMed Abstract | Crossref Full Text | Google Scholar

50. Nwanosike, EM, Conway, BR, Merchant, HA, and Hasan, SS. Potential applications and performance of machine learning techniques and algorithms in clinical practice: a systematic review. Int J Med Inform. (2022) 159:104679. doi: 10.1016/j.ijmedinf.2021.104679

PubMed Abstract | Crossref Full Text | Google Scholar

51. Rigatti, SJ. Random forest. J Insur Med. (2017) 47:31–9. doi: 10.17849/insm-47-01-31-39.1

PubMed Abstract | Crossref Full Text | Google Scholar

52. Xie, D, Ying, M, Lian, J, Li, X, Liu, F, Yu, X, et al. Serological indices and ultrasound variables in predicting the staging of hepatitis B liver fibrosis: a comparative study based on random forest algorithm and traditional methods. J Cancer Res Ther. (2022) 18:2049–57. doi: 10.4103/jcrt.jcrt_1394_22

PubMed Abstract | Crossref Full Text | Google Scholar

53. Fan, S, Lin, J, Wu, S, Mu, X, and Guo, J. Random forest model can predict the prognosis of hospital-acquired Klebsiella pneumoniae infection as well as traditional logistic regression model. PLoS One. (2022) 17:e0278123. doi: 10.1371/journal.pone.0278123

PubMed Abstract | Crossref Full Text | Google Scholar