Nomogram and Machine Learning Models Predict 1-Year Mortality Risk in Patients With Sepsis-Induced Cardiorenal Syndrome

Objective: Early prediction of long-term outcomes in patients with sepsis-induced cardiorenal syndrome (CRS) remains a great challenge in clinical practice. Herein, we aimed to construct a nomogram and machine learning model for predicting the 1-year mortality risk in patients with sepsis-induced CRS.

Methods: This retrospective study enrolled 340 patients diagnosed with sepsis-induced CRS in Shanghai Tongji Hospital between January 2015 and May 2019, as a discovery cohort. Two predictive models, the nomogram and machine learning model, were used to predict 1-year mortality. The prognostic variables used to develop the nomogram were identified based on a forward stepwise binary logistic regression, and the predictive ability of the nomogram was evaluated by the areas under the receiver operating characteristic curve (AUC) and the calibration curve. Meanwhile, machine learning (ML) techniques, such as support vector machine, random forest (RF), and gradient boosted decision tree, were assessed mainly by accuracy and AUC. Feature ranking analysis was performed using the ML algorithm. Both nomogram and ML models were externally validated by an independent cohort of 103 patients diagnosed with sepsis-induced CRS between June 2019 and December 2020.

Results: Age, sequential sepsis-related organ failure score (SOFA), serum myoglobin (MYO), vasopressor use, and mechanical ventilation were identified as independent risk factors for 1-year mortality in the nomogram predictive model. In the discovery cohort, the nomogram yielded higher AUC for predicting mortality than did the SOFA score (0.855 [95% CI: 0.815–0.895] vs. 0.756 [95% CI: 0.705–0.808]). For ML, the model developed by RF showed the highest accuracy (0.765) and AUC (0.854). In feature ranking analysis, factors such as age, MYO, SOFA score, vasopressor use, and baseline serum creatinine were identified as important features affecting 1-year prognosis. Moreover, the nomogram and RF model both performed well in external validation, with an AUC of 0.877 and 0.863, respectively.

Conclusion: Our nomogram and ML models showed that age, SOFA score, serum MYO levels, and the use of vasopressors during hospitalization were the main factors influencing the risk of long-term mortality. Our models may serve as useful tools for assessing long-term prognosis in patients with sepsis-induced CRS.

Introduction

Sepsis is a systemic inflammatory syndrome caused by the body’s unbalanced response to infection. Despite recent advances in diagnosis, treatment, and post-acute care, sepsis remains a deadly disease with unacceptably high morbidity and mortality rates (1). Sepsis can induce multiple organ dysfunction, including that of the heart and kidney, which may manifest clinically as myocardial depression, elevated myocardial injury markers, and acute renal insufficiency (2, 3). Cardiorenal syndrome (CRS) is an acute or chronic dysfunction of the heart or kidney that results in the acute or chronic dysfunction of the other organ. In 2008, the Acute Diseases Quality Initiative proposed five types of CRS, with the combined occurrence of heart and kidney dysfunction caused by sepsis and other systemic diseases defined as CRS type 5 (4). The mortality rates of patients with sepsis and CRS range from 20 to 60%. In addition, sepsis patients with newly diagnosed CRS are at a higher risk of developing a worse in-hospital prognosis than those without acute cardiorenal attack (5, 6). Despite the increasing attention being paid to this critical disease (sepsis-induced CRS), information about its prognostic factors and mortality rate remains limited to date.

The prediction of long-term outcomes in patients with sepsis-induced CRS may help to optimize decision-making and post-acute care. Machine learning (ML) techniques are currently being used as powerful and reliable tools for outcome assessment. Compared with standard methods of statistical model establishment, ML methods are capable of processing a larger number of variables and tend to output more accurate and precise results (7). A nomogram is an ancient calculator similar to the slide rule, (8) that provides graphical depictions of the logistic or Cox regression model. It has been used epidemiologically in disease diagnosis (9, 10), prognosis evaluation (11–13), and recurrence prediction (14). Although sophisticated ML methodologies may provide more accurate and generalized prediction models (15), the convenient and transparent characteristics of the nomogram still make it popular with clinicians.

Both traditional and ML models may serve as useful tools for improving prognostic judgment and decision-making by clinicians and for helping patients and their families understand the severity and prognosis of sepsis-induced CRS. Thus, this study aimed to explore the independent risk factors of 1-year mortality in sepsis-induced CRS patients and develop predictive models using traditional statistical methods and ML techniques.

Materials and Methods

Study Design and Population

This was a single-center, retrospective cohort study in which consecutive patients who developed concomitant acute cardiac and kidney injury secondary to sepsis during hospitalization at Shanghai Tongji Hospital from January 1, 2015, to December 31, 2020, were enrolled. The inclusion criteria were as follows: age ≥ 18 years; sepsis diagnosis in accordance with the third international consensus definitions published in 2016 (1); acute kidney injury (AKI) defined according to the 2012 Kidney Disease Improving Global Outcomes guidelines (16); acute myocardial injury indicated by at least of the following laboratory indicators: type B natriuretic peptide (BNP) increased ≥ 100 pg/mL, N-terminal pro-B-type natriuretic peptide (NT-proBNP) increased ≥ 300 pg/mL, and cardiac troponin I (cTnI) increased ≥ 0.03 ng/ml, within 48 h. The exclusion criteria were as follows: hospital length of stay ≤ 48 h; acute heart and kidney damage caused by non-infectious factors (e.g., urinary tract obstruction, autoimmune disease-related AKI, coronary heart disease complicated by acute myocardial infarction, etc.); active malignant tumors; mental disorders; pregnancy; and incomplete clinical data.

In the discovery cohort, a total of 1,115 patients diagnosed with sepsis at Shanghai Tongji Hospital between January 2015 and May 2019 were screened, and 483 of them had concomitant acute cardiac and kidney injury secondary to sepsis during hospitalization. Of the 483 patients initially identified, 143 were excluded according to the above criteria. Finally, 340 patients with sepsis-induced CRS were enrolled and categorized according to their living status (alive or deceased) within 1 year after diagnosis as survivors (n = 169 patients) and non-survivors (n = 171 patients) (Figure 1). In the validation cohort, a total of 103 patients diagnosed with sepsis-induced CRS between June 2019 and December 2020 were enrolled by the same process (Supplementary Figure 1). To date, sepsis-induced CRS remains a clinical diagnosis characterized by simultaneous existence of acute heart and kidney injury in the setting of sepsis (17). Considering the lack of clear and consistent diagnostic criteria for sepsis-induced CRS, in our study, heart and kidney injury were evaluated using blood markers including BNP, NT-proBNP, cTnI, and serum creatinine (Scr) (18).

FIGURE 1

Figure 1. Flow chart illustrating the patient enrollment in discovery cohort.

Data Collection and Follow-Up

Data on patient characteristics, including age, sex, department, infection site, blood culture, vital signs, smoking history, pre-existing diseases, medication history, and in-hospital treatment, were retrieved from the medical records. Vital signs included blood pressure, heart rate, axillary temperature, and respiratory rate recorded in the nursing chart on the first day of diagnosis of sepsis-induced CRS. Medication history 3 months prior to diagnosis was reviewed and determined from both outpatient and inpatient records. In-hospital treatment comprised mechanical ventilation and the use of vasopressors such as dopamine, norepinephrine, or epinephrine.

Laboratory indices of cTnI, MYO, and Scr were recorded on the first and third days of diagnosis. Based on the linear analysis between the covariates and mortality risk, continuous variables such as cTnI and Scr were transformed into categorical variables. Additionally, to explore whether the rate of change in continuous variables would affect prognosis, we calculated rate of change using laboratory data from the first and third days of diagnosis. Rate of change in cTnI and baseline Scr were not considered in the logistic regression as some of the denominators of cTnI could be 0. For baseline Scr, we found a large multicollinearity between the creatinine level on the third day and the rate of change in baseline Scr.

The baseline Scr level was measured in accordance with the varying conditions of the patients in the following order: (1) the Scr value from the most recent examination before hospitalization (within 12 months); (2) the nadir creatinine value measured during the first 3 days of hospitalization; and (3) the baseline Scr value inversely deduced using the back-estimation formula according to the population’s average GFR level of 75 ml/(min⋅1.73 m²), (19). The SOFA scores were determined by the clinician from clinical records or from the medical records and laboratory values obtained on the first day. The patients were followed up for at least 1 year after discharge, and deaths were confirmed by either electronic medical records or telephone follow-ups.

Statistical Analysis

Nomogram Prediction Model

Patient characteristics are presented as percentages for categorical data and as median with interquartile range for continuous data. As for traditional (non-ML) statistical methods, restricted cubic spine was used to assess the linear relationship between the potential variables and the outcome (Figure 2). In addition, non-linear factors were converted into categorical variables. Potential prognostic factors were evaluated using univariate logistic analysis, and the predictors (p < 0.10) were included in multiple logistic regression. The predictors enrolled in the nomogram met the requirement of 10 events per variable to reduce bias and variability of the logistic model. The variance inflation factor was used to test for multicollinearity between the predictors. Forward stepwise binary logistic regression was performed to identify the independent risk factors. The nomogram prediction model was then established based on independent prognostic factors, and internal and external validation were performed using the 500 bootstrap resampling method. The model’s discrimination ability was evaluated using the receiver operating characteristic curve (AUC), while the calibration ability was assessed using the calibration curve. The model accuracy was determined according to the Brier score, with a closer score to 0 indicating better accuracy. Statistical analyses were performed using IBM SPSS Statistics 22.0 and R software (R 4.0.2). Statistical significance was set at p < 0.05. The development process of the nomogram is shown in Figure 3A.

FIGURE 2

Figure 2. The linear analysis of numeric variables. (A) Basic serum creatinine. (B) Age. (C) Serum MYO levels on day 1. (D) Serum MYO levels on day 3.

FIGURE 3

Figure 3. (A) The process of development of nomogram. (B) The process of machine learning methods.

Machine Learning Model

The establishment and internal evaluation process of the ML model is shown in Figure 3B. The 1-year survival status was set as the label, and the other variables were set as features. Under fivefold cross-validation, 70% of the data were randomly assigned to the training set, and the remaining 30% were entered into the test set. In the training set, the predictive models (also called “classifiers”) were trained using ML algorithms such as support vector machine, random forest (RF), gradient boosted decision tree, extreme gradient boosting, and light gradient boosted machine. The classifiers were then applied to the test set for internal validation. Finally, the classifiers were externally verified in the validation cohort. The generalization capability of different classifiers was assessed by accuracy, recall, precision, F1 score, and AUC. Moreover, feature ranking analysis was performed, and those with a significant impact on survival were included in the decision tree algorithm. The developed and visualized decision tree was subjected to a post-pruning process to prevent overfitting of the data set. In the pruned decision tree, the 1-year mortality risks were classified as low (< 25%), moderate (25–49%), high (50–74%), and very high (> 75%) risk. The above algorithms were all run using Python software.

Results

Patient Characteristics

Of the 340 patients enrolled in the discovery cohort, 171 (50.3%) died within 1 year after the diagnosis of sepsis-induced CRS. The non-survivors were significantly older than the survivors (Z = –2.358, P = 0.018). Regarding infection, the most common was respiratory (56.8%), followed by digestive (19.1%), urinary tract (16.5%), and skin and soft tissue (5.0%) infections. Sex, vital signs, pre-existing diseases, and blood culture did not significantly affect prognosis. The baseline patient characteristics are shown in Table 1. In the validation cohort, 103 patients were enrolled, and 56 (54.4%) of them died within 1 year. The baseline Scr in the validation cohort was lower than that in the discovery cohort. Most of the baseline characteristics showed no statistical differences and were comparable in both cohorts (Supplementary Table 1).

TABLE 1

Table 1. Patient clinical characteristics.

Nomogram: Development and Assessment

The results of univariate and multivariate analyses for risk factors of 1 year mortality are shown in Table 2. Age, SOFA, and serum MYO on day 3 of diagnosis, use of vasopressors, and mechanical ventilation were identified as independent risk factors for 1-year mortality. The final model was constructed based on the prognostic predictors and further visualized as a nomogram using the “rms” package from the R software (Figure 4). The application methods for the nomogram are described as follows. First, we drew an ascending line from the variable axis to the “Points” axis to obtain the points for each risk factor. Then, the scores of all the variables were combined to obtain the total number of points. Finally, we drew a downward perpendicular line from the “Total Points” axis to the “Risk” axis. The corresponding number was then presented as the estimated risk of 1-year mortality. The β-coefficients for the final logistic regression model are presented in Table 3.

TABLE 2

Table 2. Univariate and multivariate analyses for prognostic factors.

FIGURE 4

Figure 4. The Nomogram for prediction of 1-year mortality risk in patients with newly diagnosed sepsis-induced cardiorenal syndrome.

TABLE 3

Table 3. Regression coefficient estimates of the 1-year predictive model.

In the discovery cohort, the prognostic nomogram achieved excellent discrimination, with an AUC of 0.855 [95% confidence interval (CI): 0.815–0.895] (Figure 5A), and relatively low optimism, with a bootstrap validated AUC of 0.843. When SOFA score was solely used to predict 1-year outcome, it only achieved an AUC of 0.756 (95% CI: 0.705–0.808). DeLong test to compare the AUC between the SOFA score and nomogram showed that the nomogram had a significantly higher discrimination capability (Figure 5B). With respect to calibration capability, the calibration curve indicated a close consistency between the predicted and actual risks (Figure 5C). In internal validation, the nomogram predictive model demonstrated good concordance with the bootstrapped corrected model (Figure 5D). Further, it showed reasonably acceptable accuracy, with a Brier score of 0.154. In addition, decision curve analysis revealed that the nomogram yielded higher net benefits in predicting 1-year mortality than did the SOFA score (Figure 6). In external validation, the nomogram demonstrated strong discrimination, accuracy, and calibration power, as it achieved an AUC of 0.877 (95% CI: 0.812–0.941), a bias-corrected AUC of 0.844, a Brier score of 0.144, a bias-corrected Brier score of 0.171, and well-fitted calibration curves (Supplementary Figure 2). Moreover, Delong test and the decision curve analysis showed that the nomogram had better prognostic ability than the SOFA score in external validation (Supplementary Figure 3).

FIGURE 5

Figure 5. The receiver operating characteristic (ROC) curve and calibration curve of nomogram. (A) the ROC curves of the nomogram in discovery cohort. (B) Comparison of AUC between the nomogram and SOFA. (C) The calibration curve of the nomogram. (D) Comparison of the calibration curves between the ideal model, the nomogram and the bias-corrected model.

FIGURE 6

Figure 6. The decision analysis curves of the nomogram and SOFA in discovery cohort.

Machine Learning Models: Selection and Evaluation

The metrics for generalization capability, such as accuracy and precision, for the various ML models are presented in Table 4, while the AUCs are shown in Figure 7. The RF model showed the highest accuracy and AUC at 0.765 and 0.854, respectively. The results of feature ranking are displayed in Figure 6A. Vasopressor use, serum MYO level on day 3 of diagnosis, rate of change in serum MYO, serum cTnI level on days 1 and 3, history of hypertension, and the SOFA score were important influencing factors of 1-year survival. After inputting the features of interest to the decision tree algorithm and post-pruning, the final decision tree was obtained (Figure 8). Based on this RF model, patients aged > 74.5 years with SOFA scores > 10.5 points and cTnI > 1.5 ng/mL on the first day of diagnosis, had a very high risk (> 75%) of 1-year mortality. In external validation, the RF model also showed good predictive ability by achieving an AUC of 0.863, a precision of 0.775 and a recall of 0.821 (Supplementary Figure 4).

TABLE 4

Table 4. The metrics of different machine learning models.

FIGURE 7

Figure 7. The feature importance and the ROC curves of machine learning models. (A) The feature ranking analysis produced by Random Forest. (B) ROC curves of Random Forest. (C) ROC curves of support vector machine (SVM). (D) ROC curves of decision tree. (E) ROC curves of gradient boosted decision tree (GBDT).

FIGURE 8

Figure 8. A pruned decision tree for predicting risk stratification of 1-year mortality in sepsis-induced CRS patients.

Discussion

The prognostic factors of sepsis-induced CRS are yet to be established. We utilized blood tests which were routinely collected in clinical setting and determined the following five important risk factors that composed the predictive nomogram: age, SOFA, serum MYO levels, use of vasopressors, and mechanical ventilation during hospitalization. We also found that age, SOFA score, serum MYO levels, and vasopressor use were important predictors for the 1-year prognosis in ML models. The nomogram and ML models developed in our study showed good predictive ability and might help clinicians evaluate sepsis severity and predict long-term outcomes in the early diagnostic period.

Sepsis is a critical acute illness that is frequently associated with multiple organ dysfunction and high mortality rates. Thus, timely prediction of the long-term prognosis in patients with sepsis-induced CRS is of great importance in clinic therapy and post-acute care. The SOFA score is known to play a significant role in the diagnosis and in-hospital prognosis of sepsis patients (1, 20). However, even though SOFA includes assessment of the heart and kidney, the SOFA score alone cannot adequately predict the long-term prognosis of patients with sepsis-induced acute CRS. The current study identified five independent risk factors that strongly affect the 1-year prognosis. The nomogram comprising of these five factors along with baseline Scr showed significantly higher discrimination power and yielded greater net benefits in predicting 1-year outcomes in comparison to the model which used the SOFA score alone. Interestingly, we found serum MYO to be an independent risk factor in both multiple logistic regression and ML algorithms. MYO is a heme protein expressed in cardiomyocytes and skeletal muscle cells (21). Serum MYO levels are elevated in patients with acute myocardial infarction, renal insufficiency, muscle trauma, shock, and severe infection (22, 23). Recent studies have shown that serum MYO is strongly correlated with the severity of sepsis, and an increase in the serum MYO levels within 28 days after diagnosis indicates a higher risk for mortality (24). Yang et al. also found MYO to be an independent prognostic factor for septic shock (25). In sepsis-induced CRS, MYO is released from impaired cardiomyocytes and accumulates in the blood stream. It then deposits in the kidney tissue, further exacerbating kidney injury, and causing oxidative damage and lipid peroxidation (26). Our study showed that serum MYO levels > 500 ng/ml on the third day of diagnosis plays an important role in promoting long-term morbidity in patients with sepsis-induced CRS. To the best of our knowledge, our study is the first to report that the serum MYO level is an independent predictor of 1-year mortality in patients with sepsis-induced CRS.

Another interesting finding is that the higher demand for vasopressors or mechanical ventilation may have a profound impact on long-term health loss and therefore result in poorer outcomes. In 2001, Rivers et al. (27) recommended early goal-directed therapy (EGDT) that included the early use of vasopressors for patients with septic shock. However, subsequent research showed no difference in 90-day mortality rates between EGDT and usual care (28). Therefore, the effect of vasopressors on clinical outcomes remains controversial despite their rapid and significant effect on blood pressure. As for mechanical ventilation, previous studies have shown that activation of TLR4 in sepsis can induce increased expression of integrin β5, which can cause higher susceptibility to mechanical ventilation-related acute lung injury (29). Other studies have also suggested that sepsis patients are at risk of diaphragm contraction dysfunction after mechanical ventilation, which could lead to a higher possibility of weaning failure (30, 31). Similarly, our study found a correlation between mechanical ventilation and a worsening 1-year mortality, which may be associated with mechanical ventilation-related lung injury that could result in long-term organ dysfunction and decline in quality of life.

Sepsis patients with pre-existing kidney injury have a higher risk of developing AKI and multi-organ failure (32). However, there are conflicting reports on the outcomes of AKI or acute renal failure in patients with CKD. Several studies have reported higher mortality rates in acute renal failure patients with lower Scr levels (33, 34). Neyra et al. also reported that AKI (stage ≥ 2) in CKD is strongly associated with worse outcomes in patients with severe sepsis (35). However, our findings suggest that neither baseline renal function nor sepsis-induced AKI greatly influenced 1-year mortality. Treatment for AKI or CKD, such as renal replacement therapy, may have a positive effect on lowering the health impact of sepsis-induced AKI or CKD.

Regarding ML, we found that the classifier trained by the RF algorithm yielded the optimal predictive capability. The risk factors identified in the logistic regression model (e.g., age, vasopressor, MYO, and SOFA score) also ranked high in the feature analysis. However, we observed a few differences between the results of logistic regression and those of ML algorithm. First, cTnI was transformed and treated as a categorical variable throughout the logistic regression analysis owing to the non-linear relationship between cTnI and 1-year mortality. The results showed that cTnI did not influence 1-year mortality. In the ML algorithms, cTnI and MYO were all treated as numeric variables, and the results presented cTnI as an important prognostic factor. Second, given the limited sample size, only a few variables were included in the logistic regression model to prevent overfitting, and the total rather than the individual SOFA score per organ was included in the analysis. Under the same sample size, more variables were included in the ML models. The results showed that in addition to the total SOFA score, the cardiovascular and liver SOFA scores also had a significant effect on 1-year survival. Third, unlike the nomogram from the logistic regression model, the decision tree presented us with a flowchart of prognosis judgment in a risk-stratified manner. Although the decision tree had lower predictive capability for mortality than the nomogram, it had the advantage of including fewer variables in decision making, thus making it more efficient in risk judgment. Finally, in our study, the results of external validation indicated that RF model does not show a significantly higher predictive ability than the nomogram, which may be due to the limited sample size and predictor variables included in this study. We believe that this study serves as a preliminary exploration of ML models to predict mortality in sepsis-induced CRS, and a large-scale, multi-center clinical research should be conducted in the future to further improve and update the ML models.

Our study has some limitations. First, the development and external validation of the models were both performed in the dataset collected from Shanghai Tongji Hospital. Hence, validation of these models in different institutions is required before clinical application. Second, we recognize that apart from the predictors analyzed in this study, factors such as antibiotic use and left ventricular ejection fraction may also affect the outcome. Antibiotics were practically used in every patient with sepsis-induced CRS, but the type, dosage, and timing of the antibiotics varied greatly in the different patients, thus making it difficult to use a unified standard for data collection in this retrospective study. We also noticed that changes in left ventricular ejection fraction often reflected the severity of acute myocardial dysfunction. However, in the process of data collection, we found that the detection rate with echocardiography was apparently low, which may be due to the inconvenience associated with critically ill patients leaving the ward and the lack of utilization of bedside echocardiography outside the ICU and cardiology ward. In addition, the limited sample size has constrained the predictors included in the study. Therefore, we believe that more potential predictors should be included in the prospective study under a larger sample size. Third, new biomarkers for AKI, such as neutrophil gelatinase-related lipocalin and renal injury molecule 1, were not considered in this study as they were not tested in Shanghai Tongji Hospital during the study period. Future studies should include more sensitive biomarkers for model development.

Conclusion

In conclusion, we constructed and internally validated predictive nomogram and ML models for the prediction of poor 1-year outcomes in patients with sepsis-induced CRS. The predictive models were developed by utilizing objective data routinely collected in clinical practice. Six optimal factors for predicting the 1-year prognosis were used to develop the clinical nomogram; they include age, SOFA score, mechanical ventilation, vasopressor use, baseline Scr, and serum MYO levels. Using both multiple logistic regression and ML algorithms, we found that age, SOFA score, vasopressor use, and serum MYO levels were key prognostic factors. In our study, both models showed strengths in predicting poor 1-year outcomes, and we believe this combination might be more clinically useful than the nomogram or ML model alone. Early prediction of long-term prognosis in patients with sepsis-induced CRS is of great importance and may greatly influence clinical treatment and post-acute care.

Data Availability Statement

The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author/s.

Ethics Statement

The studies involving human participants were reviewed and approved by the ethics committee of Shanghai Tongji Hospital. Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.

Author Contributions

CY, YL, and YyZ contributed to conception and design of the study. YL collected the clinical data, organized the database, and wrote the first draft of the manuscript. YL and YyZ performed the statistical analysis. YyZ, XZ, YfZ, YJ, and CY revised the manuscript. All authors have contributed to the article and approved the submitted version.

Funding

This work was supported by the National Natural Science Foundation of China (Grant Nos. 81900622, 81873609, and 81800631), the Shanghai Sailing Program (Grant Nos. 18YF1421300, 19YF1444200, and 19YF1444400), and the Medical Guidance Technology Support Program from Shanghai Science and Technology Committee (Grant No. 18411960900).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

We thank Huiqiang Wu for guidance in the statistical methods in our study and Tian-En Chen and Chao Zheng for participating in data collection.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fmed.2022.792238/full#supplementary-material

Supplementary Figure 1 | Flow chart illustrating the patient enrollment in validation cohort.

Supplementary Figure 2 | The receiver operating characteristic (ROC) curve and calibration curve of nomogram in external validation. (A) The ROC curves of the nomogram; (B) comparison of AUC between the nomogram and SOFA; (C) the calibration curve of the nomogram; (D) comparison of the calibration curves between the ideal model, the nomogram and the bias-corrected model.

Supplementary Figure 3 | The decision analysis curves of the nomogram and SOFA in external validation.

Supplementary Figure 4 | The ROC curves of machine learning models in external validation. (A) ROC curves of Random Forest; (B) ROC curves of decision tree.

Abbreviations

CRS, cardiorenal syndrome; AUC, area under the curve; ML, machine learning; SOFA, Sequential Organ Failure Assessment; MYO, myoglobin; Scr, serum creatinine; AKI, acute kidney injury; BNP, type B natriuretic peptide; cTnI, cardiac troponin I; NT-proBNP, N-terminal pro-B-type natriuretic peptide; RF, random forest.

References

1. Singer M, Deutschman CS, Seymour CW, Shankar-Hari M, Annane D, Bauer M, et al. The third international consensus definitions for sepsis and septic shock (Sepsis-3). JAMA. (2016) 315:801–10. doi: 10.1001/jama.2016.0287

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Kotecha A, Vallabhajosyula S, Coville HH, Kashani K. Cardiorenal syndrome in sepsis: a narrative review. J Crit Care. (2018) 43:122–7. doi: 10.1016/j.jcrc.2017.08.044

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Kakihana Y, Ito T, Nakahara M, Yamaguchi K, Yasuda T. Sepsis-induced myocardial dysfunction: pathophysiology and management. J Intensive Care. (2016) 4:22. doi: 10.1186/s40560-016-0148-1

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Ronco C, Haapio M, House AA, Anavekar N, Bellomo R. Cardiorenal syndrome. J Am Coll Cardiol. (2008) 52:1527–39. doi: 10.1016/j.jacc.2008.07.051

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Vallabhajosyula S, Sakhuja A, Geske JB, Kumar M, Kashyap R, Kashani K, et al. Clinical profile and outcomes of acute cardiorenal syndrome type-5 in sepsis: an eight-year cohort study. PLoS One. (2018) 13:e0190965. doi: 10.1371/journal.pone.0190965

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Ronco C, McCullough P, Anker SD, Anand I, Aspromonte N, Bagshaw SM, et al. Cardio-renal syndromes: report from the consensus conference of the acute dialysis quality initiative. Eur Heart J. (2010) 31:703–11. doi: 10.1093/eurheartj/ehp507

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Smith JB, Shew M, Karadaghy OA, Nallani R, Sykes KJ, Gan GN, et al. Predicting salvage laryngectomy in patients treated with primary nonsurgical therapy for laryngeal squamous cell carcinoma using machine learning. Head Neck. (2020) 42:2330–9. doi: 10.1002/hed.26246

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Grimes DA. The nomogram epidemic: resurgence of a medical relic. Ann Intern Med. (2008) 149:273–5. doi: 10.7326/0003-4819-149-4-200808190-00010

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Dong D, Fang MJ, Tang L, Shan XH, Gao JB, Giganti F, et al. Deep learning radiomic nomogram can predict the number of lymph node metastasis in locally advanced gastric cancer: an international multicenter study. Ann Oncol. (2020) 31:912–20. doi: 10.1016/j.annonc.2020.04.003

PubMed Abstract | CrossRef Full Text | Google Scholar

10. He Y, Zhu Z, Chen Y, Chen F, Wang Y, Ouyang C, et al. Development and validation of a novel diagnostic nomogram to differentiate between intestinal tuberculosis and Crohn’s disease: a 6-year Prospective Multicenter Study. Am J Gastroenterol. (2019) 114:490–9. doi: 10.14309/ajg.0000000000000064

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Li L, Zhang B, He B, Gong Z, Chen X. Critical patients with coronavirus disease 2019: risk factors and outcome nomogram. J Infect. (2020) 80:e37–8. doi: 10.1016/j.jinf.2020.03.025

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Wang S, Yang L, Ci B, Maclean M, Gerber DE, Xiao G, et al. Development and validation of a nomogram prognostic model for SCLC patients. J Thorac Oncol. (2018) 13:1338–48. doi: 10.1016/j.jtho.2018.05.037

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Dong YM, Sun J, Li YX, Chen Q, Liu QQ, Sun Z, et al. Development and validation of a nomogram for assessing survival in patients with COVID-19 pneumonia. Clin Infect Dis. (2021) 72:652–60. doi: 10.1093/cid/ciaa963

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Jeong SH, Kim RB, Park SY, Park J, Jung EJ, Ju YT, et al. Nomogram for predicting gastric cancer recurrence using biomarker gene expression. Eur J Surg Oncol. (2020) 46:195–201. doi: 10.1016/j.ejso.2019.09.143

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Alabi RO, Mäkitie AA, Pirinen M, Elmusrati M, Leivo I, Almangush A. Comparison of nomogram with machine learning techniques for prediction of overall survival in patients with tongue cancer. Int J Med Inform. (2021) 145:104313. doi: 10.1016/j.ijmedinf.2020.104313

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Kellum JA, Lameire N KDIGO AKI Guideline Work Group. Diagnosis, evaluation, and management of acute kidney injury: a KDIGO summary (Part 1). Crit Care. (2013) 17:204. doi: 10.1186/cc11454

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Martin L, Derwall M, Al Zoubi S, Zechendorf E, Reuter DA, Thiemermann C, et al. The septic heart: current understanding of molecular mechanisms and clinical implications. Chest. (2019) 155:427–37. doi: 10.1016/j.chest.2018.08.1037

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Rangaswami J, Bhalla V, Blair JEA, Chang TI, Costa S, Lentine KL, et al. Cardiorenal syndrome: classification, pathophysiology, diagnosis, and treatment strategies: a scientific statement from the American Heart Association. Circulation. (2019) 139:e840–78. doi: 10.1161/CIR.0000000000000664

PubMed Abstract | CrossRef Full Text | Google Scholar

19. De Rosa S, Samoni S, Ronco C. Creatinine-based definitions: from baseline Scr to Scr adjustment in intensive care. Crit Care. (2016) 20:69. doi: 10.1186/s13054-016-1218-4

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Karakike E, Kyriazopoulou E, Tsangaris I, Routsi C, Vincent JL, Giamarellos-Bourboulis EJ. The early change of SOFA score as a prognostic marker of 28-day sepsis mortality: analysis through a derivation and a validation cohort. Crit Care. (2019) 23:387. doi: 10.1186/s13054-019-2665-5

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Ordway GA, Garry DJ. Myoglobin: an essential hemoprotein in striated muscle. J Exp Biol. (2004) 207:3441–6. doi: 10.1242/jeb.01172

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Kurt-Mangold M, Drees D, Krasowski MD. Extremely high myoglobin plasma concentrations producing hook effect in a critically ill patient. Clin Chim Acta. (2012) 414:179–81. doi: 10.1016/j.cca.2012.08.024

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Gilkeson G, Stone MJ, Waterman M, Ting R, Gomez-Sanchez CE, Hull A, et al. Detection of myoglobin by radioimmmunoassay in human sera: its usefulness and limitations as an emergency room screening test for acute myocardial infarction. Am Heart J. (1978) 95:70–7. doi: 10.1016/0002-8703(78)90398-8

CrossRef Full Text | Google Scholar

24. Yao L, Liu Z, Zhu J, Li B, Chai C, Tian Y. Higher serum level of myoglobin could predict more severity and poor outcome for patients with sepsis. Am J Emerg Med. (2016) 34:948–52. doi: 10.1016/j.ajem.2016.01.009

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Yang B, Wang J, Tao X, Wang D. Clinical investigation on the risk factors for prognosis in patients with septic shock. Zhonghua Wei Zhong Bing Ji Jiu Yi Xue. (2019) 31:1078–82. doi: 10.3760/cma.j.issn.2095-4352.2019.09.004

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Shaver CM, Paul MG, Putz ND, Landstreet SR, Kuck JL, Scarfe L, et al. Cell-free hemoglobin augments acute kidney injury during experimental sepsis. Am J Physiol Renal Physiol. (2019) 317:F922–9. doi: 10.1152/ajprenal.00375.2018

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Rivers E, Nguyen B, Havstad S, Ressler J, Muzzin A, Knoblich B, et al. Early goal-directed therapy in the treatment of severe sepsis and septic shock. N Engl J Med. (2001) 345:1368–77. doi: 10.1056/NEJMoa010307

PubMed Abstract | CrossRef Full Text | Google Scholar

28. PRISM Investigators, Rowan KM, Angus DC, Bailey M, Barnato AE, Bellomo R, et al. Early, goal-directed therapy for septic shock - A patient-level meta-analysis. N Engl J Med. (2017) 376:2223–34. doi: 10.1056/NEJMoa1701380

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Ding X, Tong Y, Jin S, Chen Z, Li T, Billiar TR, et al. Mechanical ventilation enhances extrapulmonary sepsis-induced lung injury: role of WISP1-αvβ5 integrin pathway in TLR4-mediated inflammation and injury. Crit Care. (2018) 22:302. doi: 10.1186/s13054-018-2237-0

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Lu ZH, Guo F, Zhang G, Ge HQ, Xu LL, Chen YL. A clinical study of diaphragmatic dysfunction in subjects with mechanical ventilation in sepsis. Zhonghua Jie He He Hu Xi Za Zhi. (2018) 41:696–700. doi: 10.3760/cma.j.issn.1001-0939.2018.09.009

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Goligher EC, Dres M, Fan E, Rubenfeld GD, Scales DC, Herridge MS, et al. Mechanical ventilation-induced diaphragm atrophy strongly impacts clinical outcomes. Am J Respir Crit Care Med. (2018) 197:204–13. doi: 10.1164/rccm.201703-0536OC

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Doi K. Role of kidney injury in sepsis. J Intensive Care. (2016) 4:17. doi: 10.1186/s40560-016-0146-3

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Mehta RL, Pascual MT, Gruta CG, Zhuang S, Chertow GM. Refining predictive models in critically ill patients with acute renal failure. J Am Soc Nephrol. (2002) 13:1350–7. doi: 10.1097/01.asn.0000014692.19351.52

CrossRef Full Text | Google Scholar

34. Chertow GM, Soroko SH, Paganini EP, Cho KC, Himmelfarb J, Ikizler TA, et al. Mortality after acute renal failure: models for prognostic stratification and risk adjustment. Kidney Int. (2006) 70:1120–6. doi: 10.1038/sj.ki.5001579

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Neyra JA, Mescia F, Li X, Adams-Huet B, Yessayan L, Yee J, et al. Impact of acute kidney injury and CKD on adverse outcomes in critically ill septic patients. Kidney Int Rep. (2018) 3:1344–53. doi: 10.1016/j.ekir.2018.07.016

PubMed Abstract | CrossRef Full Text | Google Scholar