Machine learning for the prediction of acute kidney injury in patients after cardiac surgery

Cardiac surgery-associated acute kidney injury (CSA-AKI) is the most prevalent major complication of cardiac surgery and exerts a negative effect on a patient's prognosis, thereby leading to mortality. Although several risk assessment models have been developed for patients undergoing cardiac surgery, their performances are unsatisfactory. In this study, a machine learning algorithm was employed to obtain better predictive power for CSA-AKI outcomes relative to statistical analysis. In addition, random forest (RF), logistic regression with LASSO regularization, extreme gradient boosting (Xgboost), and support vector machine (SVM) methods were employed for feature selection and model training. Moreover, the calibration capacity and differentiation ability of the model was assessed using net reclassification improvement (NRI) along with Brier scores and receiver operating characteristic (ROC) curves, respectively. A total of 44 patients suffered AKI after surgery. Fatty acid-binding protein (FABP), hemojuvelin (HJV), neutrophil gelatinase-associated lipocalin (NGAL), mechanical ventilation time, and troponin I (TnI) were correlated significantly with the incidence of AKI. RF was the best model for predicting AKI (Brier score: 0.137, NRI: 0.221), evidenced by an AUC value of 0.858 [95% confidence interval (CI): 0.792–0.923]. Overall, RF exhibited the best performance as compared to other machine learning algorithms. These results thus provide new insights into the early identification of CSA-AKI.

Introduction

Acute kidney injury (AKI) is the most frequent major complication of cardiac surgery (1). Annually, over two million cardiac procedures are performed globally, and the incidence of CSA-AKI ranges from 5% to 42.3% (2, 3). Cardiac surgery is the second most frequent cause underlying AKI in the intensive care unit (ICU), resulting in three- to eight-fold higher perioperative mortality, prolonged ICU and hospital stay, and increased healthcare costs for patients with severe AKI (4, 5). At present, renal replacement therapy is the only option available for patients with advanced severe CSA-AKI due to the lack of effective therapies. Therefore, early detection of AKI will provide clinicians with the necessary guidance for its prevention and management (6). Existing clinical risk assessments are based on changes in the levels of serum creatinine (Scr) and reported risk factors. However, these cannot accurately identify patients with AKI due to limitations in their sensitivity and specificity, thus resulting in missed optimal treatment timing. Accumulating evidence focuses on novel biomarkers and clinical prediction models to identify risk factors for AKI and improve its diagnostic efficiency and accuracy (7–10).

Predictive models for cardiovascular surgery are critical for patient selection, risk stratification, tailoring treatment, and prognostic prediction. Accurate perioperative risk prediction for complications such as AKI may help in better informing these patients and their families of the risks and assist in clinical management (11). Many perioperative prediction models for AKI have been developed in recent years. However, the general utility of these models is poor due to differences in variable selection (7, 12, 13). Consequently, there is neither a consensus nor guidelines recommending the use of the existing predictive models for AKI after cardiac surgery.

Machine learning algorithms and advanced statistical tools can be used to predict the outcome of complex datasets based on iterative learning, thereby making these models more accurate and stable through the selection of variable features (14, 15). Therefore, machine learning algorithms were used in this study to develop and validate feature variables for predicting perioperative AKI risk and generating robust prediction models, in addition to traditional logistic regression analyses. We collected the patients' perioperative demographic characteristics, clinical laboratory data, and intraoperative and postoperative clinical data. Subsequently, a feature selection machine learning strategy was used to develop multivariate models to predict the risk of severe CSA-AKI.

Material and methods

Data sources

Patients who were admitted to the ICU (aged 18 years or above) after cardiac surgery by cardiopulmonary bypass (CPB) at the Department of Cardiothoracic Surgery of Nanjing First Hospital between 1 December 2019, and 30 April 2020, were enrolled. The cardiac surgery procedures included valve replacement, coronary artery bypass surgery, large vessel surgery, combined surgery, and congenital heart disease correction surgery.

The exclusion criteria in this study were as follows: (1) renal insufficiency or acute and chronic kidney disease; (2) recent administration of renal impairment drugs or glucocorticoid drugs; (3) recent or postoperative concomitant urinary tract infection; (4) preoperative hemodynamic instability, and (5) emergency surgery.

All patients provided written informed consent before participating in this study. The study design was approved by the Medical Ethics Committee of Nanjing First Hospital (KY20190404-03-KS-01) and adhered to the Declaration of Helsinki.

Diagnostic criteria and outcome definitions

The outcome of interest was the occurrence of AKI during the perioperative period. AKI was defined as follows: the diagnosis was confirmed according to the latest diagnostic criteria for AKI in the 2012 KDIGO guidelines (16) if one of the following conditions was met: (1) Scr elevation ≥0.3 mg/dl ≥26.5 µmol/L within 48 h; (2) known or presumed Scr elevation ≥1.5 times baseline occurring within seven days, or (3) sustained six h urine volume <0.5 ml/kg/h. Baseline preoperative Scr value was defined as the last Scr value detected within seven days before performing cardiac surgery with CPB.

Model development

First, support vector machine (SVM), least absolute shrinkage and selection operator (LASSO) regression, extreme gradient boosting (XGBoost), and Random Forest and Boruta (RFB) algorithms were used for filtering out the crucial clinical variables. Subsequently, the feature variables were derived to construct and validate the model.

A stratified five-fold cross-validation was performed to obtain the derivation and validation cohorts. The study population was randomly classified into five subsets with similar event rates. Four subsets (80%) were combined to form the derivation cohort, whereas the remaining (20%) was retained as the validation set. This process was repeated five times for each outcome so that each subset could serve as the validation set, thus accounting for inter-patient variability and providing a risk estimate for all cases. Since five numerical features (fatty acid-binding protein [FABP], NT-prBNP, troponin I (TnI), ultrafiltration volume, and urine dropout) had <20% missing data, predictive mean matching was used to fill in the incomplete information.

Four common machine learning algorithms and conventional logistic regression were employed to train the models, including logistic regression with LASSO regularization (logistic LASSO), logistic regression with forwarding selection variables, RF, SVM, and xgboost. The basic GLM functions were used for logistic regression. In addition, packages including glmnet, randomForest, xgboost, and e1071 were used for LASSO (17), RF (18), Xgboost (15), and SVM (19) analyses, respectively. Logistic regression was performed along with forward stepwise selection to determine the average C-index improvement for each added variable. In high-dimensional problems, backward selection techniques may be prone to greater noise. In contrast, forward selection leads to strong theoretical guarantees and excellent empirical behavior. In addition, logistic LASSO models were constructed by five-rule cross-validation and default lambda based on minimum classification errors. The number of trees for RF was 1,000, with 50 perturbation counts. Moreover, the number of trees for xgboost was 100, with a learning rate of 1 per tree, a maximum depth of 2 and trained through 50 iterations. Finally, two different kernels (linear and radial-based functions) were used in the SVM algorithm to obtain the separation function.

Model performance

Receiver operating characteristic (ROC) curves were used to estimate the model's discrimination capacity by calculating its area under the curve (AUC). Confusion matrices were plotted to assess the model's effectiveness based on multiple metrics. In addition, net reclassification improvement (NRI) was used to assess the correct reassignment between risk categories (20). Predictive calibration plots were used to plot the average risk scores relative to the observed outcome rates. In addition, the probability of the optimal performance model was assessed using the Brier score, which was defined as the mean squared error between the observed and predicted outcomes. The Brier score ranges from 0 to 1.00, with the former representing the best possible calibration. The prediction for each patient was plotted in the order of their risk to assess the predictive distribution of the model (15). Furthermore, decision curve analysis (DCA) curves were plotted to assess the discriminability of each selected factor to predict severe CSA-AKI (21). Next, the nomogram of the optimal model was constructed using the “rms” package in R. Finally, the Hosmer–Lemeshow test was conducted to assess the fit of the nomogram (22).

Statistical analyses

All graphs were plotted and analyses were performed using the R software (version: 4.1.0). Continuous variables were compared by two-tailed t-tests, whereas Fisher's exact test was used for categorical data. The significance level was set at P < 0.05 unless specified otherwise.

Results

Patient characteristics and outcomes

A total of 215 patients underwent cardiac surgery under CPB between December 1, 2019, and April 30, 2020, of which 135 were enrolled (the flow chart in Supplementary Figure S1 details inclusion and exclusion criteria and selection of the final study cohort for further analyses). According to the KDIGO clinical practice guidelines, 44 patients suffered from hospital-acquired AKI within one week of cardiac surgery; the incidence rate was 32.59%. Baseline characteristics of the patients are shown in Table 1. Moreover, the non-AKI and AKI groups included 49.5% and 65.9% of men and the median EF was 60.97% and 58.14%, respectively. The mean ICU length of stay and mean mechanical ventilation time were longer in the AKI group than those in the non-AKI group (3.05 vs. 1.48, P < 0.001; 21.27 vs. 9.63, P < 0.001). However, no significant differences were observed in Cleveland clinical scores between the two groups.

TABLE 1

Table 1. Demographic and clinical characteristics of patients with or without AKI after cardiac surgery.

Machine learning algorithms for variable selection

Thirty-four baseline clinical characteristics with at least 70% complete data were considered predictors of CSA-AKI. Table 2 summarizes the simple logistic regression and ROC analyses of clinical characteristics. Four machine learning algorithms (LASSO, RFB, SVM, and XGBoost) were used for the entire dataset to identify the most important clinical variables for AKI prediction, yielding 15, 11, 16, and 18 clinical variables, respectively (Supplementary Figure S2, detailed list is provided in Table S1). In addition, five commonly shared clinical features among the four algorithms were included as variables in the final model, comprising FABP, hemojuvelin (HJV), mechanical ventilation time, neutrophil gelatinase-associated lipocalin (NGAL), and TnI (Figure 1A); their risk ratios were statistically significant (P < 0.05).

FIGURE 1

Figure 1. Construction and evaluation of the CSA-AKI prediction model. (A) The five most important clinical features screened using four machine learning algorithms in the entire cohort; (B) schematic diagram of six machine learning algorithms in the training set trained and validated for stable clinical models by five-fold cross-validation; (C) comparison of AUC values among machine learning models, with RF having the largest AUC value; (D) confusion matrix of the best model (RF) in the entire cohort; (E) calibration curve for the RF model; (F) distribution of predicted patient risk of CSA-AKI. CSA-AKI: cardiac surgery-associated acute kidney injury.

TABLE 2

Table 2. Univariate, multivariate logistic regression and ROC analysis.

Machine learning algorithms for outcome prediction

We constructed a machine learning classifier by five-rule cross-validation (Figure 1B). Subsequently, their performances were evaluated using ROC curves (Figure 1C). The classifier trained based on five clinical features could discriminate patients with CSA-AKI accurately. RF exhibited the best performance, with an AUC value of 0.858 (95% CI, 0.792–0.923). The specific evaluation results of the six algorithms are shown in Table 3. Among them, RF showed the best overall performance with an accuracy of 0.822 for predicting CSA-AKI. Since CSA-AKI is a clinical emergency, there is a need to accurately identify patients who are likely to develop CSA-AKI. Thus, recall is also a crucial indicator. The RF model also had the highest recall value (0.591). Furthermore, the RF model had the highest Kolmogorov–Smirnov (KS) value (0.600) relative to other models, suggesting a leading advantage in differentiating patients with CSA-AKI. RF also exhibited the greatest improvement in discrimination or classification of CSA-AKI as compared to conventional logistic regression (NRI = 0.221). The confusion matrix of the optimal model is shown in Figure 1D. Detailed results of the confusion matrix for all six models are provided in Supplementary Tables S1, S2.

TABLE 3

Table 3. Evaluation results of models for AKI risk of patients after cardiac surgery.

Calibration of models and predictive distributions

The final RF model was well calibrated. The mean Brier score of the model for predicting CSA-AKI was 0.137 (close to 0), indicating a well-calibrated model. Figure 1E shows the calibration curve for the model. The prediction distribution plot of the RF model incorporating patients sorted by risk order suggested positive clustering of patients with CSA-AKI (Figure 1F). Therefore, the RF model could accurately stratify the patients at risk of developing CSA-AKI.

Construction of tools for patient classification

The nomogram is a graphical representation of the association between clinical variables and the probability of a clinical event (e.g., critical illness). In addition, it provides an intuitive way to interpret predictive models. In this study, we constructed an intuitive nomogram to specifically quantify the risk of developing CSA-AKI based on the predicted values of the RF model and their characteristic clinical variables (Figure 2A). The results of the H–L test suggested that the nomogram was well calibrated (P > 0.05, Figure 2B). Furthermore, DCA suggested an increased net benefit of the nomogram in predicting CSA-AKI as compared to the RF model or the characteristic clinical variables alone (Figure 2C).

FIGURE 2

Figure 2. Quantifying patients’ risk of CSA-AKI. (A) Nomogram used to quantify the risk of CSA-AKI in patients. Redline presents the detailed score of a certain patient, with a total of 101 points and a 29.1% risk of developing CSA-AKI; (B) calibration curve for the nomogram; (C) DCA curves for performance comparisons between clinical characteristics alone, the RF model, and the nomogram by plotting the net benefit of the prediction model and clinical predictors against the threshold probabilities, wherein the horizontal axis represents the threshold (the reference probability of the patient receiving treatment) and the vertical axis represents the net benefit rate after subtracting the disadvantage from the advantage. Using the model, under the same threshold probability, a larger net benefit indicates that the patient can obtain the greatest benefit. DCA: decision curve analysis.

Discussion

Accurate prediction of prognosis is essential for patient-centric care, both for informing and selecting treatment strategies for inclusive decision-making. In this study, we used five alternative machine learning algorithms to characterize the risk of CSA-AKI incidence using postoperative hospitalization data in patients who underwent cardiac surgery. The RF model optimally stratified patients' risk with excellent calibration and good internal validation. In addition, clinical characteristics of patient risk that may be underestimated in clinical practice were identified, including FABP, HJV, mechanical ventilation time, NGAL, and TnI. These models provide a foundation for future clinical utility for patient care and accurate outcome risk stratification.

In addition to several published reports on predictive models or clinical scales to assess patients' risk of developing CSA-AKI, our findings extend this knowledge in several important ways. The commonly used validated scale in the clinical settings to predict CSA-AKI is the Cleveland score proposed by Thakar et al. in 2005 (23–25). However, in clinical practice, the influence of plasma and urine markers needs to be considered, in addition to the Cleveland score (26–28). Moreover, intraoperative and postoperative information is crucial to assess the risk of AKI (29). The candidate variables included in the model presented herein integrated intra- and postoperative clinical information of patients and biomarkers in urine and plasma and were based on a comprehensive set of variables that could explain the complex interactions and more accurately predict the incidence of CSA-AKI. Our final model demonstrated higher predictive efficacy as compared to traditional Cleveland scores and predictors alone. In previous studies, logistic regression models have been the traditional statistical approach for event prediction; however, machine learning can handle nonlinear interactions and combine more variables to improve predictive efficacy for dealing with data of higher dimensions (15, 30). We used advanced machine learning algorithms to assess the risk of such complex syndromes, which exhibited better performance than traditional logistic regression analysis.

Our model incorporated five common clinical characteristics and laboratory results that are available in most hospitals. However, <20% values of FABP and TnI were missing. Previous studies have not explicitly addressed the impact of missing values on predictive performances (15, 31, 32). Thus, we included missing variables systematically in the modeling approach. The final model exhibited satisfactory performance. However, it is recommended that all clinical characteristics be collected at the time of admission to gain more benefits from this model. Missing data for certain variables are inevitable in the real world, especially from small or poorly equipped hospitals. Therefore, our model also allows for interpolation and estimation of missing values.

The existing treatment modalities for CSA-AKI are severely inadequate, thus renal replacement therapy is the only available choice (6). Therefore, early intervention for patients at risk of CSA-AKI should be given high priority. In this study, we developed a model with a higher recall value to identify those at risk of CSA-AKI as early as possible. A total of 26 patients with AKI were identified in the entire cohort, of which only six were not distinguished as having AKI. The model offers more possibilities for early intervention and its clinical and economic value for postoperative cardiac management is high, especially considering the unusually rapid disease progression and the high mortality associated with CSA-AKI.

Research limitations

This study has some limitations. First, the model did not involve subsequent baseline characteristics of the patients due to the absence of follow-up data. Although a dynamic model incorporating baseline data during hospitalization may have a better phenotype, the present model could still be used to predict the incidence of CSA-AKI with reasonable accuracy using the postoperative clinical data. Second, patient data were obtained from a single hospital. Third, the predictive accuracy using clinical medications (e.g., commonly used vasopressors, inotropic agents, and some specific drugs used during surgery) and other data (e.g., novel biomarkers, imaging, environmental factors, and atherosclerotic burden) may further improve the model. The addition of these variables could be performed in the future to improve the RF model constructed in this study. Finally, subgroup analyses based on various procedures were not performed because of the small number of patients within each subgroup. Consequently, the clinical utility of predicting postoperative CSA-AKI based on the specific procedure type is limited.

Conclusion

In conclusion, we developed a model that integrated advanced machine learning algorithms and easily accessible patient characteristics to predict the risk of developing CSA-AKI among patients undergoing cardiac surgery. The model may provide powerful assistance to clinicians to identify patients with a higher risk of AKI early in the postoperative period. Overall, the findings of this study can assist in developing timely diagnostic and treatment strategies for the clinical management of patients undergoing cardiac surgery.

Data availability statement

The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author/s.

Ethics statement

Written informed consent was obtained from the individual(s) for the publication of any potentially identifiable images or data included in this article.

Author contributions

XX performed the data analyses and wrote the manuscript; ZL performed the experiment; TX contributed significantly to analysis and manuscript preparation; WC helped perform the analysis with constructive discussions; XC contributed to the conception of the study. All authors contributed to the article and approved the submitted version.

Acknowledgments

The authors hereby express their gratitude to all participants who supported the study.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fsurg.2022.946610/full#supplementary-material.

References

1. Bove T, Monaco F, Covello RD, Zangrillo A. Acute renal failure and cardiac surgery. HSR Proc Intensive Care Cardiovasc Anesth. (2009) 1(3):13–21.23439962

PubMed Abstract | Google Scholar

2. Weisse AB. Cardiac surgery: a century of progress. Tex Heart Inst J. (2011) 38(5):486–90.22163121

PubMed Abstract | Google Scholar

3. Hobson CE, Yavas S, Segal MS, Schold JD, Tribble CG, Layon AJ, et al. Acute kidney injury is associated with increased long-term mortality after cardiothoracic surgery. Circulation. (2009) 119(18):2444–53. doi: 10.1161/circulationaha.108.800011

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Mao H, Katz N, Ariyanon W, Blanca-Martos L, Adýbelli Z, Giuliani A, et al. Cardiac surgery-associated acute kidney injury. Cardiorenal Med. (2013) 3(3):178–99. doi: 10.1159/000353134

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Ortega-Loubon C, Fernández-Molina M, Carrascal-Hinojal Y, Fulquet-Carreras E. Cardiac surgery-associated acute kidney injury. Ann Card Anaesth. (2016) 19(4):687–98. doi: 10.4103/0971-9784.191578

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Wang Y, Bellomo R. Cardiac surgery-associated acute kidney injury: risk factors, pathophysiology and treatment. Nat Rev Nephrol. (2017) 13(11):697–711. doi: 10.1038/nrneph.2017.119

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Chen Z, Chen L, Yao G, Yang W, Yang K, Xiong C. Novel blood cytokine-based model for predicting severe acute kidney injury and poor outcomes after cardiac surgery. J Am Heart Assoc. (2020) 9(22):e018004. doi: 10.1161/jaha.120.018004

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Mishra J, Dent C, Tarabishi R, Mitsnefes MM, Ma Q, Kelly C, et al. Neutrophil gelatinase-associated lipocalin (NGAL) as a biomarker for acute renal injury after cardiac surgery. Lancet (London, England). (2005) 365(9466):1231–8. doi: 10.1016/s0140-6736(05)74811-x

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Katagiri D, Doi K, Honda K, Negishi K, Fujita T, Hisagi M, et al. Combination of two urinary biomarkers predicts acute kidney injury after adult cardiac surgery. Ann Thorac Surg. (2012) 93(2):577–83. doi: 10.1016/j.athoracsur.2011.10.048

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Parikh CR, Thiessen-Philbrook H, Garg AX, Kadiyala D, Shlipak MG, Koyner JL, et al. Performance of kidney injury molecule-1 and liver fatty acid-binding protein and combined biomarkers of AKI after cardiac surgery. Clin J Am Soc Nephrol. (2013) 8(7):1079–88. doi: 10.2215/cjn.10971012

PubMed Abstract | CrossRef Full Text | Google Scholar

11. McIlroy DR. Predicting acute kidney injury after cardiac surgery: much work still to be done. Br J Anaesth. (2021) 127(6):825–8. doi: 10.1016/j.bja.2021.09.005

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Lee HC, Yoon HK, Nam K, Cho YJ, Kim TK, Kim WH, et al. Derivation and validation of machine learning approaches to predict acute kidney injury after cardiac surgery. J Clin Med. (2018) 7(10):322. doi: 10.3390/jcm7100322

CrossRef Full Text | Google Scholar

13. Thongprayoon C, Hansrivijit P, Bathini T, Vallabhajosyula S, Mekraksakit P, Kaewput W, et al. Predicting acute kidney injury after cardiac surgery by machine learning approaches. J Clin Med. (2020) 9(6):1767. doi: 10.3390/jcm9061767

CrossRef Full Text | Google Scholar

14. Deo RC. Machine learning in medicine. Circulation. (2015) 132(20):1920–30. doi: 10.1161/circulationaha.115.001593

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Angraal S, Mortazavi BJ, Gupta A, Khera R, Ahmad T, Desai NR, et al. Machine learning prediction of mortality and hospitalization in heart failure with preserved ejection fraction. JACC Heart Fail. (2020) 8(1):12–21. doi: 10.1016/j.jchf.2019.06.013

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Khwaja A. KDIGO Clinical practice guidelines for acute kidney injury. Nephron Clin Pract. (2012) 120(4):c179–84. doi: 10.1159/000339789

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Friedman J, Hastie T, Tibshirani R. Regularization paths for generalized linear models via coordinate descent. J Stat Softw. (2010) 33(1):1–22. doi: 10.18637/jss.v033.i01

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Sauerbrei W, Royston P, Binder H. Selection of important variables and determination of functional form for continuous predictors in multivariable model building. Stat Med. (2007) 26(30):5512–28. doi: 10.1002/sim.3148

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Kim S. Margin-maximised redundancy-minimised SVM-RFE for diagnostic classification of mammograms. Int J Data Min Bioinform. (2014) 10(4):374–90. doi: 10.1504/ijdmb.2014.064889

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Mosley JD, Gupta DK, Tan J, Yao J, Wells QS, Shaffer CM, et al. Predictive accuracy of a polygenic risk score compared with a clinical risk score for incident coronary heart disease. JAMA. (2020) 323(7):627–35. doi: 10.1001/jama.2019.21782

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Making. (2006) 26(6):565–74. doi: 10.1177/0272989(06295361

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Nattino G, Pennell ML, Lemeshow S. Assessing the goodness of fit of logistic regression models in large samples: a modification of the Hosmer-Lemeshow test. Biometrics. (2020) 76(2):549–60. doi: 10.1111/biom.13249

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Thakar CV, Arrigain S, Worley S, Yared JP, Paganini EP. A clinical score to predict acute renal failure after cardiac surgery. J Am Soc Nephrol. (2005) 16(1):162–8. doi: 10.1681/asn.2004040331

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Candela-Toha A, Elías-Martín E, Abraira V, Tenorio MT, Parise D, de Pablo A, et al. Predicting acute renal failure after cardiac surgery: external validation of two new clinical scores. Clin J Am Soc Nephrol. (2008) 3(5):1260–5. doi: 10.2215/cjn.00560208

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Englberger L, Suri RM, Li Z, Dearani JA, Park SJ, Sundt TM 3rd, Validation of clinical scores predicting severe acute kidney injury after cardiac surgery. Am J Kidney Dis. (2010) 56(4):623–31. doi: 10.1053/j.ajkd.2010.04.017

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Schunk SJ, Zarbock A, Meersch M, Küllmar M, Kellum JA, Schmit D, et al. Association between urinary dickkopf-3, acute kidney injury, and subsequent loss of kidney function in patients undergoing cardiac surgery: an observational cohort study. Lancet. (2019) 394(10197):488–96. doi: 10.1016/s0140-6736(19)30769-x

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Elmedany SM, Naga SS, Elsharkawy R, Mahrous RS, Elnaggar AI. Novel urinary biomarkers and the early detection of acute kidney injury after open cardiac surgeries. J Crit Care. (2017) 40:171–7. doi: 10.1016/j.jcrc.2017.03.029

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Heringlake M, Charitos EI, Erber K, Berggreen AE, Heinze H, Paarmann H. Preoperative plasma growth-differentiation factor-15 for prediction of acute kidney injury in patients undergoing cardiac surgery. Crit Care. (2016) 20(1):317. doi: 10.1186/s13054-016-1482-3

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Crosina J, Lerner J, Ho J, Tangri N, Komenda P, Hiebert B, et al. Improving the prediction of cardiac surgery-associated acute kidney injury. Kidney Int Rep. (2017) 2(2):172–9. doi: 10.1016/j.ekir.2016.10.003

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Levy JJ, O’Malley AJ. Don’t dismiss logistic regression: the case for sensible extraction of interactions in the era of machine learning. BMC Med Res Methodol. (2020) 20(1):171. doi: 10.1186/s12874-020-01046-3

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Fritz BA, Cui Z, Zhang M, He Y, Chen Y, Kronzer A, et al. Deep-learning model for predicting 30-day postoperative mortality. Br J Anaesth. (2019) 123(5):688–95. doi: 10.1016/j.bja.2019.07.025

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Xue B, Li D, Lu C, King CR, Wildes T, Avidan MS, et al. Use of machine learning to develop and evaluate models using preoperative and intraoperative data to identify risks of postoperative complications. JAMA Network Open. (2021) 4(3):e212240. doi: 10.1001/jamanetworkopen.2021.2240

PubMed Abstract | CrossRef Full Text | Google Scholar