Machine learning algorithms to predict major bleeding after isolated coronary artery bypass grafting

Objectives: Postoperative major bleeding is a common problem in patients undergoing cardiac surgery and is associated with poor outcomes. We evaluated the performance of machine learning (ML) methods to predict postoperative major bleeding.

Methods: A total of 1,045 patients who underwent isolated coronary artery bypass graft surgery (CABG) were enrolled. Their datasets were assigned randomly to training (70%) or a testing set (30%). The primary outcome was major bleeding defined as the universal definition of perioperative bleeding (UDPB) classes 3–4. We constructed a reference logistic regression (LR) model using known predictors. We also developed several modern ML algorithms. In the test set, we compared the area under the receiver operating characteristic curves (AUCs) of these ML algorithms with the reference LR model results, and the TRUST and WILL-BLEED risk score. Calibration analysis was undertaken using the calibration belt method.

Results: The prevalence of postoperative major bleeding was 7.1% (74/1,045). For major bleeds, the conditional inference random forest (CIRF) model showed the highest AUC [0.831 (0.732–0.930)], and the stochastic gradient boosting (SGBT) and random forest models demonstrated the next best results [0.820 (0.742–0.899) and 0.810 (0.719–0.902)]. The AUCs of all ML models were higher than [0.629 (0.517–0.641) and 0.557 (0.449–0.665)], as achieved by TRUST and WILL-BLEED, respectively.

Conclusion: ML methods successfully predicted major bleeding after cardiac surgery, with greater performance compared with previous scoring models. Modern ML models may enhance the identification of high-risk major bleeding subpopulations.

Introduction

Postoperative bleeding is a significant cause of morbidity and mortality in patients undergoing cardiac surgery, particularly after coronary artery bypass grafting (CABG) because they often are treated with potent antithrombotic drugs (1). Excessive bleeding after cardiac surgery is related to significant blood product transfusion, infections, longer mechanical ventilation, prolonged intensive care unit (ICU) stay, emergency surgical re-exploration, acute kidney injury, sepsis, and thromboembolic events (2–4). In addition, several studies have reported that major bleeding after cardiac surgery was also associated with increased hospital costs (5, 6). Therefore, identifying patients at high risk of bleeding will allow clinicians to take additional precautionary action to mitigate risk.

Several risk stratification models have been developed to support clinical decision, making such as the TRUST and WILL-BLEED risk scores (7, 8). All the models in current use are based on logistic regression (LR), which relied on manually inputting variables. Missing the complex interactions among features may result in model misspecification (9). Artificial intelligence is a rapidly growing field in cardiovascular medicine (10). Machine learning (ML) is a sub-field of artificial intelligence that may produce better predictive ability than traditional statistical analysis to predict postoperative outcomes. These approaches better account for high-order, non-linear interactions between predictors and can gain more stable predictions (11). Despite the success of ML in other clinical applications, no study has yet examined the utility of modern ML models to predict postoperative major bleeding in patients with isolated CABG.

The goal of this study was to develop ML models to accurately predict major bleeding after isolated CABG. We also compared the performance of several ML approaches with the reference LR model, the and the TRUST and WILL-BLEED scores.

Materials and methods

Data source and study population

We analyzed electronic health record data from Fuwai Hospital, Beijing, China from 1,045 consecutive adult patients who underwent elective isolated CABG between September 2017 and May 2018. The subjects were assigned randomly to training and test sets at a ratio of 7:3. This retrospective cohort study was approved by the Institutional Review Board with waived written consent.

Study variables

We collected datasets that included demographics, surgery-related data, and clinical and laboratory parameters. The continuous anti-platelet therapy was provided only in patients with acute coronary syndrome presenting for CABG. All data were obtained from the electronic health records as in recent articles (12, 13). Our database initially comprised over 100 easily obtained variables. Features that were missing for more than 50% of cases were excluded. The final set of input features included 60 clinical variables. Missing values (Supplementary Table 2) were imputed using the mean imputation method, which replaces the missing values with the mean of the available cases.

Definition of major bleeding

Development of postoperative major bleeding was defined according to the universal definition of perioperative bleeding (UDPB) in classes 3 and 4 (14, 15). The UDPB defines 5 perioperative bleeding classes, which are intended to characterize the severity of bleeding, regardless of its source. The details of the UDPB are presented in Supplementary Table 1.

Statistical analysis

Data are presented as means with SD, medians and interquartile ranges (IQRs), or frequencies and proportions depending on variable type and distribution. Differences between groups were compared using the Student’s t-test for normally distributed continuous variables, the Mann-Whitney U-test for non-normally distributed continuous variables, and the Pearson’s chi-square test of independence for categorical variables.

To implement the ML algorithms, we randomly assigned patient data into a training set (70%) for model derivation and a testing set (30%) to be used for model validation. Assignment to the training/testing sets was stratified on primary outcome status.

To select features for LR models, we conducted a backward stepwise selection procedure based on the Akaike information criterion (AIC). Model development included several common ML methods, such as support vector machines (SVM), stochastic gradient boosting (SGBT), extreme gradient boosting (XGBoost), random forest (RF), conditional inference random forest (CIRF), boosted classification trees, Naïve Bayes (NB), and bagged classification and regression tree (CART). We report the variables included in each model in Supplementary Table 4. The details of the R package were presented in Supplementary Material. We conducted a fivefold cross-validation to tune the parameters of the ML models on the training set and determined the best hyperparameter. Discrimination was evaluated with the receiver-operating-characteristic curve (ROC) and the area under the ROC curve (AUROC) (16). To evaluate the performance of ML algorithms, we compared the ROC between the models using Delong’s test by the pROC package (17). Calibration, which is the agreement between predicted probabilities and observed frequencies of postoperative major bleeding, was assessed with calibration belts (18). The net reclassification improvement was used to quantify whether a new model provides clinically relevant improvements in prediction (19, 20). We graphically demonstrated the net benefit of each model through a range of threshold probabilities of the outcome as a decision curve. The variable importance from ensemble algorithms was also determined, with the top 20 important variables from each algorithm identified. Analysis code is available upon request. All statistical analyses were performed using R software (version 4.1.1; The Comprehensive R Archive Network).

Results

Baseline characteristics

We assigned 732 and 313 patients to the training and testing sets, respectively. The variables remained constant between the two sets (Supplementary Table 3). The postoperative major bleeding rate were similar between the training and test sets. Overall, the median age was 62 years (IQR 55–66 years) and 78.4% were men. At baseline, patients who developed a major bleeding event were older, had lower body surface area (BSA), and were more likely to have a history of anemia. Other demographics and perioperative variables of the training set are listed in Table 1.

TABLE 1

Table 1. Baseline characteristics of the training set.

Development of postoperative major bleeding prediction model

The overall major bleeding rate was 7.1% (74 of 1,045 patients) in this study population. The variables included in each model are presented in Supplementary Table 4. Multivariable LR analysis with backward stepwise variable selection based on AIC is shown in Table 2. ROC analysis showed an adequate discriminatory ability of the LR model [0.702 (0.577–0.827)]. The AUC of the CRIF and SVM models performed significantly better (p < 0.05) than the LR model in the test set. The discrimination ability of models selected in the testing set is presented in Table 3.

TABLE 2

Table 2. Final logistic regression model.

TABLE 3

Table 3. Prediction models of postoperative major bleeding in the test set.

We used the following ML methods in putting all the variables, including SVM, SGBT, XGBoost, RF, CIRF, boosted classification trees, NB, and bagged CART to predict major bleeding after isolated CABG. The ROC of the evaluated models are presented in Figure 1. The AUC of the CIRF model was 0.831 (0.732–0.930), which was the highest among the ML models. The SGBT and RF models achieved the next and the third-highest AUC value of 0.820 (0.742-0.899) and 0.810 (0.719–0.902), respectively. The AUC values of TRUST and WILL-BLEED for the prediction of major bleeding were 629 (0.517–0.741) and 0.557 (0.449–0.665), respectively. All ML models except the NB model performed remarkably better than the TRUST and WILL-BLEED risk scores (p < 0.05).

FIGURE 1

Figure 1. Receiver-operating-characteristics (ROC) curves of the conventional model and machine learning models for postoperative major bleeding in the test set. The corresponding values of the area under the receiver-operating-characteristics curve (AUC) for each model are presented in Table 2.

The decision curve analysis (Figure 2) showed that the net benefit of ML surpassed that of the conventional scoring systems throughout the range of threshold probabilities, indicating that the ML models yielded a higher net benefit. The threshold probability is a level of certainty, above which the patient or physician would choose to intervene.

FIGURE 2

Figure 2. Decision curve analysis of the conventional models and machine learning-based models. The X-axis indicates the threshold probability and Y-axis indicates the net benefit. The red solid line indicates the net benefit of all patients developing postoperative major bleeding events. Ochre’s solid line indicates the net benefit of no patients developing postoperative major bleeding events. LR, logistic regression; CIRF, conditional inference random forest; SGBT, stochastic gradient boosting.

The calibration belts of the ML models and the LR model for major bleeding are shown in Figure 3. The RF and SGBT models showed better calibration, while CIRF and LR models were not well-calibrated.

FIGURE 3

Figure 3. Calibration belts of (A) logistic regression. (B) Random forest. (C) Conditional inference random forest. (D) Stochastic gradient boosting for postoperative major bleeding prediction in the test set.

Variable importance of machine learning models

Figure 4 shows the importance matrix plot of machine learning models. To gain insights into the relevance of each predictor, Figure 4A summarizes the 20 most important predictors used in the CIRF model. Of the top 20 most important features, 15 were preoperative variables and 5 were intraoperative variables. In the CIRF model, intraoperative transfusion was the most important variable in predicting postoperative major bleeding, followed by operation time and BSA. The determinative factors in the ML and logistic models were roughly the same.

FIGURE 4

Figure 4. Variable importance of predictors in the ML models. The variable importance is a scaled measure to have a maximum value of 100. The predictors with variable importance of the top 20 are shown. (A) Conditional inference random forest. (B) Xgboost. (C) Random forest. (D) Stochastic gradient boosting.

Discussion

The clinical application of ML has emerged as a new tool to improve patient care. In this study, we compared the predictive accuracy of models for major bleeding after isolated CABG among ML techniques, a traditional statistical approach, and previous risk scoring models. The models developed using ML algorithms better predicted major bleeding events than conventional scoring systems, such as TRUST and WILL-BLEED.

To date, accurate prediction models that use statistical models for the risk of postoperative major bleeding have been lacking. The TRUST and WILL-BLEED scores using clinical and laboratory variables were developed using only preoperative factors (7, 8). These two prediction models were derived from multicenter databases and have demonstrated accurate prediction of bleeding or transfusion in cardiac surgery patients. However, these approaches showed poor performance in our dataset. Our study demonstrated the value of intraoperative and immediate postoperative data, which reflected the surgical-related factors to bleeding prediction. Integrating these features may elucidate the improved discrimination of our LR model compared with the TRUST and WILL-BLEED scores.

The LR model used in this study shares several risk factors for bleeding common to other prediction models. Interestingly, patients with hypertension could have a lower risk of postoperative major bleeding in this study. Patients with hypertension undergoing CABG had greater blood pressure variability, and lability as a protective marker demonstrating that autonomic nervous system integrity is associated with improved outcomes (21, 22). A retrospective analysis found that hypertension is a positive prognostic indicator in most patients undergoing anesthesia and surgery (23). A prior study has demonstrated that patients who developed major bleeding have lower systolic blood pressures than those who did not have major bleeding (24). These may be potential explanations for this contrary perception.

By capturing the non-linear association of predictors ML approaches may improve prediction in major bleeding (25). ML models in this study performed the best but at the cost of possibly increased complexity and lessened clinical significance. While overfitting in conventional models is often problematic, this study adopted a cross-validation method to mitigate the overfitting of ML models. A recent systematic review on the application of ML methods for predicting operative mortality following cardiac surgery demonstrated that ML models provide better discrimination ability (26). The XGBoost ML algorithm offered modest improvements across a variety of model performance measures compared with The Society of Thoracic Surgeons Predicted Risk of Mortality (STS PROM) (27). Zea-Vera and associates (28) recently developed an ML model to predict mortality, major morbidity, high total hospitalization cost, and 30-day re-admission. Nevertheless, Benedetto et al. (29) reported that the performance of ML techniques was not superior to an LR model in predicting operative mortality after cardiac surgery. Therefore, the additional value of ML in the development of prediction models for a variety of clinical conditions needs further investigation.

Decision curve analysis was developed as a method to determine whether the use of a prediction model in the clinic to inform decision-making would do more good than harm (30). This analysis indicated that the CIRF model improved the net benefit for predicting the postoperative major bleeding over the LR model and TRUST and WILL-BLEED scores. The threshold ranges displayed above the curves (in Figure 2) indicate how the ML models will apply to clinical practice. For example, if a patient’s threshold probability is 15%, according to the decision curve, the use of the CIRF model adds more benefit than either the treat-all-patients scheme or the treat-none scheme and even the other models.

Preventing postoperative bleeding is imperative to improve patient outcomes and decrease hospital costs. Prompt identification of patients at high-risk could help allow deployment of additional resources to achieve better outcomes, including precautions, such as preoperative correction of anemia, earlier discontinuation of antiaggregant drugs, detailed coagulation tests, better surgical homeostasis, and various procoagulant agent during the operation (31). Understanding which patients are at high risk for postoperative bleeding will also help guide future practice and mitigate worse outcomes associated with bleeding. Meticulous surgical technique aimed to reduce surgical site bleeding seems to be essential to prevent severe bleeding, exposure to blood products, and improve the prognosis after CABG.

Several potential limitations exist in our study. First, because it employed retrospective data from a single center at a tertiary hospital, the generalizability of these models will need further validation. Second, the study identified the most important variables concerning predicting postoperative major bleeding events, but we could not obtain certain degrees of risk, such as the relative risk, which is a common limitation of ML algorithms. In addition, we were only able to provide a comparison with the TRUST and WILL-BLEED scores and were unable to compute other risk scores because we lacked appropriate data. Finally, the most important variables reported in the ML model are not modifiable and accurate risk prediction may not be followed by improved patient outcomes. Thus, further prospective studies are required to evaluate the application of ML-based predictive models to clinical practice for the reduction of postoperative major bleeding risks.

Conclusion

Our study demonstrated that the ML technique of CIRF showed significantly better performance than the traditional LR analysis or previous scoring models in predicting postoperative major bleeding events for isolated CABG. By capturing the non-linear association of predictors, the ML approaches may facilitate optimal candidate selection and prognostication of patients undergoing isolated CABG. Future prospective studies are required to evaluate the application of predictive models based on ML to clinical practice and reduce bleeding risks.

Data availability statement

The dataset generated and analyzed during the current study is available from the corresponding author upon reasonable request.

Ethics statement

The studies involving human participants were reviewed and approved by the Ethics Committee of Fuwai Hospital. Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.

Author contributions

YG and HA: designed the study, collected and analyzed the data, drafted the manuscript, and submitted the final version. XL and HA: writing–reviewing and editing. YG, XL, and LW: visualization, coordinated, and supervised data collection. SW, YY, YD, and JW: collected, checked, and validated the data. All authors read and approved the manuscript for submission.

Acknowledgments

We gratefully acknowledge Jon K. Moon (Ph.D., President, MEI Research, Ltd.) for his help with the language polishing.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fcvm.2022.881881/full#supplementary-material

References

1. Knapik P, Knapik M, Zembala MO, Prybylowski P, Nadziakiewicz P, Hrapkowicz T, et al. In-hospital and mid-term outcomes in patients reoperated on due to bleeding following coronary artery surgery (from the KROK Registry). Interact Cardiovasc Thorac Surg. (2019) 29:237–43. doi: 10.1093/icvts/ivz089

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Billah B, Reid CM, Shardey GC, Smith JA. A preoperative risk prediction model for 30-day mortality following cardiac surgery in an Australian cohort. Eur J Cardiothorac Surg. (2010) 37:1086–92. doi: 10.1016/j.ejcts.2009.11.021

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Kinnunen EM, De Feo M, Reichart D, Tauriainen T, Gatti G, Onorati F, et al. Incidence and prognostic impact of bleeding and transfusion after coronary surgery in low-risk patients. Transfusion. (2017) 57:178–86. doi: 10.1111/trf.13885

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Stone GW, Clayton TC, Mehran R, Dangas G, Parise H, Fahy M, et al. Impact of major bleeding and blood transfusions after cardiac surgery: analysis from the acute catheterization and urgent intervention triage strategY (ACUITY) trial. Am Heart J. (2012) 163:522–9. doi: 10.1016/j.ahj.2011.11.016

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Newcomb AE, Dignan R, McElduff P, Pearse EJ, Bannon P. Bleeding after cardiac surgery is associated with an increase in the total cost of the hospital stay. Ann Thorac Surg. (2020) 109:1069–78. doi: 10.1016/j.athoracsur.2019.11.019

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Mehaffey JH, Hawkins RB, Byler M, Charles EJ, Fonner C, Kron I, et al. Cost of individual complications following coronary artery bypass grafting. J Thorac Cardiovasc Surg. (2018) 155:875.e–82.e. doi: 10.1016/j.jtcvs.2017.08.144

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Alghamdi AA, Davis A, Brister S, Corey P, Logan A. Development and validation of transfusion risk understanding scoring tool (TRUST) to stratify cardiac surgery patients according to their blood transfusion needs. Transfusion. (2006) 46:1120–9. doi: 10.1111/j.1537-2995.2006.00860.x

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Biancari F, Brascia D, Onorati F, Reichart D, Perrotti A, Ruggieri VG, et al. Prediction of severe bleeding after coronary surgery: the WILL-BLEED Risk Score. Thromb Haemost. (2017) 117:445–56. doi: 10.1160/TH16-09-0721

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Christodoulou E, Ma J, Collins GS, Steyerberg EW, Verbakel JY, Van Calster B. A systematic review shows no performance benefit of machine learning over logistic regression for clinical prediction models. J Clin Epidemiol. (2019) 110:12–22. doi: 10.1016/j.jclinepi.2019.02.004

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Quer G, Arnaout R, Henne M, Arnaout R. Machine learning and the future of cardiovascular care: JACC state-of-the-art review. J Am Coll Cardiol. (2021) 77:300–13. doi: 10.1016/j.jacc.2020.11.030

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Rajkomar A, Dean J, Kohane I. Machine learning in medicine. N Engl J Med. (2019) 380:1347–58. doi: 10.1056/NEJMra1814259

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Qu J, Zhang D, Zhang H, Rao C, Chen S, Zhao Y, et al. Preoperative clopidogrel and outcomes in patients with acute coronary syndrome undergoing coronary artery bypass surgery. J Thorac Cardiovasc Surg. (2020) 163:1044.e–52.e. doi: 10.1016/j.jtcvs.2020.03.118

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Wang C, Gao Y, Tian Y, Wang Y, Zhao W, Sessler DI, et al. Prediction of acute kidney injury after cardiac surgery from preoperative N-terminal pro-B-type natriuretic peptide. Br J Anaesth. (2021) 127:862–70. doi: 10.1016/j.bja.2021.08.015

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Dyke C, Aronson S, Dietrich W, Hofmann A, Karkouti K, Levi M, et al. Universal definition of perioperative bleeding in adult cardiac surgery. J Thorac Cardiovasc Surg. (2014) 147:1458.e–63.e. doi: 10.1016/j.jtcvs.2013.10.070

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Bartoszko J, Wijeysundera DN, Karkouti K, Karkouti K, Callum J, Rao V, et al. Comparison of two major perioperative bleeding scores for cardiac surgery trials: universal definition of perioperative bleeding in cardiac surgery and European coronary artery bypass grafting bleeding severity grade. Anesthesiology. (2018) 129:1092–100. doi: 10.1097/ALN.0000000000002179

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Steyerberg EW, Vergouwe Y. Towards better clinical prediction models: seven steps for development and an ABCD for validation. Eur Heart J. (2014) 35:1925–31. doi: 10.1093/eurheartj/ehu207

PubMed Abstract | CrossRef Full Text | Google Scholar

17. DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. (1988) 44:837–45. doi: 10.2307/2531595

CrossRef Full Text | Google Scholar

18. Nattino G, Finazzi S, Bertolini G. A new calibration test and a reappraisal of the calibration belt for the assessment of prediction models based on dichotomous outcomes. Stat Med. (2014) 33:2390–407. doi: 10.1002/sim.6100

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Fitzgerald M, Saville BR, Lewis RJ. Decision curve analysis. JAMA. (2015) 313:409–10. doi: 10.1001/jama.2015.37

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Making. (2006) 26:565–74. doi: 10.1177/0272989X06295361

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Krzych LJ, Krejca M, Bis J, Wasiak M, Vialek K, Szurlej D, et al. Perioperative blood pressure variability in patients undergoing coronary artery bypass grafting: its magnitude and determinants. Blood Press Monit. (2011) 16:129–33. doi: 10.1097/MBP.0b013e328346d69f

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Filipovic M, Jeger R, Probst C, Girard T, Pfisterer M, Gürke L, et al. Heart rate variability and cardiac troponin I are incremental and independent predictors of one-year all-cause mortality after major noncardiac surgery in patients at risk of coronary artery disease. J Am Coll Cardiol. (2003) 42:1767–76. doi: 10.1016/j.jacc.2003.05.008

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Levin MA, Fischer GW, Lin HM, McCormick PJ, Krol M, Reich DL, et al. Intraoperative arterial blood pressure lability is associated with improved 30 day survival. Br J Anaesth. (2015) 115:716–26. doi: 10.1093/bja/aev293

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Mathews R, Peterson ED, Chen AY, Wang TY, Chin CT, Fonarow GC, et al. In-hospital major bleeding during ST-elevation and non-ST-elevation myocardial infarction care: derivation and validation of a model from the ACTION Registry^®-GWTG™. Am J Cardiol. (2011) 107:1136–43. doi: 10.1016/j.amjcard.2010.12.009

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Kuhn M, Johnson K. Applied Predictive Modeling. New York, NY: Springer (2013).

Google Scholar

26. Benedetto U, Dimagli A, Sinha S, Cocomello L, Gibbison B, Caputo M, et al. Machine learning improves mortality risk prediction after cardiac surgery: systematic review and meta-analysis. J Thorac Cardiovasc Surg. (2020) 163:2075–2087e.9. doi: 10.1016/j.jtcvs.2020.07.105

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Kilic A, Goyal A, Miller JK, Gjekmarkaj E, Tam WL, Gleason TG, et al. predictive utility of a machine learning algorithm in estimating mortality risk in cardiac surgery. Ann Thorac Surg. (2020) 109:1811–9. doi: 10.1016/j.athoracsur.2019.09.049

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Zea-Vera R, Ryan CT, Havelka J, Corr SJ, Nguyen TC, Chatterjee S, et al. Machine learning to predict outcomes and cost by phase of care after coronary artery bypass grafting. Ann Thorac Surg. (2021) 25:S000304975.

Google Scholar

29. Benedetto U, Sinha S, Lyon M, Dimagli A, Gaunt TR, Angelini G, et al. Can machine learning improve mortality prediction following cardiac surgery? Eur J Cardiothorac Surg. (2020) 58:1130–6. doi: 10.1093/ejcts/ezaa229

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Vickers AJ, Holland F. Decision curve analysis to evaluate the clinical benefit of prediction models. Spine J. (2021) 21:1643–8. doi: 10.1016/j.spinee.2021.02.024

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Pagano D, Milojevic M, Meesters MI, Venedetto U, Bolliger D, von Heymann C, et al. 2017 EACTS/EACTA guidelines on patient blood management for adult cardiac surgery. Eur J Cardiothorac Surg. (2018) 53:79–111. doi: 10.1093/ejcts/ezx325

PubMed Abstract | CrossRef Full Text | Google Scholar