Prediction of low cardiac output syndrome in patients following cardiac surgery using machine learning

Background: This study aimed to develop machine learning models to predict Low Cardiac Output Syndrome (LCOS) in patients following cardiac surgery using machine learning algorithms.

Methods: The clinical data of cardiac surgery patients in Nanjing First Hospital between June 2019 and November 2020 were retrospectively extracted from the electronic medical records. Six conventional machine learning algorithms, including logistic regression, support vector machine, decision tree, random forest, extreme gradient boosting and light gradient boosting machine, were employed to construct the LCOS predictive models with all predictive features (full models) and selected predictive features (reduced models). The discrimination of these models was evaluated by the area under the receiver operating characteristic curve (AUC) and the calibration of the models was assessed by the calibration curve. Shapley Additive explanation (SHAP) and Local Interpretable Model-Agnostic Explanations (LIME) were used to interpret the predictive models.

Results: Data from 1,585 patients [982 (62.0%) were male, aged 18 to 88, 212 (13.4%) with LCOS] were employed to train and validate the LCOS models. Among the full models, the RF model (AUC: 0.909, 95% CI: 0.875–0.943; Sensitivity: 0.849, 95% CI: 0.724–0.933; Specificity: 0.835, 95% CI: 0.796–0.869) and the XGB model (AUC: 0.897, 95% CI: 0.859–0.935; Sensitivity: 0.830, 95% CI: 0.702–0.919; Specificity: 0.809, 95% CI: 0.768–0.845) exhibited well predictive power for LCOS. Eleven predictive features including left ventricular ejection fraction (LVEF), first post-operative blood lactate (Lac), left ventricular diastolic diameter (LVDd), cumulative time of mean artery blood pressure (MABP) lower than 65 mmHg (MABP < 65 time), hypertension history, platelets level (PLT), age, blood creatinine (Cr), total area under curve above threshold central venous pressure (CVP) 12 mmHg and 16 mmHg, and blood loss during operation were used to build the reduced models. Among the reduced models, RF model (AUC: 0.895, 95% CI: 0.857–0.933; Sensitivity: 0.830, 95% CI: 0.702–0.919; Specificity: 0.806, 95% CI: 0.765–0.843) revealed the best performance. SHAP and LIME plot showed that LVEF, Lac, LVDd and MABP < 65 time significantly contributed to the prediction model.

Conclusion: In this study, we successfully developed several machine learning models to predict LCOS after surgery, which may avail to risk stratification, early detection and management of LCOS after cardiac surgery.

Introduction

Low Cardiac Output Syndrome (LCOS), a clinical manifestation of insufficient cardiac output and peripheral tissue perfusion, was first proposed by Rao et al. (1). Previous studies have shown that all-cause mortality in LCOS ranges from 14.8 to 62.5% in the short term (1 month post onset) and 21.4–36.6% in the long term (2 months to 1 year post onset) (2). LCOS following cardiac surgery not only leads to tissue malperfusion, but also multiple organ dysfunction of brain, lung, liver, kidney, and gastrointestinal tract, thereby increasing health care resource utilization and associated costs (3). More importantly, LCOS may be a state of reversible cardiac output (CO) reduction after cardiac surgery and early recognition and appropriate treatment of LCOS may avoid its progression to refractory cardiogenic shock and improve clinical outcomes, with early detection being of great significance (4–9).

The most common definition of LCOS (1) includes a decrease in the cardiac output index (CI) to < 2.2 L/min/m² and a systolic blood pressure of < 90 mmHg, in conjunction with signs of tissue malperfusion (cold periphery, clammy skin, confusion, oliguria, elevated lactate level) in the absence of hypovolemia. Accordingly, it is of necessity to monitor CO by the pulmonary artery catheter (PAC) or pulse indicator continuous cardiac output (PICCO). All these current monitoring technologies, however, are too costly to be routinely applied in the setting of patients undergoing cardiac surgery, which consequently increases the difficulty of early recognition and prevention. Studies showed that significant independent risk factors for LCOS include age, preoperative left ventricular ejection fraction (LVEF), emergency surgery, temperature during cardiopulmonary bypass (CPB), application of cardioprotective drugs and echocardiographic parameters (10–12). Nevertheless, there were few studies on prediction models for LCOS. Therefore, this study was aimed to apply machine learning to develop models for the precise prediction of LCOS following cardiac surgery using preoperative variables and intraoperative time-series data, with the potential to avail early recognition and management of LCOS.

Materials and methods

Data sources and study population

This retrospective study was conducted on 1,681 consecutive patients admitted and received cardiac surgery at Nanjing First Hospital from June 2019 to November 2020. Patients who received cardiac surgery during the study period were recruited as the study objects, including but not limited to coronary artery bypass, heart valve surgery, aortic dissection (AD) repair surgery, etc. Exclusion criteria: (1) Patients under 18 years of age. (2) Patients who died or were discharged during or within 48 h after the operation. (3) Patients with incomplete clinical data, such as pre-operation echocardiographic measurements or intraoperative hemodynamic data. Data were collected from electronic medical records (EMR) database, and approval was gained from the Ethics Committee of Nanjing First Hospital (KY20220518-KS-01).

Definition of low cardiac output syndrome

According to previous reports (10, 13), the criteria for LCOS in our study included: (1) Patients with a cardiac index (CI) reduced to < 2.2 L/min/m²; (2) Patients with systolic blood pressure < 90 mmHg, in conjunction with signs of tissue hypoperfusion [oliguria (urine output < 1 ml/kg.h), elevated lactate level > 3.0 mmol/L]; (3) Patients requiring mechanical circulatory support or inotropic agents (dopamine or dobutamine at least 4 μg/kg.min for a minimum of 12 h and/or epinephrine at least 0.2 μg/kg.min and/or milrinone at least 0.02 μg/kg.min and/or levosimendan at least 0.05 μg/kg.min) to maintain hemodynamics after optimizing preload. Patients who received vasoconstricting medication to increase peripheral vascular resistance in the presence of normal cardiac output were not considered to have LCOS.

Data collection and preprocessing of data

Clinical variables extracted from electronic medical records (EMR) database included demographics: age, sex, height, weight; comorbidities: hypertension, diabetes, myocardial infarction, hyperlipidemia, cerebral vascular disease, atrial fibrillation, chronic obstructive pulmonary disease (COPD), congestive heart failure, renal disease, liver disease; preoperative echocardiographic parameters: left ventricular diastolic diameter (LVDd), left atrial diameter (LAD), interventricular septum thickness in diastole (IVSd), left ventricular posterior wall thickness (LVPWT), pulmonary artery systolic pressure (PASP), left ventricular ejection fraction (LVEF); preoperative laboratory results: white blood cell count (WBC), neutrophil ratio (NEU), lymphocyte ratio (LYM), platelets level (PLT), hemoglobin (Hb), blood creatinine (Cr), blood urea nitrogen (BUN), aspartate aminotransferase (AST), alanine aminotransferase (ALT), total protein (TP), total bilirubin (TB), low density lipoprotein (LDL), creatine kinase-MB (CKMB), triiodothyronine (T3), thyroxine (T4), thyroid stimulating hormone (TSH), brain natriuretic peptide (BNP); operation information: operation time, cardiopulmonary bypass (CPB) time, aortic occlusion (AO) time, Emergency surgery, urine output (UO) during operation, blood loss during operation, operation type; intraoperative hemodynamics: mean arterial blood pressure (MABP), central venous pressure (CVP); postoperative hemodynamics: cardiac output (via pulmonary artery catheter for some patients), systolic artery blood pressure (SABP), CVP, inotropes (dopamine, dobutamine, epinephrine, milrinone, and levosimendan) usage, urine output and first post-operative blood lactate levels (within 30 min post operation), prognosis variables: mechanical ventilation (MV) time, ICU stay time and hospital stay time. Renal disease was defined as preoperative glomerular filtration rate < 30 ml/min/1.73 m² (body surface area) (14). Hyperlipidemia was defined as total cholesterol > 200 mg/dl and/or triglyceridemic value > 150 mg/dl. Other comorbidities were identified from diagnosis before operation using the International Classification of Disease, Tenth edition (ICD-10). ICD-10 codes used for the identification of comorbidities are outlined in Supporting Information (Supplementary Table 1).

During operation, MABP and CVP were continuously monitored using invasive peripheral artery, central vein or pulmonary artery catheter and saved as time-series data. Artifactual data were removed according to previously published criteria (15). Thresholds for MABP (< 65, < 60, < 55, < 50 mmHg) and CVP (> 12, > 16, > 20 mmHg) were used to assess the site of hypotension and central venous congestion occurred during operation. To comprehensively assess the time-series data, cumulative time under or above thresholds, total area under curve under or above threshold (AUT) and time weighted average (TWA) of MABP and CVP for corresponding threshold were calculated based on a previous study (16).

Model construction and evaluating

The entire dataset was randomly stratified into the training and test sets (7:3), meaning that the ratio of patients with LCOS to those without LCOS was maintained consistent in both subsets. The training set was applied to train the model with 10-fold cross-validation and test set was used later to assess the models’ performance. All variables with a missing rate > = 10% were excluded from the analysis (Supplementary Figure 1). Variables with a missing rate < 10% were imputed by the k-nearest neighbors (KNN) imputation procedure (17). The low incidence of LCOS and the large number of variables we included in this study made it typical unbalanced high dimension data, Synthetic Minority Oversampling Technique (SMOTE) was applied to overcome this imbalance.

Six conventional machine learning algorithms were employed to construct the LCOS prediction models will all variables (full models), including logistic regression (LR), support vector machine (SVM), decision tree (DT), random forest (RF), extreme gradient boosting (XGB), and light gradient boosting machine (LGB).

Boruta and the least absolute shrinkage and selection operator (LASSO) were used to select the optimal subset of variables. All variables confirmed as important by the Boruta algorithm were entered the LASSO regressing. Finally, variables identified by LASSO regression were included for constructing reduced models using the same six machine learning algorithms.

Statistical analyses

Baseline characteristics of patients in the training and test sets were compared. Measurements conforming to a normal distribution were described as mean ± standard deviation. Student’s t-test was employed for comparisons. Measurement data that did not conform to a normal distribution were denoted as median [lower quartile-upper quartile]. Wilcoxon rank-sum tests were performed to draw comparisons. The enumeration data were represented as frequency and percentage and compared by performing Pearson χ² test. Fisher’s exact test was performed under the expected frequencies of one or more cells less than 5. The difference was considered with statistical significance at P < 0.05.

The discriminations of models were evaluated by the area under curve (AUC) of the receiver operating characteristic (ROC), accuracy, sensitivity, specificity and calibration of the models were assessed by the calibration curve and Brier score. Shapley Additive explanation (SHAP) and Local Interpretable Model-Agnostic Explanations (LIME) were used to provide consistent and locally accurate values for each variable within the prediction models. All analyses were conducted in R (version 3.6.3) and Python (version 3.7).

Results

Overall, the eligibility of 1,681 patients who underwent cardiac surgery and were admitted to the Cardiovascular ICU of Nanjing First Hospital, Nanjing Medical University, from June 2019 to November 2020 was assessed. The excluded cases were as follows: 35 cases were younger than 18 years old, 8 patients were dead or discharged within 48 h after surgery and 53 patients had uncompleted data. Finally, 1585 patients [982 (62.0%) male, 18 to 88 years old] were enrolled for analyses. Among them, 386 (24.4%) patients received PAC insertion during the surgery, and the proportion of PAC use varied by surgery types (Supplementary Table 2). Among patients with PAC, 61 (15.8%) were diagnosed with LCOS by CI criterion, and among the other 1,199 patients without PAC, 151 (12.6%) were diagnosed with LCOS by other criteria. Overall, 212 (13.4%) patients developed LCOS postoperatively. Compared to patients without LCOS, patients with LCOS had prolonged MV time (20.25 [13.08,40.38] vs. 9.50 [7.00,15.33] hours, P < 0.001), longer ICU stay time (3.0 [2.0,6.0] vs. 1.0 [1.0,2.0] days, P < 0.001) and hospital stay time (21.0 [16.0,27.0] vs. 17.0 [14.0,21.0] days, P < 0.001). There was no significant difference in morbidity between patients with LCOS diagnosed by CI criterion and other criteria (15.6 vs. 12.5%, P = 0.127), and patients with LCOS diagnosed by different criteria had similar prognoses (Supplementary Figure 2), indicating consistency between the different criteria.

We randomized 70% of these 1,585 patients into the training set and the remaining 30% into the test set. The clinical variables of patients in training and test set are listed in Table 1. There was no significant difference between patients in training and test sets for these variables.

TABLE 1

Table 1. Patient characteristics and clinical variables.

The full models were conducted with all variables, using the six algorithms including LR, DT, SVM, RF, XGB, and LGB for LCOS predicting, and the AUC, accuracy, sensitivity, and specificity of each full model on test set were presented in Figure 1 and Table 2. Among the full models, the RF model (AUC: 0.909, 95% CI: 0.875–0.943; Sensitivity: 0.849, 95% CI: 0.724–0.933; Specificity: 0.835, 95% CI: 0.796–0.869) and the XGB model (AUC: 0.897, 95% CI: 0.859–0.935; Sensitivity: 0.830, 95% CI: 0.702–0.919; Specificity: 0.809, 95% CI: 0.768–0.845) showed well predictive power for LCOS. The main parameters of the full RF model were set as follows: bootstrap = True, criterion = “gini,” n_estimators = 500, max_depth = None, min_samples_leaf = 1, min_sample_split = 2. The main parameters of the full XBG model were set as follows: n_estimators = 200, learning_rate = 0.1, max_depth = 9, gamma = 0. The calibration plot and Brier score indicated all the full models have well calibration (Figure 2).

FIGURE 1

Figure 1. Comparison of AUCs among different machine learning models. LR, logistic regression; DT, decision tree; RF, random forest classifier; LGB, light gradient boosting machine; XGB, extreme gradient boosting machine; SVM, support vector machine.

TABLE 2

Table 2. The performance of each model.

FIGURE 2

Figure 2. The calibration curves and the Brier score of different machine learning models. LR, logistic regression; DT, decision tree; RF, random forest classifier; LGB, light gradient boosting machine; XGB, extreme gradient boosting machine; SVM, support vector machine.

Feature selection was performed by the following two steps. First, Boruta algorithm was employed and 35 features were confirmed important to the prediction of LCOS (Supplementary Figure 3). Then, Lasso regression was applied to select the best subset features from the 35 confirmed important features (Supplementary Figure 4). Eleven variables were finally selected by Boruta and LASSO features selection procedure, including LVEF, lactate (Lac), LVDd, cumulative time of mean artery blood pressure (MABP) lower than 65 mmHg (MABP < 65 time), hypertension history, PLT, age, blood Cr, AST, total area under curve above CVP 12 mmHg (CVP_AUT_12), total area under curve above threshold CVP 16 mmHg (CVP_AUT_16) and blood loss during operation. Six reduced models with these eleven variables and the same six algorithms were then developed. Among the reduced models, among which, RF model (AUC:0.895, 95% CI: 0.857–0.933; Sensitivity:0.830, 95% CI: 0.702–0.919; Specificity: 0.806, 95% CI: 0.765–0.843) revealed the best performance. The main parameters of the reduced RF model were set as follows: bootstrap = True, criterion = “gini,” n_estimators = 700, max_depth = None, min_samples_leaf = 1, min_sample_split = 2. The AUC, accuracy, sensitivity and specificity of the full and reduced models are presented in Table 2.

The SHAP summary plot (Figure 3) and dependence plot (Figure 4) represented the contributions of these eleven variables to the prediction of the RF model, with SHAP values above zero indicating an increased risk of developing LCOS and SHAP values below zero indicating a decreased risk of LCOS. For example, SHAP values for high LVEF (red) were usually less than zero, indicating a decreased risk of LCOS in patients with higher LVEF. In addition, Figure 3A displays the ranking of the features based on the average absolute SHAP value. Among the eleven variables, LVEF, Lac, LVDd and MABP < 65 time were the four variables with the greatest influence on prediction power. Lower LVEF, higher Lac, larger LVDd and longer MABP < 65 time indicated an increased possibility of the onset of LCOS. We randomly selected two patients with LCOS (Figure 5A) and without LCOS (Figure 5B) and used LIME algorithm to interpret how they were predicted to be have a 68% possibility of LCOS and 92% possibility without LCOS. The first patient (Figure 5A) was predicted to be with possibility of prospective LCOS due to low LVEF (38%), high Lac (2.9 mmol/L), large LVDd (84 mm), long MABP < 65 time (185 min) and advanced age (76 years). The second patient (Figure 5B) was predicted to be without prospective LCOS due to relatively normal variables: Lac (1.0 mmol/L), LVDd (52 mm), hypertension, CVP_AUT_16 (0 min), Blood loss (700 ml), CVP_AUT_12 (15 min), age (64 years), MAPB < 65 time (125 min), PLT (256 * 10^9/L).

FIGURE 3

Figure 3. SHAP summary plot of the reduced RF model. The plot showed the importance of each variable (A) and the specific distribution between variables and Shapely value (B) using SHAP algorithm.

FIGURE 4

Figure 4. SHAP dependence plot of the reduced RF model. ECHO, echocardiography; LVDd, left ventricular diastolic diameter; LVPWT, left ventricular posterior wall thickness; LVEF, left ventricular ejection fraction; WBC, white blood cell count; NEU, neutrophil properties; Cr, blood creatinine; BUN, blood urea nitrogen; AST, aspartate aminotransferase; Lac, blood lactate; CPB, cardiopulmonary bypass; AO, aortic occlusion; CABG, coronary artery bypass graft; MABP < 65 time, cumulative time of mean artery blood pressure lower than 65 mmHg; MABP < 60 time, cumulative time of mean artery blood pressure lower than 60 mmHg; MABP < 55 time, cumulative time of mean artery blood pressure lower than 55 mmHg; MABP < 50 time, cumulative time of mean artery blood pressure lower than 50 mmHg; MABP_AUT_65, total area under curve below threshold mean artery blood pressure 65 mmHg; MABP_AUT_60, total area under curve below threshold mean artery blood pressure 60 mmHg; MABP_AUT_55, total area under curve below threshold mean artery blood pressure 55 mmHg; MABP_AUT_50, total area under curve below threshold mean artery blood pressure 50 mmHg; MABP_TWA_65, time weighted average mean artery blood pressure below threshold 65 mmHg; MABP_TWA_60, time weighted average mean artery blood pressure below threshold 60 mmHg; MABP_TWA_55, time weighted average mean artery blood pressure below threshold 55 mmHg; MABP_TWA_50, time weighted average mean artery blood pressure below threshold 50 mmHg; CVP > 12 time, cumulative time of central venous pressure upper than 12 mmHg; CVP > 16 time, cumulative time of central venous pressure upper than 16 mmHg; CVP > 20 time, cumulative time of central venous pressure upper than 20 mmHg; CVP_AUT_12, total area under curve above threshold central venous pressure 12 mmHg; CVP_AUT_16, total area under curve above threshold central venous pressure 16 mmHg; CVP_AUT_20, total area under curve above threshold central venous pressure 20 mmHg; CVP_TWA_12, time weighted average central venous pressure above threshold 12 mmHg; CVP_TWA_16, time weighted average central venous pressure above threshold 16 mmHg; CVP_TWA_20, time weighted average central venous pressure above threshold 20 mmHg; LCOS, low cardiac output syndrome.

FIGURE 5

Figure 5. LIME plot for individual case explanation on two random patients for the test set of the reduced RF model. LIME plot included one patient with LCOS (A) and one patient without LCOS (B), explained by LIME algorithm. ECHO, echocardiography; LVDd, left ventricular diastolic diameter; LVPWT, left ventricular posterior wall thickness; LVEF, left ventricular ejection fraction; WBC, white blood cell count; NEU, neutrophil properties; Cr, blood creatinine; BUN, blood urea nitrogen; AST, aspartate aminotransferase; Lac, blood lactate; CPB, cardiopulmonary bypass; AO, aortic occlusion; CABG, coronary artery bypass graft; MABP < 65 time, cumulative time of mean artery blood pressure lower than 65 mmHg; MABP < 60 time, cumulative time of mean artery blood pressure lower than 60 mmHg; MABP < 55 time, cumulative time of mean artery blood pressure lower than 55 mmHg; MABP < 50 time, cumulative time of mean artery blood pressure lower than 50 mmHg; MABP_AUT_65, total area under curve below threshold mean artery blood pressure 65 mmHg; MABP_AUT_60, total area under curve below threshold mean artery blood pressure 60 mmHg; MABP_AUT_55, total area under curve below threshold mean artery blood pressure 55 mmHg; MABP_AUT_50, total area under curve below threshold mean artery blood pressure 50 mmHg; MABP_TWA_65, time weighted average mean artery blood pressure below threshold 65 mmHg; MABP_TWA_60, time weighted average mean artery blood pressure below threshold 60 mmHg; MABP_TWA_55, time weighted average mean artery blood pressure below threshold 55 mmHg; MABP_TWA_50, time weighted average mean artery blood pressure below threshold 50 mmHg; CVP > 12 time, cumulative time of central venous pressure upper than 12 mmHg; CVP > 16 time, cumulative time of central venous pressure upper than 16 mmHg; CVP > 20 time, cumulative time of central venous pressure upper than 20 mmHg; CVP_AUT_12, total area under curve above threshold central venous pressure 12 mmHg; CVP_AUT_16, total area under curve above threshold central venous pressure 16 mmHg; CVP_AUT_20, total area under curve above threshold central venous pressure 20 mmHg; CVP_TWA_12, time weighted average central venous pressure above threshold 12 mmHg; CVP_TWA_16, time weighted average central venous pressure above threshold 16 mmHg; CVP_TWA_20, time weighted average central venous pressure above threshold 20 mmHg; LCOS, low cardiac output syndrome.

Discussion

Big data and machine learning are enabling the shift from conventional to customized treatment, which could soon result in the birth of a new health system (18, 19). To the best of our knowledge, no machine learning prediction model has been established to predict the occurrence of LCOS following cardiac surgery. In the present study, in the cooperation of clinicians and information technology engineers, we successfully developed several machine learning models to predict LCOS following cardiac surgery. Six conventional machine learning algorithms were employed to construct the LCOS prediction models, including LR, SVM, DT, RF, XGB, and LGB, indicating that RF and XGB models exhibited the best performance. RF is a homologous ensemble algorithm that constructs a great number of decision trees during training, which helps to build robust prediction models with able to deal with non-linear data. XGB is a distributed algorithm with fast operation speed and high fault tolerance, which could accurately predicts the outcome of multiple diseases in ICU (20–22). Commendably, our study demonstrated that the performance of machine learning models was significantly superior to the traditional logistic regression models in the prediction of LCOS following cardiac surgery.

We adopted a dual definition of LCOS, similar to the prior studies (10, 13), with the CI criterion requiring perioperative PAC monitoring. However, even in the field of cardiothoracic surgery, the usage of the PAC has declined over the years. The most accurate way to evaluate the pulmonary artery and cardiac output in patients with pulmonary hypertension and heart failure, however, is through the use of PAC. The large proportion of PAC usage in various procedures, such as heart transplant, adult congenital surgery, and challenging combination CABG + valve surgery, can be attributed to its potential benefit in patients with a high risk of RV failure (8, 23) (Supplementary Table 2). PAC could continually provide important hemodynamic measurements like pulmonary circulation resistance, right heart afterload, cardiac output, etc. Those measurements are imperative in perioperative management of critically ill patients after those types of surgery. However, PAC use was reported to be associated with a poorer outcome in patients receiving cardiac surgical. As an invasive hemodynamic monitoring method, the difficulty of placement and consequent side effects may contribute to iatrogenic adverse outcomes for patients (24). The similar prognosis outcome between LCOS patients diagnosed by CI criterion and other criteria indicated consistency across different criteria. Importantly, our prediction models could provide a non-invasive, precious, interpretable way to predict LCOS, perhaps reducing the need for intrusive monitoring techniques like PAC.

Low Cardiac Output Syndrome could be corrected by timely and effective intervention and a variety of therapeutic strategies can be applied to the treatment of LCOS, when it is early recognized, including optimization of ventricular preload and afterload; inotropic agents; positive pressure ventilation; heart rhythm and rate control; metabolic and hormonal disorders correction; and in extreme circumstances mechanical circulatory support (8, 9, 25, 26). Most features we included in the full and reduced models were preoperative clinical and intraoperative hemodynamic variables. Our prediction model could be integrated into the EMR system for use in everyday practice, and the HER database could automatically provide the model with the data it needs. In the very early phase after surgery, LCOS models could provide LCOS risk prediction and shed a light on further strategies for postoperative management and initiation of individualized therapy.

The present study showed that LVEF, Lac, LVDd, MABP < 65 time, hypertension, PLT, age, Cr, CVP_AUT_16, CVP_AUT_12, and blood loss significantly contributed to the prediction. The reduced RF model using these features also showed little discrimination loss in the prediction of LCOS (AUC:0.895 vs. 0.909) but it could significantly increase the efficiency and convenience, which may contribute to risk stratification and short-term decision making for LCOS.

Traditionally, machine learning models have been less interpretable when compared to traditional regression models. This black-box behavior has hindered their application in clinical settings. To enhance the interpretability of machine learning, we utilized SHAP and LIME interpreter techniques to visualize how features affect the prediction of LCOS, both globally and individually accordingly. SHAP summary plot revealed that LVEF, Lac, LVDd and MABP < 65 time were the most significant predictors of LCOS, with lower LVEF, higher Lac, larger LVDd and longer MABP < 65 time indicating increased possibility of the prospective onset of LCOS. LVEF is the most widely used estimate of left ventricular systolic function and a decreased LVEF is an independent risk factor for LCOS (27, 28). Serum lac is a well-recognized biomarker of tissue perfusion, and elevated lac can serve as a sensitive indicator of LCOS. Ventricular dilatation is a common compensatory response to decreased myocardial contractility (29), which can explain the association between enlarged LVDd and the risk of LCOS. MABP < 65 time also serves as a surrogate for a hypotension state subsequent to reduced cardiac output. We tried multiple thresholds of MABP (< 65, < 60, < 55, < 50 mmHg) and MABP < 65 mmHg showed a better predictive value than other thresholds, suggesting 65 mmHg was a good MABP threshold regarding maintenance of tissue perfusion (30). In our study, patients with a history of hypertension were less likely to develop LCOS after cardiac surgery, which is consistent with a previous study (3). Hypertension is usually associated with myocardial hypertrophy and is accompanied by enhanced myocardial contractility. Notably, hypertension patients were also reported to have higher mortality after the onset of LCOS, because myocardial hypertrophy would exacerbate the deficiency of oxygen supply subsequent to LCOS (31).

Our study had several advantages when compared to previous studies. Firstly, our study included comprehensive variables including demographics, commodities, echocardiographic and laboratory measurements and operation related information including intraoperative hemodynamic data, which could reflect the patient’s profile in multiple dimensions. Secondly, we incorporated various hemodynamic time-series features that were considered difficult to incorporate in prediction models (32–34). In our previous study, we demonstrated an association between hemodynamic time-series data and postoperative organ dysfunction (35). It is well known that intraoperative hypotension and venous congestion may be a reflection of LCOS. We examined cumulative time, total area under curve and time weighted average under or above pre-specified thresholds other than using static measures as in other studies (36). In this way, we could assess both the duration and severity of hypotension and venous congestion, so as to better contribute to the prediction of LCOS.

Thirdly, we analyzed the predictive value of all features and selected features for LCOS prediction. Some studies made feature selection only based on linear regression or stepwise logistic regression, which may exclude features that were not statistically significant but have causal effects on the output variable due to non-linear relationships or interactions between the variables and outcomes (37). As a wrapper built around the random forest classification algorithm, Boruta performed classification by voting on multiple unbiased weak decision trees (38), which could deal with non-linear and complex relationships between the features and the outcome. Thus, our approach reduced the possibility of missing important or previously unreported features.

This study was subject to some limitations. First, we did not compare the performance of our models with previous LCOS risk scores because some of the variables required in the risk scores were not available. Second, the models have not been verified in the external validation queue. Third, this is a single center retrospective study. Further multi-center studies with external validation are needed to further verify our findings and prospective studies could be of more importance in assessing the performance of our predictive models.

Conclusion

In the present study, we successfully developed several machine learning models to predict LCOS following cardiac surgery, which may avail to risk stratification, early detection and management of LCOS following cardiac surgery.

Data availability statement

The original contributions presented in this study are included in the article/Supplementary material, further inquiries can be directed to the corresponding authors.

Ethics statement

The study protocol was conducted in accordance with the Declaration of Helsinki and was approved by the Ethics Committee of Nanjing First Hospital, Nanjing Medical University (KY20220518-KS-01). Informed consent was not obtained due to the observational and anonymous nature of data collection.

Author contributions

LH, HX, CG, DG, and CZ conceived the conception of the study. LH, HX, CG, HT, and XSh acquired the data. XSo, DG, and CZ participated in data analyses. CG and DG constructed the predictive model. LH, HX, CG, and HT prepared the first draft of the manuscript. CZ and DG led the project and supervised the study. All authors were involved in writing or editing the manuscript, read, and approved the final version of the manuscript.

Funding

This study was funded by the Nanjing Key Medical Science and Technology Development Foundation (ZKX19021). The study sponsors had no involvement in the study design, the collection, analysis, and interpretation of data, the writing of the report, or the decision to submit the manuscripts for publication.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fmed.2022.973147/full#supplementary-material

Supplementary Figure 1. Missing value in the study.

Supplementary Figure 2. Prognosis variables in patients with LCOS diagnosed by CI criterion and other criteria and patients without LCOS, including mechanical ventilation time (A), ICU stay time (B) and hospital stay time (C).

Supplementary Figure 3. Feature importance determined through Boruta algorithm.

Supplementary Figure 4. Subset feature selection through Lasso regression.

Abbreviations

LCOS, low cardiac output syndrome; EMR, electronic medical records; LR, logistic regression; SVM, support vector machine; DT, decision tree; RF, random forest; XGB, extreme gradient boosting; LGB, light gradient boosting machine; AUC, area under the receiver operating characteristic curve; SHAP, shapley additive explanation; LIME, local interpretable model-agnostic explanations; LVEF, left ventricular ejection fraction; CO, cardiac output; CI, cardiac index; PAC, pulmonary artery catheter; PICCO, pulse indicator continuous cardiac output; CPB, cardiopulmonary bypass; CABG, coronary artery bypass graft; AD, aortic dissection; EMR, electronic medical record; LVDd, left ventricular diastolic diameter; LAD, left atrial diameter; IVSd, interventricular septum thickness in diastole; LVPWT, left ventricular posterior wall thickness; PASP, pulmonary artery systolic pressure; WBC, white blood cell count; NEU, neutrophil ratio; LYM, lymphocyte ratio; PLT, platelets level; Hb, Hemoglobin; Cr, blood creatinine; BUN, blood urea nitrogen; AST, aspartate aminotransferase; ALT, alanine aminotransferase; TP, total protein; TB, total bilirubin; LDL, low density lipoprotein; CKMB, creatine kinase-MB; T3, triiodothyronine; T4, thyroxine; TSH, thyroid stimulating hormone; BNP, brain natriuretic peptide; Lac, blood lactate; AO, aortic occlusion; UO, urine output; MABP, mean arterial blood pressure; CVP, central venous pressure; MV, mechanical ventilation; AUC, area under curve; ROC, receiver operating characteristic; SABP, systolic artery blood pressure; KNN, k-nearest neighbors; LASSO, least absolute shrinkage and selection operator.

References

1. Rao V, Ivanov J, Weisel RD, Ikonomidis JS, Christakis GT, David TE. Predictors of low cardiac output syndrome after coronary artery bypass. J Thorac Cardiovasc Surg. (1996) 112:38–51. doi: 10.1016/s0022-5223(96)70176-9

CrossRef Full Text | Google Scholar

2. Uhlig K, Efremov L, Tongers J, Frantz S, Mikolajczyk R, Sedding D, et al. Inotropic agents and vasodilator strategies for the treatment of cardiogenic shock or low cardiac output syndrome. Cochrane Database Syst Rev. (2020) 11:CD009669. doi: 10.1002/14651858.CD009669.pub4

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Duncan AE, Kartashov A, Robinson SB, Randall D, Zhang K, Luber J, et al. Risk factors, resource use, and cost of postoperative low cardiac output syndrome. J Thorac Cardiovasc Surg. (2022) 163:1890–8.e10. doi: 10.1016/j.jtcvs.2020.06.125

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Zangrillo A, Pappalardo F, Dossi R, Di Prima AL, Sassone ME, Greco T, et al. Preoperative intra-aortic balloon pump to reduce mortality in coronary artery bypass graft: a meta-analysis of randomized controlled trials. Crit Care. (2015) 19:10. doi: 10.1186/s13054-014-0728-1

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Vallabhajosyula S, Arora S, Sakhuja A, Lahewala S, Kumar V, Shantha GPS, et al. Trends, predictors, and outcomes of temporary mechanical circulatory support for postcardiac surgery cardiogenic shock. Am J Cardiol. (2019) 123:489–97. doi: 10.1016/j.amjcard.2018.10.029

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Weber C, Esser M, Eghbalzadeh K, Sabashnikov A, Djordjevic I, Maier J, et al. Levosimendan reduces mortality and low cardiac output syndrome in cardiac surgery. Thorac Cardiovasc Surg. (2020) 68:401–9. doi: 10.1055/s-0039-3400496

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Sunny, Yunus M, Karim HMR, Saikia MK, Bhattacharyya P, Dey S. Comparison of levosimendan, milrinone and dobutamine in treating low cardiac output syndrome following valve replacement surgeries with cardiopulmonary bypass. J Clin Diagn Res. (2016) 10:UC05–08. doi: 10.7860/JCDR/2016/23584.8987

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Lomivorotov VV, Efremov SM, Kirov MY, Fominskiy EV, Karaskov AM. Low-cardiac-output syndrome after cardiac surgery. J Cardiothorac Vasc Anesth. (2017) 31:291–308. doi: 10.1053/j.jvca.2016.05.029

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Kumar G, Iyer PU. Management of perioperative low cardiac output state without extracorporeal life support: what is feasible? Ann Pediatr Cardiol. (2010) 3:147–58. doi: 10.4103/0974-2069.74045

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Ding W, Ji Q, Shi Y, Ma R. Predictors of low cardiac output syndrome after isolated coronary artery bypass grafting. Int Heart J. (2015) 56:144–9. doi: 10.1536/ihj.14-231

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Du X, Chen H, Song X, Wang S, Hao Z, Yin L, et al. Risk factors for low cardiac output syndrome in children with congenital heart disease undergoing cardiac surgery: a retrospective cohort study. BMC Pediatr. (2020) 20:87. doi: 10.1186/s12887-020-1972-y

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Posada-Martinez EL, Fritche-Salazar JF, Arias-Godinez JA, Ortiz-Leon XA, Balderas-Muñoz K, Ruiz-Esparza ME, et al. Right ventricular longitudinal strain predicts low-cardiac- output syndrome after surgical aortic valve replacement in patients with preserved and mid-range ejection fraction. J Cardiothorac Vasc Anesth. (2021) 35:1638–45. doi: 10.1053/j.jvca.2020.12.008

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Algarni KD, Maganti M, Yau TM. Predictors of low cardiac output syndrome after isolated coronary artery bypass surgery: trends over 20 years. Ann Thorac Surg. (2011) 92:1678–84. doi: 10.1016/j.athoracsur.2011.06.017

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Gomez-Marcos MA, Martinez-Salgado C, Gonzalez-Sarmiento R, Hernandez-Rivas JM, Sanchez-Fernandez PL, Recio-Rodriguez JI, et al. Association between different risk factors and vascular accelerated ageing (EVA study): study protocol for a cross-sectional, descriptive observational study. BMJ Open. (2016) 6:e011031. doi: 10.1136/bmjopen-2016-011031

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Salmasi V, Maheshwari K, Yang D, Mascha EJ, Singh A, Sessler DI, et al. Relationship between intraoperative hypotension, defined by either reduction from baseline or absolute thresholds, and acute kidney and myocardial injury after noncardiac surgery: a retrospective cohort analysis. Anesthesiology. (2017) 126:47–65. doi: 10.1097/ALN.0000000000001432

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Gregory A, Stapelfeldt WH, Khanna AK, Smischney NJ, Boero IJ, Chen Q, et al. Intraoperative hypotension is associated with adverse clinical outcomes after noncardiac surgery. Anesth Analg. (2021) 132:1654–65. doi: 10.1213/ANE.0000000000005250

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Beretta L, Santaniello A. Nearest neighbor imputation algorithms: a critical evaluation. BMC Med Inform Decis Mak. (2016) 16(Suppl. 3):74. doi: 10.1186/s12911-016-0318-z

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Alyass A, Turcotte M, Meyre D. From big data analysis to personalized medicine for all: challenges and opportunities. BMC Med Genomics. (2015) 8:33. doi: 10.1186/s12920-015-0108-y

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Yang J, Li Y, Liu Q, Li L, Feng A, Wang T, et al. Brief introduction of medical database and data mining technology in big data era. J Evid Based Med. (2020) 13:57–69. doi: 10.1111/jebm.12373

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Yang L, Wu H, Jin X, Zheng P, Hu S, Xu X, et al. Study of cardiovascular disease prediction model based on random forest in eastern China. Sci Rep. (2020) 10:5245. doi: 10.1038/s41598-020-62133-5

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Hou N, Li M, He L, Xie B, Wang L, Zhang R, et al. Predicting 30-days mortality for MIMIC-III patients with sepsis-3: a machine learning approach using XGboost. J Transl Med. (2020) 18:462. doi: 10.1186/s12967-020-02620-5

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Qian Q, Wu J, Wang J, Sun H, Yang L. Prediction models for AKI in ICU: a comparative study. Int J Gen Med. (2021) 14:623–32. doi: 10.2147/IJGM.S289671

PubMed Abstract | CrossRef Full Text | Google Scholar

23. De Backer D, Vincent J. The pulmonary artery catheter: is it still alive? Curr Opin Crit Care. (2018) 24:204–8. doi: 10.1097/MCC.0000000000000502

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Joseph C, Garrubba M, Smith J, Melder A. Does the use of a pulmonary artery catheter make a difference during or after cardiac surgery? Heart Lung Circ. (2018) 27:952–60. doi: 10.1016/j.hlc.2018.02.004

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Chandler HK, Kirsch R. Management of the low cardiac output syndrome following surgery for congenital heart disease. Curr Cardiol Rev. (2016) 12:107–11. doi: 10.2174/1573403X12666151119164647

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Mebazaa A, Pitsis AA, Rudiger A, Toller W, Longrois D, Ricksten S-E, et al. Clinical review: practical recommendations on the management of perioperative heart failure in cardiac surgery. Crit Care Lond Engl. (2010) 14:201. doi: 10.1186/cc8153

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Zarragoikoetxea I, Vicente R, Pajares A, Carmona P, Lopez M, Moreno I, et al. Quantitative transthoracic echocardiography of the response to dobutamine in cardiac surgery patients with low cardiac output syndrome. J Cardiothorac Vasc Anesth. (2020) 34:87–96. doi: 10.1053/j.jvca.2019.08.019

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Amabili P, Benbouchta S, Roediger L, Senard M, Hubert MB, Donneau A-F, et al. Low cardiac output syndrome after adult cardiac surgery: predictive value of peak systolic global longitudinal strain. Anesth Analg. (2018) 126:1476–83. doi: 10.1213/ANE.0000000000002605

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Lai H-L, Grachoff M, Marion AL, Khan FF, Warren CM, Chowdhury SAK, et al. Maintenance of adult cardiac function requires the chromatin factor Asxl2. J Mol Cell Cardiol. (2012) 53:734–41. doi: 10.1016/j.yjmcc.2012.08.014

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Evans L, Rhodes A, Alhazzani W, Antonelli M, Coopersmith CM, French C, et al. Surviving sepsis campaign: international guidelines for management of sepsis and septic shock 2021. Intensive Care Med. (2021) 47:1181–247. doi: 10.1007/s00134-021-06506-y

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Maganti MD, Rao V, Borger MA, Ivanov J, David TE. Predictors of low cardiac output syndrome after isolated aortic valve surgery. Circulation. (2005) 112:I448–52. doi: 10.1161/CIRCULATIONAHA.104.526087

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Zeger SL, Irizarry R, Peng RD. On time series analysis of public health and biomedical data. Annu Rev Public Health. (2006) 27:57–79. doi: 10.1146/annurev.publhealth.26.021304.144517

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Tseng P-Y, Chen Y-T, Wang C-H, Chiu K-M, Peng Y-S, Hsu S-P, et al. Prediction of the development of acute kidney injury following cardiac surgery by machine learning. Crit Care. (2020) 24:478. doi: 10.1186/s13054-020-03179-9

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Bihorac A, Ozrazgat-Baslanti T, Ebadi A, Motaei A, Madkour M, Pardalos PM, et al. MySurgeryRisk: development and validation of a machine-learning risk algorithm for major complications and death after surgery. Ann Surg. (2019) 269:652–62. doi: 10.1097/SLA.0000000000002706

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Chen L, Hong L, Ma A, Chen Y, Xiao Y, Jiang F, et al. Intraoperative venous congestion rather than hypotension is associated with acute adverse kidney events after cardiac surgery: a retrospective cohort study. Br J Anaesth. (2022) 128:785–95. doi: 10.1016/j.bja.2022.01.032

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Rettig TCD, Peelen LM, Geuzebroek GSC, van Klei WA, Boer C, van der Veer JW, et al. Impact of intraoperative hypotension during cardiopulmonary bypass on acute kidney injury after coronary artery bypass grafting. J Cardiothorac Vasc Anesth. (2017) 31:522–8. doi: 10.1053/j.jvca.2016.07.040

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Smith G. Step away from stepwise. J Big Data. (2018) 5:32. doi: 10.1186/s40537-018-0143-6

CrossRef Full Text | Google Scholar

38. Kursa MB, Rudnicki WR. Feature selection with the boruta package. J Stat Softw. (2010) 36:1–13. doi: 10.18637/jss.v036.i11

CrossRef Full Text | Google Scholar