A Novel Risk Prediction Model for Severe Acute Kidney Injury in Intensive Care Unit Patients Receiving Fluid Resuscitation

Background: To develop a risk prediction model for the occurrence of severe acute kidney injury (AKI) in intensive care unit (ICU) patients receiving fluid resuscitation.

Methods: We conducted a secondary analysis of the Crystalloid vs. Hydroxyethyl Starch Trial (CHEST) trial, a blinded randomized controlled trial that enrolled ICU patients who received intravenous fluid resuscitation. The primary outcome was the first event in a composite outcome of doubling of serum creatinine and/or treatment with renal replacement treatment (RRT) within 28 days of randomization. The final model developed using multivariable logistic regression with backwards elimination was validated internally and then translated into a predictive equation.

Results: Six thousand seven hundred twenty-seven ICU participants were studied, among whom 745 developed the study outcome. The final model having six variables, including admission diagnosis of sepsis, illness severity score, mechanical ventilation, tachycardia, baseline estimated glomerular filtration rate and emergency admission. The model had good discrimination (c-statistic = 0.72, 95% confidence interval 0.697–0.736) and calibration (Hosmer-Lemeshow test, χ² = 14.4, p = 0.07) for the composite outcome, with a c-statistic after internal bootstrapping validation of 0.72, which revealed a low degree of over-fitting. The positive predictive value and negative predictive value were 58.8 and 89.1%, respectively. The decision curve analysis indicates a net benefit in prediction of severe AKI using the model across a range of threshold probabilities between 5 and 35%.

Conclusions: Our model, using readily available clinical variables to identify ICU patients at high risk of severe AKI achieved good predictive performance in a clinically relevant population.

Background

Acute kidney injury (AKI) is common in the intensive care unit (ICU), with a reported incidence of 7–25% for adults (1, 2). The close association between AKI and a range of adverse outcomes, including death, is well accepted, but there are few validated tools to identify ICU patients most at risk of these outcomes (3–5).

The development of consensus definitions for AKI has allowed more consistent diagnosis and comparison between different populations. There are three main AKI definitions in widespread use (6–8) which are broadly similar, using combinations of reductions in urine output, increases in serum creatinine and/or treatment with renal replacement therapy (RRT), to classify the severity of kidney injury. These definitions categorize AKI into stages of increasing severity that correlate with adverse outcomes such as increased mortality and prolonged length of hospital stay (9).

To facilitate early diagnosis and treatment, several models to predict AKI in ICU patients have been proposed (10). However, these models have examined a mixture of approaches (11–15), with data from varying numbers of ICUs and including both randomized trial cohorts and observational cohorts, some of which include specific biomarkers that are not in routine clinical use yet (12–15). In addition, the studied populations are often selected and this may affect the predictive ability and applicability to broader ICU populations of the resultant models. Patients receiving intravenous fluid resuscitation in ICU are common, readily identifiable, and may represent an enriched population at increased risk of severe AKI.

The Crystalloid vs. Hydroxyethyl Starch Trial (CHEST) was a multicenter, prospective, randomized-controlled, clinical trial (RCT) that compared the efficacy and safety of 6% hydroxyethyl starch (HES) (130/0.4) and 0.9% sodium chloride (normal saline) for intravenous fluid resuscitation in patients treated in ICU (16). A key secondary outcome was the incidence and severity of AKI using the Risk, Injury, Failure, Loss and End-stage kidney injury (RIFLE) criteria (7) and the incidence and duration of associated RRT. Using this trial database, we developed a prediction model to determine the risk of developing severe AKI for ICU patients receiving fluid resuscitation.

Methods

Study Design

In brief, CHEST was a multicenter, prospective, randomized-controlled, clinical trial (RCT) that compared the use of HES and saline for intravenous fluid resuscitation in patients in 32 ICUs in Australia and New Zealand (Clinicaltrials.gov identifier#: NCT00935168). Detailed descriptions of the study protocol, statistical analysis plan and results have been published previously (16, 17). The CHEST study was approved by the human research Ethics Committee of Northern Sydney Central Coast Health (AU RED Ref: HREC/09/HARBR/14), and by each participating institution. The present study is a post-hoc analysis of the CHEST database with the objective of developing a model for the prediction of severe AKI in ICU patients receiving fluid resuscitation.

Study Outcomes

The primary outcome of the model is severe AKI, defined as the first event in a composite outcome incorporating doubling of serum creatinine (from the pre-randomization value) and/or treatment with RRT within 28 days of randomization. It is important to note that this outcome differs from the reported AKI outcome in the primary CHEST article, which used the RIFLE criteria (17) defined by changes in either urine output or serum creatinine from randomization.

Demographic and Clinical Variables

In order to minimize selection bias, our analysis included all patients from the original study population for whom consent to use of their data was obtained and were not lost to follow-up. Demographic and clinical variables were collected as described previously (16), and variables collected immediately before randomization were deemed as baseline variables. We examined 17 candidate baseline variables that were prospectively considered by the authors to be potential indicators of AKI risk: age, sex, weight, heart rate (HR), central venous pressure (CVP), mean arterial pressure (MAP), urine output during 6 h prior to randomization, serum creatinine (serum Cr), estimated glomerular filtration rate (eGFR), serum lactate concentration, Acute Physiology and Chronic Health Evaluation (APACHE) II score (18), Sequential Organ Failure Assessment (SOFA) score for the cardiovascular system (19), the presence of sepsis (20), the presence of trauma, source of admission, treatment with mechanical ventilation, and the nature of ICU admission (surgical or nonsurgical). As the allocated study treatment (HES or saline) was assigned at randomization, this variable was not considered as a baseline variable and excluded from multivariate regression models.

eGFR was calculated using the Chronic Kidney Disease Epidemiology Collaboration (CKD-EPI) equation (21). The source of admission was defined as the place from where the patients had been transferred to ICU and was classified as: hospital floor, emergency department, operating room following elective surgery, operating room following emergency surgery, and other hospitals. Mechanical ventilation included either invasive ventilation via an endotracheal tube or non-invasive respiratory support via a mask or other non-invasive interface.

Statistical Analysis

As the analysis focused on the baseline clinical variables, all patients in the baseline analysis of the original study publication with complete demographic data and information about the study outcome were included. Continuous variables were presented as median (interquartile range) or mean ± standard deviation (SD), categorical variables were presented as number (percentage). APACHE II score was considered as a continuous variable. Univariable regression was used to examine the relationship between candidate predictors and the study outcome. Multivariable logistic regression model was used to identify independent risk factors for the study outcome.

For modeling, we followed a three-step procedure. First, we developed a primary model by multivariable logistic regression using a backward elimination approach (threshold p < 0.05) to select variables (22) after excluding variables that would potentially have high risk of in the development population. Possible first-order interactions were also explored unavailability in real-world clinical practice based on their proportions of missing data and kept in the model if statistically significant (p < 0.05). Second, we developed a secondary model by removing interaction terms in the primary model and compared the performance of these two models using c-statistics (equal to the area under the receiver-operating characteristic curve). If no difference between these primary and secondary models was found, the secondary model was preferred due to its greater simplicity and avoidance of over-fitting, consistent with the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statements (23).Third, we simplified the model from step two by removing variables that were not statistically significant and tested the performance of the final parsimonious model. The three models were compared, and the final model was chosen as the best balance between calibration, discrimination and clinical practicability. To test the performance of the model, discrimination was assessed by c-statistics (24), and calibration was assessed by Hosmer-Lemeshow test (25) and calibration plot.

In addition we performed two sensitivity analyses. The first tested whether forcing the randomization treatment (HES or normal saline) into the multivariable regression model, altered the model's predictive performance, and the second examined whether the timing of the primary outcome, at 7 days rather than 28 days from randomization altered model performance to be consistent with current diagnostic criteria of AKI.

For validation, we randomly sampled from the study dataset using the bootstrapping method (26) (n = 10,000 replications) to evaluate the over-optimism inherent in the final model. Confidence intervals of the over-fitting in c-statistics for the bootstrap corrections were computed based on the assumption of normal distributions. The bootstrap-adjusted performance was calculated and the final risk prediction model was translated into a predictive equation. Positive predictive value (PPV) and negative predictive value (NPV) were calculated, and a decision curve analysis was performed to evaluate the net benefit of using the model in predicting the primary outcome across a range of threshold probabilities.

The reporting of this prognostic model study followed the TRIPOD statement (23). The risk of bias of the final model was assessed using the Prediction Model Risk of Bias Assessment Tool (PROBAST) (27). In addition, we used the final model and REDCap electronic data capture tools (28) securely hosted at Sichuan Provincial People's Hospital to develop a web-based risk assessment tool accessible to readers.

All analyses were performed using SAS/STAT software v.9.1 (SAS Institute Inc., Cary, NC, USA) with statistical significance set at P < 0.05.

Results

Baseline Characteristics and Study Outcome

Baseline variables and the study outcome in the study population are listed in Table 1. Of the 6,742 patients reported in the original publication, 15 patients were excluded because of missing RRT follow up data, leaving 6,727 patients for analysis. The majority of the 6,727 patients entered ICU from the operating room or emergency department. Data for urine output during 6 h prior to randomization, baseline CVP and baseline serum lactate were available for only 2,802, 2,300 and 5,555 patients respectively, so these three variables were excluded from modeling (see Table 1). The proportion of individuals with at least one variable with missing value was 2.79% (188/6727).

TABLE 1

Table 1. Baseline characteristics and the study outcome of the study population.

Within 28 days after randomization, 514 (7.6%) participants experienced a doubling of serum Cr, 427 (6.4%) were treated with RRT and 196 (2.9%) experienced both events. In total 745 (11.1%) participants developed the study outcome (Figure 1).

FIGURE 1

Figure 1. Participant flow diagram. Cr, creatinine; RRT, renal replacement treatment.

Univariate Analysis of Candidate Predictors

The univariate relationships of all candidate model variables with the study outcome are presented in Table 2. The majority of these variables were associated with the study outcome, but sex, MAP, admission following emergency surgery (compared to admission from emergency department) and mechanical ventilation were not.

TABLE 2

Table 2. Univariate odds ratios of candidate predictive variables for the study outcome.

Prediction Model

Evaluated variables in each model were examined and compared (Supplementary Table S1). In addition to stand-alone significant variables including age, baseline eGFR, heart rate, APACHE II score, sepsis, mechanical ventilation at admission, and admission source, the primary model (Model A) also included eight interactive terms as significant factors. However, the performance of the secondary model (Model B) after removing interactive terms did not materially differ from the primary model (Figure 2). After removing these interactive terms, seven variables became non-significant, resulting in the final model (Model C) including six significant predictors (Table 3). The comparison of all three models indicated no meaningful difference in C-statistics for the primary, secondary and final models (Figure 2).

FIGURE 2

Figure 2. Receiver Operating Curves comparing the discrimination of the three models. Each model was adjusted for randomly assigned treatments. See details in Methods.

TABLE 3

Table 3. Odds ratios of independently significant predictors in the final model.

The sensitivity analysis of including randomization treatment in the final model showed no change in the effect of the six predictors or the C-statistic (Table 4). The sensitivity analysis of the ability of the final model to predict doubling of serum creatinine and RRT within 7 days after randomization again showed no change in the effect of the model variables indicated all predictors remained significant except for mechanical ventilation at admission (Supplementary Table S2).

TABLE 4

Table 4. Sensitivity analysis of including randomization treatment in the multivariable regression model.

Final Model Performance and Internal Validation

The observed and predicted risk of the composite primary outcome based on the final model were similar, with a test of goodness of fit indicating good calibration (modified Hosmer-Lemeshow test, χ² = 14.4, p = 0.07) (Figure 3). Internal validation using the bootstrap method revealed the degree of over-optimism on c-statistics of the final prediction model was 0.0055 (95% CI, −0.0129 to 0.0240), resulting in an equivalent c-statistic after bootstrap validation of 0.711. The PPV and NPV of the final model to predict severe AKI in ICU patients receiving fluid resuscitation within 28 days from admission were 58.8 and 89.1%, respectively. The decision curve analysis indicates a net benefit in prediction of severe AKI using the model across a range of threshold probabilities between 5 and 35% (Figure 4), illustrating the additional benefit of the model in predicting severe AKI over and above the approaches of “intervention for all” and “intervention for none”.

FIGURE 3

Figure 3. Calibration plot of the final model. Predicted risk was indicated on horizontal axis, whereas observed risk was indicated on vertical axis. These results were based on patients grouped into deciles of predicted risk. Modified Hosmer-Lemeshow test indicated good fitness (p = 0.07).

FIGURE 4

Figure 4. Decision curve based on the final model. Decision curve analysis is a relatively recent approach, seeking to overcome the limitation of the usual model assessment tools such as calibration and discrimination in their clinical application. If one doesn't use the model, then any intervention could be applied to everyone (intervention for all) or no-one (intervention for none). Between these two interventions sits the impact of the model, and the fact that our curve sits above the intersection of both curves, across the range of probabilities for AKI in our study population, suggest that the model will be of net benefit in this population. At a population level, the net benefit of the intervention treatment will be realized across a larger number of patients across the spectrum of risk.

Risk of Bias Assessment of the Final Model

According to the PROBAST assessment, 19 of 20 signaling questions were rated as “Yes,” and 1 was rated as “Probably Yes,” thus the overall risk of bias of the final model was rated as low risk of bias. Detailed rationales of answers were described in Supplementary Table S3.

Risk Predictive Equation for the Study Outcome

From the above computation, the following risk prediction equation was derived:

$\begin{array}{ll}{\text{P}}_{\text{outcome}}=1/(1+exp(-A))& (1)\end{array}$

where P_outcome indicates the probability for the study outcome occurring, and A = −0.0101 ^* eGFR + 0.0161 ^* Heart Rate + 0.0378^*APACHE II score + 0.4577^* (if sepsis) + 0.3747^* (if admitted from hospital floor) + 0.2080^* (if admitted from OR after elective surgery) + 0.2577^* (if admitted from OR after emergency surgery) + 0.3699^* (if admitted from other hospitals) + 0.2154^* (if mechanical ventilation on admission) – 4.1529.

The prediction model was translated using REDCap to an online calculator for readers' convenience (to visit, please scan the QR code in Supplementary Figure S1 using mobile devices or visit: http://redcap.scrds.net/surveys/ and enter code: 7KMH8HN7N). The probability would be shown automatically at the completion of all predictors. Readers are encouraged to submit their data that can be collected for further improvement of the model.

Discussion

We report the development of a model for predicting severe AKI risk in a population of ICU patients who received fluid resuscitation, using data from a large, multi-center randomized trial. The final model includes six significant clinical predictors: admission diagnosis of sepsis, illness severity score, mechanical ventilation, tachycardia, baseline estimated glomerular filtration rate and emergency admission; all of which are readily evaluable close to the point of ICU admission and, together, demonstrate robust predictive ability, discrimination and calibration.

Our findings broadly accord with those of other similar recent studies (12, 15, 29, 30), but also highlight some important challenges in this field. Flechet et al.'s model (12) is the closest in design to our analysis, being a randomized trial dataset collected across 7 centers (the Early Parenteral Nutrition Completing Enteral Nutrition in Adult Critically Ill Patients Study, EPaNIC), and both models share a number of predictive variables such as baseline renal function, requirement for emergency surgery and clinical suspicion of sepsis at baseline. Flechet's report included several models defined by different time points in patients' hospital stay, with their “admission model” the most analogous to our final model. Its discriminative ability for AKI within the first week of ICU stay was similar to our finding (c statistic of 0.75), and improved when additional variables from the first 24 h following admission to ICU were included (c statistic rising to 0.82). The underlying populations had some important differences, with the EPaNIC study having a higher proportion of patients entering ICU following elective cardiac surgery and a 30% lower mortality rate, but similar rates of renal replacement therapy use, compared to the CHEST population (16, 31).

Malhotra and colleagues used prospective data from a single center to derive a model and validated it using data from another US facility (29). This model included several chronic disease conditions along with acute risk factors such as acidosis, treatment with mechanic ventilation and the presence of sepsis. The discrimination of this model was high in both development and validation cohorts, but the inclusion of data from the first 48 h of participants ICU stay likely played a part in this, as Flechet et al. also saw increases in the c statistic as data from later in the patient journey was added to their admission model (12). Koyner et al.'s model, which included both general hospital and ICU patients, using extensive electronic health record data, had a better discrimination compared with our model, which may be in part due to greater number of variables and the use of data from after the patients' admission (30).

Our model benefits from the prospective nature of the CHEST study, which examined a large, clearly defined cohort of general ICU patients, was successful in recruiting patients within a median of 12 h of ICU admission, and uses data from a large number of centers. The resultant final model is relatively simple, enhancing usability and reducing the risk of model over-fitting, and estimates a probability for more severe AKI in the 28 days after baseline. For example, a septic patient with a baseline eGFR of 30 ml/min/1.73 m², heart rate of 90, APACHE II score of 18 who has been transferred to ICU from the hospital floor on mechanical ventilation, the risk of developing the composite event (doubling of serum Cr or treatment with RRT) in the following 28 days would be 21.8%. However, external validation of these findings is an important next step.

Our modeling is the first in this area to use decision curve analysis, first described in 2006 (32), as a tool to understand the “net benefit” of a predictive model compared to strategies that clinically act upon all or none of defined patient groups. A challenge in interpreting these results is understanding the nature of clinical responses to a severe AKI diagnosis, where there is an absence of effective treatments proven to reduce the occurrence, or outcomes, of severe AKI. However, identification of populations at high risk of severe AKI may assist in the prognostic enrichment of clinical trials that test treatments or strategies to prevent AKI, and could be used to assess baseline balance in risk of developing AKI in clinical trials where AKI is a trial endpoint. Additionally, our model may also have value in smaller intensive care units, where RRT support is limited, by aiding timely decisions to transfer patients at higher risk to centers with greater capacity to treat AKI. As such, the low risk of harms from applying the model and responding to a diagnosis of severe AKI, would suggest the threshold probability for its use is at the lower end of the presented range, where the net benefit is most pronounced.

Our study has limitations. First, the study population was derived from an existing clinical trial dataset and, whilst the inclusion criteria for the CHEST study were broad, it excluded important patient groups such as those with advanced AKI at baseline and children. Second, a relatively small proportion of the patients developed the study outcome, accounting for 11.1% of the original population, which was lower than the average reported prevalence of AKI in general ICU population (1, 2), and was likely a function of study entry criteria. Third, the prediction model did not include urine output, which is a part of most definitions for AKI, as the data were incomplete for a large proportion of the study population; an issue also seen in other published models (10, 29). Fourth, the PPV is relatively low, indicating that we are less confident in predicting a patient who would develop AKI in 28 days than in ruling out a patient who would not develop the same outcomes. Fifth, it needs to be borne in mind that the dataset underlying this model includes 620 deaths that did not experience the study outcome, which mostly occurred within the first 10 days of the study. Finally, external validation is central to fully understanding the value of our model.

Conclusions

We have developed a novel risk prediction model for severe acute kidney injury in a population of general ICU patients receiving fluid resuscitation. The final model includes six significant predictors and has good discrimination and calibration for the composite outcome of doubling of serum creatinine or treatment with RRT within 28 days. External validation is needed to explore its generalizability.

Data Availability Statement

The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author.

Ethics Statement

The studies involving human participants were reviewed and approved by human research Ethics Committee of Northern Sydney Central Coast Health. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

JM, SF, MJ, VP, AW, and MG conceived the study. QL, YF, and MJ did the data analyses. The article was written by YF, with input from all co-authors. All authors interpreted the results, contributed to the critical revising of the manuscript, and have approved the final version.

Funding

The CHEST study was funded by a project grant from the National Health and Medical Research Council of Australia and by unrestricted grants from the New South Wales Ministry of Health and Fresenius Kabi, the manufacturer of Voluven^®. FYL is supported in part by an Australian Government Research Training Program Scholarship (RTP) for study toward a Ph.D. in the Faculty of Medicine of UNSW, Australia. AW is supported by RACP Jacquot Research Establishment Fellowship, Australia. Funding agencies had no input into the design, conduct, data collection, statistical analysis, or writing of the manuscript.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

The authors thank everybody involved in the CHEST trial: research staff and investigators, clinical staff, patients, and their families.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fcvm.2022.840611/full#supplementary-material

References

1. de Mendonca A, Vincent JL, Suter PM, Moreno R, Dearden NM, Antonelli M, et al. Acute renal failure in the ICU: risk factors and outcome evaluated by the SOFA score. Intensive Care Med. (2000) 26:915–21. doi: 10.1007/s001340051281

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Yang L, Xing G, Wang L, Wu Y, Li S, Xu G, et al. Acute kidney injury in China: a cross-sectional survey. Lancet. (2015) 386:1465–71. doi: 10.1016/S0140-6736(15)00344-X

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Uchino S, Kellum JA, Bellomo R, Doig GS, Morimatsu H, Morgera S, et al. Acute renal failure in critically ill patients: a multinational, multicenter study. JAMA. (2005) 294:813–8. doi: 10.1001/jama.294.7.813

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Kaddourah A, Basu RK, Bagshaw SM, Goldstein SL, Investigators A. Epidemiology of Acute Kidney Injury in Critically Ill Children and Young Adults. N Engl J Med. (2017) 376:11–20. doi: 10.1056/NEJMoa1611391

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Fuhrman DY, Kane-Gill S, Goldstein SL, Priyanka P, Kellum JA. Acute kidney injury epidemiology, risk factors, and outcomes in critically ill patients 16-25 years of age treated in an adult intensive care unit. Ann Intensive Care. (2018) 8:26. doi: 10.1186/s13613-018-0373-y

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Mehta RL, Kellum JA, Shah SV, Molitoris BA, Ronco C, Warnock DG, et al. Acute Kidney Injury Network: report of an initiative to improve outcomes in acute kidney injury. Crit Care. (2007) 11:R31. doi: 10.1186/cc5713

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Bellomo R Ronco C Kellum JA Mehta RL Palevsky P and Acute Dialysis Quality Initiative w. Acute renal failure - definition, outcome measures, animal models, fluid therapy and information technology needs: the Second International Consensus Conference of the Acute Dialysis Quality Initiative (ADQI) Group. Crit Care. (2004) 8:R204–12. doi: 10.1186/cc2872

PubMed Abstract | CrossRef Full Text | Google Scholar

8. KDIGO AKI Work Group: KDIGO clinical practice guideline for acute kidney injury. Kidney Int Suppl. (2012) 2:1–138. doi: 10.1038/kisup.2012.1

CrossRef Full Text | Google Scholar

9. Zeng X, McMahon GM, Brunelli SM, Bates DW, Waikar SS. Incidence, outcomes, and comparisons across definitions of AKI in hospitalized individuals. Clin J Am Soc Nephrol. (2014) 9:12–20. doi: 10.2215/CJN.02730313

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Neyra JA, Leaf DE. Risk Prediction Models for Acute Kidney Injury in Critically Ill Patients: Opus in Progressu. Nephron. (2018) 140:99–104. doi: 10.1159/000490119

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Sutherland SM, Chawla LS, Kane-Gill SL, Hsu RK, Kramer AA, Goldstein SL, et al. Utilizing electronic health records to predict acute kidney injury risk and outcomes: workgroup statements from the 15(th) ADQI Consensus Conference. Can J Kidney Health Dis. (2016) 3:11. doi: 10.1186/s40697-016-0099-4

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Flechet M, Guiza F, Schetz M, Wouters P, Vanhorebeek I, Derese I, et al. AKIpredictor, an online prognostic calculator for acute kidney injury in adult critically ill patients: development, validation and comparison to serum neutrophil gelatinase-associated lipocalin. Intensive Care Med. (2017) 43:764–73. doi: 10.1007/s00134-017-4678-3

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Basu RK WY, Wong HR, Chawla LS, Wheeler DS, Goldstein SL. Urinary biomarker incorporation into the renal angina index early in intensive care unit admission optimizes acute kidney injury prediction in critically ill children: a prospective cohort study. Nephrol Dial Transplant. (2016) 31:9. doi: 10.1093/ndt/gfv457

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Isshiki R, Asada T, Sumida M, Hamasaki Y, Nangaku M, Noiri E, et al. Modest Impact of Serial Measurements of Acute Kidney Injury Biomarkers in an Adult Intensive Care Unit. Nephron. (2018) 139:243–53. doi: 10.1159/000488219

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Kashani K, Al-Khafaji A, Ardiles T, Artigas A, Bagshaw SM, Bell M, et al. Discovery and validation of cell cycle arrest biomarkers in human acute kidney injury. Crit Care. (2013) 17:R25. doi: 10.1186/cc12503

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Myburgh JA, Finfer S, Bellomo R, Billot L, Cass A, Gattas D, et al. Hydroxyethyl starch or saline for fluid resuscitation in intensive care. N Engl J Med. (2012) 367:1901–11. doi: 10.1056/NEJMoa1209759

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Crystalloid versus Hydroxyethyl Starch Trial Management C. The Crystalloid versus Hydroxyethyl Starch Trial: protocol for a multi-centre randomised controlled trial of fluid resuscitation with 6% hydroxyethyl starch (130/0.4) compared to 0.9% sodium chloride (saline) in intensive care patients on mortality. Intensive Care Med. (2011) 37:816–23. doi: 10.1007/s00134-010-2117-9

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Knaus WA, Draper EA, Wagner DP, Zimmerman JE. APACHE II a severity of disease classification system. Crit Care Med. (1985) 13:818–29. doi: 10.1097/00003246-198510000-00009

CrossRef Full Text | Google Scholar

19. Vincent JL dMA, Cantraine F, Moreno R, Takala J, Suter PM, Sprung CL, Colardyn F, Blecher S. Use of the SOFA score to assess the incidence of organ dysfunction/failure in intensive care units: results of a multicenter, prospective study. Crit Care Med. (1998) 26:7. doi: 10.1097/00003246-199811000-00016

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Bone RC BR, Cerra FB, Dellinger RP, Fein AM, Knaus WA, Schein RM, et al. Definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis. The ACCP/SCCM Consensus Conference Committee American College of Chest Physicians/Society of Critical Care Medicine. Chest. (1992) 101:12. doi: 10.1378/chest.101.6.1644

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Levey AS, Inker LA, Coresh J. GFR estimation: from physiology to public health. Am J Kidney Dis. (2014) 63:820–34. doi: 10.1053/j.ajkd.2013.12.006

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Jardine MJ, Hata J, Woodward M, Perkovic V, Ninomiya T, Arima H, et al. Prediction of kidney-related outcomes in patients with type 2 diabetes. Am J Kidney Dis. (2012) 60:770–8. doi: 10.1053/j.ajkd.2012.04.025

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Collins GS, Reitsma JB, Altman DG, Moons KG. Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): the TRIPOD statement. Ann Intern Med. (2015) 162:55–63. doi: 10.7326/M14-0697

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Pencina MJ. DAR Overall C as a measure of discrimination in survival analysis: model specific population value and confidence interval estimation. Stat Med. (2004) 23:15. doi: 10.1002/sim.1802

PubMed Abstract | CrossRef Full Text | Google Scholar

25. D'Agostino RB NB. Evaluation of the performance of survival analysis models: discrimination and calibration models. Handb Stat. (2004) 23:25. doi: 10.1016/S0169-7161(03)23001-7

CrossRef Full Text | Google Scholar

26. Harrell FE Jr, Lee KL, Mark DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med. (1996) 15:361–87. doi: 10.1002/(SICI)1097-0258(19960229)15:4<361::AID-SIM168>3.0.CO

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Wolff RF, Moons KGM, Riley RD, Whiting PF, Westwood M, Collins GS, et al. PROBAST: a tool to assess the risk of bias and applicability of prediction model studies. Ann Intern Med. (2019) 170:51–8. doi: 10.7326/M18-1376

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)–a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. (2009) 42:377–81. doi: 10.1016/j.jbi.2008.08.010

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Malhotra R, Kashani KB, Macedo E, Kim J, Bouchard J, Wynn S, et al. A risk prediction score for acute kidney injury in the intensive care unit. Nephrol Dial Transplant. (2017) 32:814–22. doi: 10.1093/ndt/gfx026

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Koyner JL, Carey KA, Edelson DP, Churpek MM. The Development of a Machine Learning Inpatient Acute Kidney Injury Prediction Model. Crit Care Med. (2018) 46:1070–7. doi: 10.1097/CCM.0000000000003123

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Casaer MP, Hermans G, Wilmer A, Van den Berghe G. Impact of early parenteral nutrition completing enteral nutrition in adult critically ill patients (EPaNIC trial): a study protocol and statistical analysis plan for a randomized controlled trial. Trials. (2011) 12:21. doi: 10.1186/1745-6215-12-21

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Making. (2006) 26:565–74. doi: 10.1177/0272989X06295361

PubMed Abstract | CrossRef Full Text | Google Scholar