Development and Validation of a Risk Score for Predicting Post-acute Myocardial Infarction Infection in Patients Undergoing Percutaneous Coronary Intervention: Study Protocol for an Observational Study

Background: Post-acute myocardial infarction (post-AMI) infection is an infrequent but important and serious complication in patients with ST-segment elevation myocardial infarction (STEMI) treated with percutaneous coronary intervention (PCI). Predicting its occurrence is essential for future prevention. However, little is known about the prediction of post-AMI infection in such patients to date. This study aims to develop and validate a new risk score based on risk factors for early prediction of infection in STEMI patients undergoing PCI.

Methods: This prospective, multi-center and observational study assesses the predictive value of risk score for post-AMI infection among a cohort of patients hospitalized due to STEMI. The STEMI patients undergoing PCI enrolled between January 1st 2010 and May 31st 2016 were served as a development cohort while those enrolled from June 1st 2016 to May 31st 2018 were served as validation cohort. The primary endpoint was post-AMI infection during hospitalization, defined as infection requiring antibiotics (reflecting the clinical influence of infection compatible with the necessity for additional treatment), and all-cause death and major adverse cardiovascular events (MACE) including all-cause death, recurrent myocardial infarction, target vessel revascularization, and stroke were considered as secondary endpoints. The risk score model based on risk factors was established using stepwise logistic regression, and will be validated in other centers and external patients with non-ST-elevation acute coronary syndrome (NSTE-ACS).

Results: This study will provide evidence on prognostic property, reliability of scoring, comparative performance, and suitability of the novel model for screening purpose in order to be recommended for clinical practice.

Discussion: Our study is designed to develop and validate a clinical risk score for predicting infection in participants with STEMI who have undergone PCI. This simple tool may therefore improve evaluation of post-AMI infection and enhance future researches into the best practices to prevent or reduce infection in such patients.

Clinical Trial Registration: www.chictr.org.cn, identifier: ChiCTR1900028278.

Introduction

ST-segment elevation myocardial infarction (STEMI) accounts for one-third of patients with acute coronary syndrome (ACS) worldwide, which resulted in poor outcomes and enormous financial losses every year (1). Post-acute myocardial infarction infection (post-AMI) is uncommon but is an important and serious complication in such patients. In addition, infections are crucial causes of morbidity and mortality and are associated with prolonged length of hospital stay and a substantial increase in health care costs (2, 3). Taking these undesirable effects into consideration, infection prevention has been one of the highest priorities for medical resources and government. Given that most infections are preventable, identification of STEMI patients at high-risk of infection could allow interventions to prevent and rapidly treat these infections among these patients.

Although some risk-prevention tools for mortality have been developed for STEMI patients, such as the Global Registry of Acute Coronary Events (GRACE) score (4) and the Thrombolysis in Myocardial Infarction (TIMI) risk score (5), there is no specific risk score for identifying STEMI patients undergoing percutaneous coronary intervention (PCI) to estimate the risk of post-AMI infection. In recent articles, we validated the age, creatinine and left ventricular ejection fraction (ACEF) risk score (6) and the Canada Acute Coronary Syndrome (CACS) risk score (7) for post-AMI infection, however, these scores were not developed specially for infection and the discriminatory accuracy were relatively low. In addition, various risk systems have been developed to predict infections following cardiac surgery (8, 9). Fowler et al. established a model that can identify patients undergoing cardiac surgery who were at high- risk for major infection (9). However, no study to date has validated a simplified scoring system to assess patients' individual risk of infection in STEMI patients undergoing PCI.

Given these data, this research protocol aims to develop and validate a practical risk score to identify the risk of post-AMI infection in patients with STEMI undergoing PCI. And this new risk score based on seven simple independent predictors would be divided into four levels according to the corresponding marks (low-, moderate-, high-, and very high-risk groups) to improve the clinical utility. Therefore, the present risk score could be easily applicable in clinical practice for identifying patients at high-risk of infection and in-hospital outcomes in patients with STEMI undergoing PCI and may help clinical decision-making allowing for timely measures to improve clinical outcomes.

Materials and Methods

Overview

We will follow the recommended procedures by the Prognosis Research Strategy (PROGRESS) Group (10–12) and show results according to the TRIPOD (Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis) statement (13, 14). In this research, we develop the risk score and perform the internal validation in patients admitted to Guangdong Provincial People's Hospital, and external validation is conducted in other centers and NSTE-ACS patients.

Study Design

This proposed protocol involves a prospective, multi-center, observational cohort study (Figure 1). This study expected for enrollment from January 1st 2010 to May 31st 2018. Our study protocol has been approved by the ethics and research committee of Guangdong Provincial People's Hospital (GDREC2016378H) and was registered in the Chinese Clinical Trial Registry (www.chictr.org.cn, ChiCTR1900028278). This study will be designed and performed in accordance with the Code of Ethics of the 1964 Declaration of Helsinki and its later amendments. All enrolled patients will be required to provide written informed consent in this study. Contact information will be provided for participants for further enquiry.

FIGURE 1

Figure 1. Flow chart of study design.

Study Oversight

The data safety committee has been guaranteed for this research, consisting of several cardiologists and one statistician. The cardiologists will be responsible for the review of the in-hospital course of the participants, while the statistician will conduct the safety analysis, and the data will be divided into development and validation cohorts.

Study Population

STEMI was defined as the manifestation of typical chest pain and concomitant symptoms for ≥ 30 min but <12 h, with ST-segment elevation ≥ 1 mm in ≥ 2 continuous leads or new or undetermined duration of left branch bundle block accompanying with ≥ 2 times increase in troponin I or T (15).

The inclusion criteria were as follows: (a) age ≥ 18 years; patients with STEMI undergoing PCI, admitted to the cardiac intensive care unit; (b) patients could understand and agree with informed consent.

The exclusion criteria were as follows: (a) patients diagnosed with infection or treated with antibiotics prior to admission; (b) patients with tumor or chronic inflammatory diseases; (c) patients on hemodialysis at admission; (d) patients undergoing emergency cardiac surgery; (e) patients died within 24 h after admission; (f) readmission to hospital; and (g) patients with missing variables that are needed to calculate the risk scores.

Eligible patients were consecutively enrolled between January 1st 2010 and May 31st 2016, which were served as a development cohort. In accordance with the inclusion and exclusion criteria, patients enrolled from June 1st 2016 to May 31st 2018, with complete data regarding risk factors identified in the development cohort, will be served as a validation cohort. Furthermore, the patients screened according to the criteria in other centers will be enrolled for external validation.

To extend the application, this risk score will also be validated in our previous study including patients diagnosed with non-ST-elevation acute coronary syndrome (NSTE-ACS) following PCI according to the same exclusion criteria (16). This retrospective cohort study was designed to evaluate the association between parenteral anticoagulation therapy and clinical outcomes in patients mentioned above. Patients with principal discharge diagnosis of non-ST elevation myocardial infarction (NSTEMI) or unstable angina (UA) and underwent PCI during the index hospital stay were included. And patients with elevated cardiac biomarker and without new ST-elevation on ECG prior to index PCI were classified as NSTEMI while those were classified as UA if there were no positive cardiac biomarker findings and without new ST-elevation on ECG prior to index PCI (16).

Laboratory Testing

Blood samples were obtained routinely and tested for electrolytes, white blood cell count, hemoglobin, cardiac enzymes, serum creatinine, blood lipids, serum albumin, total bilirubin, estimated glomerular filtration rate (eGFR) that was evaluated using the modified Modification of Diet in Renal Disease (MDRD) equation for Chinese people (17) and other routine laboratory parameters with 24 h after admission, while the medical history, the vital signs including body temperature, pulse rate, respiration rate and blood pressure, and Killip classification were also recorded at the same time.

Procedure and Medications

All eligible participants received a conventional chest X-ray and ultrasonic cardiography. The left ventricular ejection fraction (LVEF) was measured by echocardiography after admission. Furthermore, blood cultures were examined at the discretion of the responsible physicians. Before the procedure, administration of 300 mg aspirin and 180 mg ticagrelor or clopidogrel (300 or 600 mg) were mandatory for all included patients. PCI was performed by the interventional cardiologists in the conventional manner, and coronary stents were, or other procedures were used when required. Hydration with intravenous normal saline solution at a hydration rate of 1 ml/kg/h or 0.5 ml/kg/h if the LVEF was <40% was initiated during the procedure and maintained until 6 to 12 h after the procedure. After completion of this procedure, lifelong aspirin (100 mg/day) was prescribed and at least 12 months of clopidogrel (100 mg/day) or ticagrelor (90 mg twice/day) were recommended to all patients. Additionally, the prescription of anticoagulants, β blockers, glycoprotein IIb/IIIa inhibitors, or angiotensin-converting enzyme inhibitors/ angiotensin receptor blockers (ACEI/ARB) was at the discretion of the operators according to current guidelines.

Data Collection and Missing Data

Clinical data including patient demographics, diagnosis, medical histories, clinical and laboratory characteristics, medication treatment, procedural details and main outcomes were obtained through review of electronic medical records. The principal data, such as clinical outcomes, were collected by two investigators independently, and inconsistent data were verified by a third investigator.

Variables were included as candidates for this risk score after removing variables with ≥ 10% missing values (Table 1), and for those with incomplete data, we assumed that patients with the missing data ≤ 5% occurred at random depending on the clinical variables and used multiple imputations. This risk score model was developed based on imputed complete cases.

TABLE 1

Table 1. Observations and endpoints.

Study Endpoints

Primary Endpoint

The primary outcome was post-AMI infection during hospital admission, which was defined as infection requiring antibiotics (reflecting the clinical influence of infection compatible with the necessity for additional treatment) (18). The types of infections included pulmonary infections, urinary infections or others (including abdominal sepsis, primary bacteremia, and unidentified primary infection site) based on the clinical records during hospitalization. Appropriate antibiotics were used once infection was confirmed (19). The antibiotics were validated by the hospital infection control committee that approved the initiation of therapy for each infection case.

Secondary Endpoints

The secondary endpoints were in-hospital all-cause mortality and major adverse cardiovascular events (MACE) including all-cause death, recurrent myocardial infarction, target vessel revascularization, and stroke (20) (Table 2). All these adverse events will be recorded in detail during in-hospital period.

TABLE 2

Table 2. Outcome definitions in MACE.

Statistical Analysis

Sample Size Consideration

A rule of thumb, namely, that the event per variable (EPV) was 10 or greater under this circumstance was applied for the sample size determination. Given no more than 8 risk factors in developing a predictive model, thus at least 80 infections were needed. According to the previous studies (3), the incidence of infection was 10%, and at least 800 patients should be finally included.

Data Analysis

The student's t-test or Wilcoxon rank-sum test was used to compare the continuous variables and the chi-square test or Fisher exact test was used as appropriate for categorical variables for the comparison of patients with or without infections in both datasets.

The risk score was developed using a backward stepwise logistic regression model. All candidate predictors were included in the development of the risk score, and to avoid overfitting data and select the best subset of risk factors, a bootstrap method was used. For missing data, 10 multiple imputations were performed using the SAS MI procedure, and 100 bootstrap repeats were conducted for each imputation. Variables selected in at least 90% of 1,000 bootstrap repeats were included in the final model. The risk score was developed for each imputed data via using the selected variables and the SAS MIANALYZE procedure was used to combine the results of the analyses. For this risk score, the scoring method was analogous to that of Sullivan et al. (21) and was used based on the developed risk score. For the scoring purpose, continuous variables were categorized into groups regarding the clinical significance. The risk score of post-AMI consisted of four levels (low-risk, moderate-risk, high-risk, and very high-risk) in order to enhance the clinical utility.

Both discriminatory accuracies assessed by the C-statistic and calibration measured by the Hosmer-Lemeshow χ² statistic and calibration plot were used to evaluate the predictive accuracy of the risk score. The bootstrap method with 1,000 bootstrapped replications was used to perform an internal cross-validation of the risk score. The average of the C-statistic was reported. In addition, external validation and subgroup validation (older, gender, diabetes, anemia, and chronic kidney disease) of the risk score model was conducted to assess the stability of the model. The risk score was also validated in patients with NSTE-ACS undergoing PCI, and the area under curve (AUC) of this risk score was compared to the previous scoring systems (6, 7). Furthermore, comparison of the area under the receiver operating characteristic (ROC) curve was according to the nonparametric approach of DeLong et al. (22). To evaluate the clinical utility of this risk score, decision curve analysis was introduced and the risk score with a higher net benefit indicated a better clinical effect (23).

All statistical analyses were performed using SAS (version 9.4, SAS Institute, 210 Cary, North Carolina, USA).

Discussion

The results of this study will develop and validate a relatively simple risk score based on easily acquired clinical variables to identify the patients with STEMI undergoing PCI at high-risk of post-AMI infection. In addition, the predictive accuracy of this risk score will be validated in several subgroups and an external validation cohort including patients with NSTE-ACS.

Infection is known to have unacceptably high and alterable mortality. Thus, early risk stratification of patients is a crucial clinical task. Some studies have reported the risk scores or markers to assess the probability of infection in patients with cardiovascular diseases or in critically ill patients. Fowler et al. developed a simplified risk score with twelve variables to identify high-risk for major infection in patients undergoing cardiac surgery (9). The variables included in this risk score were age, body mass index (BMI), diabetes, renal failure, congestive heart failure, peripheral vascular disease, female gender, chronic lung disease, cardiogenic shock, myocardial infarction, concomitant surgery and intraoperative variables including perfusion time and intra-aortic balloon pump. The model that limited to preoperative characteristics and combined model including both preoperative and intraoperative characteristics achieved a C-index of 0.697 and 0.708, respectively. Therefore, the discriminatory accuracies of these prediction models were relatively low, and the model only reflected the overall clinical objective due to broader definitions (9). Raja SG et al. reported that the Brompton Harefield Infection Score (BHIS) could effectively predict surgical site infection (SSI) risks and contribute to risk stratification in patients following coronary artery bypass graft (CABG) (24). The baseline risk score consisting of female, diabetes or HbA1c >7.5%, BMI ≥ 35, LVEF <45% and emergency surgery offered an effective predictive ability (ROC curve was 0.727). However, this model had several limitations, the relatively small number of SSI events (n=96) limited the ability to identify associations with a large number of variables. Additionally, the accuracy and utility of the BHIS score tool were only validated internally but not in an external dataset (24). Furthermore, several other risk score models, such as the intensive care infection score and the clinical pulmonary infection score were established to assess the infection risks in critically ill patients (25, 26). However, their practical utility was limited because of the lower predictive value (25) and the circumscribed application to pulmonary infection (26). Up to now, few studies have proposed an infection prediction model for patients with STEMI before our recent researches, and there is no scoring system established specifically to assess patients' individual risk of infection among STEMI following PCI. Therefore, this study aimed to develop and validate a novel risk score based on risk factors for early prediction of infection in these patients.

This observational study protocol has several limitations. First, this risk score was exclusive to STEMI patients and validated in patients with NSTE-ACS, clinicians should take special cautions when applying these results to other patients, such as stable coronary artery disease. Second, infection was difficult to diagnose to some extent and would be overestimated sometimes. However, the infection in this study was those treated with antibiotics which were strictly confirmed and approved by the hospital infection control committee. Third, the candidate risk variables are limited to those available in the patient cohort of this study. The more variables the scoring system contains, the higher the predictive accuracy of the risk score achieved. Finally, the current risk score did not include any new biomarkers. Incorporating new biomarkers may create a more robust model. However, this risk score was derived from clinical data routinely collected and would be more accessible in clinical practice.

In conclusion, to the extent of our knowledge, our risk score will be the first risk model for early prediction of infection in patients with STEMI undergoing PCI. This simple tool may therefore improve evaluation of post-AMI infection and enhance future researches into the best practices to prevent or reduce the infection in such patients.

Ethics Statement

The studies involving human participants were reviewed and approved by Ethics Committee of Guangdong Provincial People's Hospital. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

PH, NT, and JC: project administration and resources. PH, CD, NT, and YLiu: study design. YLiu and LW: writing-original draft. YLiu, LW, PC, and LX: writing-review and editing. LW, YLin, WC, ZX, PC, YD, LZ, and HF: data collection. All authors contributed to the article and approved the submitted version.

Funding

This study was supported by a grant from the Shuangqing talent program of Guangdong Provincial People's Hospital (No. KJ012019095 and KJ012019084), and Science and Technology Planning Project of Guangzhou City (Grant No.201906010089), and Guangdong Provincial People's Hospital Clinical Research Fund (Y012018085). The funders had no role in the study design, data collection and analysis, the decision to submit the protocol for publication. The work was not funded by any industry sponsors.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

The authors would like to thank Dr Wenfei He (Department of Cardiology, The Second People's Hospital of Nanhai District, Guangdong Provincial People's Hospital's Nanhai Hospital, Foshan, China) for collecting clinical data.

References

1. Eagle KA, Goodman SG, Avezum A, Budaj A, Sullivan CM, López-Sendón J, et al. Practice variation and missed opportunities for reperfusion in ST-segment-elevation myocardial infarction: findings from the global registry of acute coronary events (GRACE). Lancet. (2002) 359:373–7. doi: 10.1016/S0140-6736(02)07595-5

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Truffa AA, Granger CB, White KR, Newby LK, Mehta RH, Hochman JS, et al. Serious infection after acute myocardial infarction: incidence, clinical features, and outcomes. JACC Cardiovasc Interv. (2012) 5:769–76. doi: 10.1016/j.jcin.2012.03.018

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Putot A, Chague F, Manckoundia P, Cottin Y, Zeller M. Post-infectious myocardial infarction: new insights for improved screening. J Clin Med. (2019) 8:827. doi: 10.3390/jcm8060827

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Eagle KA, Lim MJ, Dabbous OH, Pieper KS, Goldberg RJ, Van de Werf F, et al. A validated prediction model for all forms of acute coronary syndrome: estimating the risk of 6-month postdischarge death in an international registry. JAMA. (2004) 291:2727–33. doi: 10.1001/jama.291.22.2727

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Morrow DA, Antman EM, Charlesworth A, Cairns R, Murphy SA, de Lemos JA, et al. TIMI risk score for ST-elevation myocardial infarction: a convenient, bedside, clinical score for risk assessment at presentation: an intravenous nPA for treatment of infarcting myocardium early II trial substudy. Circulation. (2000) 102:2031–7. doi: 10.1161/01.CIR.102.17.2031

CrossRef Full Text | Google Scholar

6. Wang L, Huang G, Peng Q, Duan C, Dai Y, Shao S, et al. Risk predictive ability of ACEF score for infection in patients with ST-segment elevation myocardial infarction undergoing percutaneous coronary intervention. Eur J Prev Cardiol. (2020) 27:220–2. doi: 10.1177/2047487319873142

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Liu Y, Dai Y, Chen J, Huang C, Duan C, Shao S, et al. Predictive value of the Canada acute coronary syndrome risk score for post-acute myocardial infarction infection. Eur J Intern Med. (2020) 71:57–61. doi: 10.1016/j.ejim.2019.10.012

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Kansy A, Jacobs JP, Pastuszko A, Mirkowicz-Małek M, Manowska M, Jezierska E, et al. Major infection after pediatric cardiac surgery: external validation of risk estimation model. Ann Thorac Surg. (2012) 94:2091–5. doi: 10.1016/j.athoracsur.2012.07.079

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Fowler VG Jr, O'Brien SM, Muhlbaier LH, Corey GR, Ferguson TB, Peterson ED. Clinical predictors of major infections after cardiac surgery. Circulation. (2005) 112:I358–65. doi: 10.1161/CIRCULATIONAHA.104.525790

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Hemingway H, Croft P, Perel P, Hayden JA, Abrams K, Timmis A, et al. Prognosis research strategy (PROGRESS) 1: a framework for researching clinical outcomes. BMJ. (2013) 346:e5595. doi: 10.1136/bmj.e5595

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Riley RD, Hayden JA, Steyerberg EW, Moons KGM, Abrams K, Kyzas PA, et al. Prognosis Research Strategy (PROGRESS) 2: prognostic factor research. PLoS Med. (2013) 10:e1001380. doi: 10.1371/journal.pmed.1001380

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Steyerberg EW, Moons KGM, van der Windt DA, Hayden JA, Perel P, Schroter S, et al. Prognosis research strategy (PROGRESS) 3: prognostic model research. PLoS Med. (2013) 10:e1001381. doi: 10.1371/journal.pmed.1001381

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Collins GS, Reitsma JB, Altman DG, Moons KG. Transparent reportingof a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. Br J Surg. (2015) 102:148–58. doi: 10.1002/bjs.9736

CrossRef Full Text | Google Scholar

14. Moons KG, Altman DG, Reitsma JB, Ioannidis JP, Macaskill P, Steyerberg EW, et al. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): explanation and elaboration. Ann Intern Med. (2015) 162:W1–73. doi: 10.7326/M14-0698

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Ibanez B, James S, Agewall S, Antunes MJ, Bucciarelli-Ducci C, Bueno H, et al. 2017 ESC Guidelines for the management of acute myocardial infarction in patients presenting with ST-segment elevation: the Task Force for the management of acute myocardial infarction in patients presenting with ST-segment elevation of the European society of cardiology (ESC). Eur Heart J. (2018) 39:119–77. doi: 10.1093/eurheartj/ehx393

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Chen JY, He PC, Liu YH, Wei XB, Jiang L, Guo W, et al. Association of parenteral anticoagulation therapy with outcomes in chinese patients undergoing percutaneous coronary intervention for non-ST-segment elevation acute coronary syndrome. JAMA Intern Med. (2019) 179:186–94. doi: 10.1001/jamainternmed.2018.5953

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Ma YC, Zuo L, Chen JH, Luo Q, Yu XQ, Li Y, et al. Modified glomerular filtration rate estimating equation for Chinese patients with chronic kidney disease. J Am Soc Nephrol. (2006) 17:2937–44. doi: 10.1681/ASN.2006040368

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Thiele H, Kurz T, Feistritzer HJ, Stachel G, Hartung P, Lurz P, et al. General versus local anesthesia with conscious sedation in transcatheter aortic valve implantation: the randomized SOLVE-TAVI Trial Statistical Analysis. Circulation. (2020) 142:1437–47. doi: 10.1161/CIRCULATIONAHA.120.046451

CrossRef Full Text | Google Scholar

19. Piccaro de Oliveira P, Gonzales V, Lopes RD, Schmidt MM, Garofallo S, Santos RP, et al. Serious infections among unselected patients with ST-elevation myocardial infarction treated with contemporary primary percutaneous coronary intervention. Am Heart J. (2016) 181:52–9. doi: 10.1016/j.ahj.2016.08.005

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Mendis S, Thygesen K, Kuulasmaa K, Giampaoli S, Mähönen M, Ngu Blackett K, et al. World health organization definition of myocardial infarction: 2008-09 revision. Int J Epidemiol. (2011) 40:139–46. doi: 10.1093/ije/dyq165

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Sullivan LM, Massaro JM, D'Agostino RB Sr. Presentation of multivariate data for clinical use: the framingham study risk score functions. Stat Med. (2004) 23:1631–60. doi: 10.1002/sim.1742

PubMed Abstract | CrossRef Full Text | Google Scholar

22. DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. (1988) 44:837–45. doi: 10.2307/2531595

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Making. (2006) 26:565–74. doi: 10.1177/0272989X06295361

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Raja SG, Rochon M, Jarman JWE. Brompton harefield infection score (BHIS): development and validation of a stratification tool for predicting risk of surgical site infection after coronary artery bypass grafting. Int J Surg. (2015) 16:69–73. doi: 10.1016/j.ijsu.2015.02.008

PubMed Abstract | CrossRef Full Text | Google Scholar

25. van der Geest PJ, Mohseni M, Linssen J, Duran S, de Jonge R, Groeneveld AB. The intensive care infection score- a novel marker for the prediction of infection and its severity. Crit Care. (2016) 20:180. doi: 10.1186/s13054-016-1366-6

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Zilberberg MD, Shorr AF. Ventilator-associated pneumonia: the clinical pulmonary infection score as a surrogate for diagnostics and outcome. Clin Infect Dis. (2010) 51(Suppl. 1):S131–5. doi: 10.1086/653062

PubMed Abstract | CrossRef Full Text | Google Scholar