Development and validation of a sepsis diagnostic scoring model for neonates with suspected sepsis

Background: We aimed to develop and validate a diagnostic model for sepsis among neonates evaluated for suspected sepsis, by incorporating thromboelastometry parameters, maternal/neonatal risk factors, clinical signs/symptoms and laboratory results.

Methods: This retrospective cohort study included 291 neonates with presumed sepsis, hospitalized in a NICU, from 07/2014 to 07/2021. Laboratory tests were obtained on disease onset and prior to initiating antibiotic therapy. Τhromboelastometry extrinsically activated (EXTEM) assay was performed simultaneously and Tοllner and nSOFA scores were calculated. Sepsis diagnosis was the outcome variable. A 10-fold cross-validation least absolute shrinkage and selection operator logit regression procedure was applied to derive the final multivariable score. Clinical utility was evaluated by decision curve analysis.

Results: Gestational age, CRP, considerable skin discoloration, liver enlargement, neutrophil left shift, and EXTEM A10, were identified as the strongest predictors and included in the Neonatal Sepsis Diagnostic (NeoSeD) model. NeoSeD score demonstrated excellent discrimination capacity for sepsis and septic shock with an AUC: 0.918 (95% CI, 0.884–0.952) and 0.974 (95% CI, 0.958–0.989) respectively, which was significantly higher compared to Töllner and nSOFA scores.

Conclusions: The NeoSeD score is simple, accurate, practical, and may contribute to a timely diagnosis of sepsis in neonates with suspected sepsis. External validation in multinational cohorts is necessary before clinical application.

Introduction

Sepsis is a leading cause of morbidity and mortality in neonates, with an estimated worldwide incidence of approximately 22 neonates per 1,000 livebirths and a mortality rate of 11%–19%, depending on the study population and the definition of cases (1). Sepsis is the third most common cause of death during the neonatal period and accounts for one fourth of all neonatal deaths (2).

Early diagnosis of neonatal sepsis is crucial and remains a major challenge for the neonatologists; yet a single respective definition does not exist. Determination of diagnostic criteria for neonatal septicemia is of great importance, as the delay in management of severe bacterial infections could negatively affect the clinical outcome of neonates. The clinical signs of neonatal sepsis are non-specific, especially at early stages, and they are often attributed to non-infectious conditions. As early diagnosis of neonatal sepsis is difficult, due to non-specific symptoms and laboratory tests with limited diagnostic value, empirical and non-judicious use of antibiotics is common practice in Neonatal Intensive Care Units (NICUs) (3). Thus, the development and validation of diagnostic scores for neonatal sepsis is essential.

The course of septic neonates is dramatically aggravated and their prognosis is worsened by the presence of Disseminated Intravascular Coagulation (DIC) or Multi-Organ Dysfunction Syndrome (MODS) (4). Sepsis is frequently complicated with hemostatic derangement due to interaction of coagulation components with inflammatory mediators (5, 6).

Understanding of the multifactorial mechanism of coagulation is considered as a prerequisite for diagnosis of hemostatic disorders, especially in neonates. Rotational thromboelastography (TEG) and thromboelastometry (ROTEM), as point-of-care tests, evaluate global clot formation and dissolution. In adult septic patients, the diagnostic and prognostic role of TEG/ROTEM has been extensively investigated, suggesting that these tests may facilitate diagnosis of clotting abnormalities, and additionally, improve prognosis of sepsis (9). Limited data are available on the role of TEG/ROTEM variables as indicators of neonatal sepsis (7, 8, 10–13).

Diagnosis of sepsis is crucial, yet challenging among ill neonates, as differential diagnosis is extremely complicated in this subpopulation. On the other hand, overdiagnosis of sepsis leads to unnecessary and potentially harmful therapeutic interventions in neonates. To date, no reliable or validated diagnostic score for neonatal sepsis has been established in clinical practice (14, 15). Our aim was to develop and validate a sepsis diagnostic model for hospitalized in NICU neonates undergoing evaluation for sepsis, examining ROTEM parameters in different combinations with maternal/neonatal risk factors, inflammatory response parameters, laboratory tests and physical examination findings.

Materials and methods

Participants

This single-center, observational study includes full-term and pre-term neonates with sepsis or suspected sepsis, hospitalized in the NICU of General Hospital of Nikea, Piraeus, Greece, between July 2014 and July 2021. Our hospital is a referral center, and a large proportion of patients are immigrants and low-income native population, with no antenatal care in most cases. The study protocol was designed and conducted in compliance with the Declaration of Helsinki, was approved by the Institutional Review Board of Nikea General Hospital (15 July 2014, 32/3) and all methods were performed in accordance with the relevant guidelines and regulations. Written informed consent has been obtained by parents or guardians of enrolled neonates.

Definitions

Early onset sepsis (EOS) was defined as sepsis occurring within the first 72 h of life.

Late onset sepsis (LOS) was defined as sepsis occurring at or after 72 h of life.

Sepsis was defined as the clinical condition characterized by isolation of bacteria or fungi in a blood and/or cerebrospinal fluid (CSF) culture. Neonates >72 h of life with clinical signs of sepsis and a positive urine culture were also considered as septic (15). For diagnosis of Coagulase-negative staphylococci (CoNS) sepsis, 2 consecutive positive blood cultures with the same CoNS bacteria or 1 positive blood culture and simultaneous presence of 2 clinical signs or laboratory data, were required (15).

Neonatal septic shock was defined using the consensus criteria for pediatric sepsis (16, 17), and MODS was defined as concurrent dysfunction of ≥2 systems (18).

Inclusion criteria

The presence of clinical signs and/or laboratory data suggestive of sepsis necessitating initiation of antibiotic therapy (15, 17): detailed relevant information provided in the Supplementary data. Absence of exposure to antibiotics for at least 48 h was required for inclusion in the study.

Exclusion criteria

Neonates with perinatal asphyxia, major congenital malformation or those transfused with fresh frozen plasma or platelets, were excluded. Additional exclusion criteria were: difficulties during blood sampling, known history of maternal acquired or congenital coagulopathy and treatment interfering with hemostasis.

Study design

The development and internal validation of the diagnostic score in this study was performed and reported in agreement with the TRIPOD guideline (19).

On the first day (day 1) of sepsis or suspected sepsis, data on demographics, maternal and pregnancy history, and medications received during pregnancy, were recorded. Prior to initiating antibiotic therapy, blood specimens for culture, routine biochemical tests, arterial blood gas, complete blood count, peripheral blood smear and CRP were obtained. Simultaneously, 300 µl of residual arterial blood was analyzed on the ROTEM analyzer (Tem Innovations GmbH, Munich, Germany) using the standard extrinsically activated (EXTEM) assay. ROTEM, which evolved from the original TEG system, is a whole blood viscoelastic, dynamic method for global evaluation of hemostasis. EXTEM assay is activated by recalcification and addition of tissue thromboplastin. It reflects the activity of the extrinsic pathway of coagulation, as generation and formation of the clot mainly depends on coagulation factors VII, X, V, II, and fibrinogen (20). The following EXTEM parameters were measured: Clotting Time (CT, seconds), defined as the time from the beginning of the test to a clot amplitude of 2 mm; Clot Formation Time (CFT, seconds), indicating time from CT until clot amplitude of 20 mm is achieved; Alpha-angle (α°), the angle between the baseline and a tangent of the clotting curve at 2 mm; Clot firmness amplitude recorded at 10, 20, and 30 min (A10, A20, and A30); Maximum Clot Firmness (MCF, mm), reflecting the final strength of the clot; Lysis Index at 60 min (LI 60%), defined as the percentage of remaining clot in relation to the MCF following the 60-min observation period after CT; and Maximum Lysis (ML), representing the percent decrease of maximal amplitude over time.

GA, birthweight, delivery mode, and potential maternal risk factors for sepsis [prolonged and/or premature rupture of membranes, fever, chorioamnionitis, positive colonization with Group B Streptococcus (GBS) or history of previous infant with GBS infection and GBS bacteriuria] were recorded.

The evaluated clinical variables included 5-minutes Apgar score, temperature instability, jaundice, cyanosis, edema or sclerema, petechiae, poor peripheral perfusion assed by capillary refill time (CRT, defined as “normal” for CRT <3 s; “impaired” for CRT 3 to 5 s; and “considerably impaired” for CRT > 5 s), change in skin color (evaluated as “normal” for bright pink or pink coloration, “moderate change” for mild or inconstant skin discoloration, “considerable change” for conspicuous and constant change of color towards green grey skin coloration), poor feeding, irritability, seizure, lethargy, gastrointestinal symptoms, vomiting, abdominal distention, hepatomegaly (palpable liver >2 cm below the right costal margin), apnea, tachypnea, need for respiratory support, grunting, tachycardia, bradycardia, mean blood pressure, hypotension, and urine output.

Laboratory data included: blood pH, base deficit, plasma lactate levels, PaO₂/FIO₂, hematocrit level, white blood cells, neutrophil count and neutrophil left shift (immature leukocytes present in peripheral blood), platelet count, CRP, serum albumin, serum glutamic oxaloacetic transaminase (SGOT), serum glutamic pyruvic transaminase (SGPT), total bilirubin, direct/indirect bilirubin, and plasma creatinine. The distribution of each candidate continuous variable was inspected. Prior to any analysis, we applied a log10-transformation to serum albumin, white blood cells, CRP, SGOT, SGPT, total bilirubin, direct bilirubin, indirect bilirubin, creatinine, as their distribution was severely right skewed.

Chest radiograph, cerebrospinal fluid culture and urine culture were performed whenever clinically indicated. In these neonates, serial blood samples for sepsis biomarkers were obtained, when deemed necessary, until antibiotic therapy was completed.

In all cases of sepsis or suspected sepsis, Tοllner and Neonatal Sequential Organ Failure Assessment (nSOFA) scores were calculated (21, 22).

Statistical analysis and model development

We presented the baseline characteristics and ROTEM parameters of the study population as means ± standard deviations (SDs), medians and interquartile ranges (IQRs), or percentages, when appropriate.

Our aim was to build a practical, multivariable diagnostic score for the prompt diagnosis of sepsis in neonates with suspected sepsis, using ROTEM parameters and clinical and biochemical characteristics.

Initially, we investigated the association between ROTEM parameters and sepsis. We assessed multicollinearity by using the non-parametric Spearman correlation test. Most parameters were very highly correlated (r_S > 0.80 and p < 0.05). To avoid substantial multicollinearity in our model development, the ROTEM parameter most associated with sepsis in a univariable logistic regression (i.e., highest likelihood ratio chi-squared test), i.e., A10 was retained as the candidate predictor (23). A complete list of all candidate predictors is available in our extended method section (Supplementary data).

The multivariable model was selected and fitted my means of L1-penalized least absolute shrinkage and selection operator (LASSO) logit regression (24–26). Model fitness was checked by using the Hosmer-Lemeshow test which is a goodness-of-fit test comparing observed and expected probabilities into quantiles of linear predictor (27). Missing data did not exceed 2% for any candidate predictor. We considered them as missing at random and we performed a complete-case analysis.

We developed a diagnostic score [Neonatal Sepsis Diagnostic (NeoSeD) score] by converting the β coefficient of each predictor in the final model into a weighted score using integer values while preserving monotonicity and simplicity. We reported the full model specifications in a table including the model intercept, as recommended by the TRIPOD guidelines (19). We produced a nomogram plot transforming all possible total point scores into the corresponding absolute risk of sepsis. Finally, we performed a decision curve analysis as recommended by Vickers et al. (28) to assess the clinical utility of the NeoSeD score by quantifying the net benefit when different threshold probabilities for a diagnosis of sepsis were considered (a guide to interpreting decision curve analysis is provided in Supplementary data Appendix S1).

Internal validity, calibration and performance measures

We selected variables entering the final model through 10-fold cross-validation (CV) LASSO logit regression, a procedure maximizing the internal validity (ie, reproducibility) of the resulting model (19).

We verified the calibration of our final model through a predicted vs. observed probability plot: the calibration belt (29). To check the presence of any difference between the mean predicted probability and the observed proportion of sepsis cases, we also computed the calibration in the large measure (i.e., it should ideally equal zero) and accompanied it with a formal statistical test under the null hypothesis of no difference (19).

The discrimination capacity of our preliminary diagnostic score was assessed through ROC analyses which allowed the calculation of the AUC (19). We tested the ability of the model to generalize to new cases by using 10-fold cross validation. We calculated the discrimination capacity of our model in each of the 10-fold cross validation subsamples and averaged it reporting the mean cross-validated AUC to provide a more realistic measure of model performance. We also calculated the brier score as a measure of diagnostic accuracy (30). The proportion of variance explained by the model on the logit scale was calculated with the McKelvey–Zavoina pseudo R², taking values from 0 (i.e., no variance explained) to 1 (i.e., the entire variance is explained) (31).

The net benefit and the area under curve of the NeoSeD score were formally compared with those of the Tollner and those of the nSOFA scores (21, 22, 32). Though the NeoSeD score was developed as a diagnostic tool for sepsis, we also tested its overall performance on septic shock (secondary outcome measure) and on blood culture-confirmed sepsis. More information regarding our methodology can be found in the Supplementary data. All tests were two-sided. The STATA (Stata Corp., College Station, Texas, United States) and R software were used for statistical modeling and analysis. A two-sided p-value <0.05 was considered statistically significant.

Results

Characteristics of study subjects

The study population consisted of 291 full-term and pre-term neonates with suspected sepsis. We report the baseline characteristics of these neonates in Supplementary Table S1. About half (N = 142; 49%) of the included newborns were preterm (<37 weeks of gestation), and one fourth (N = 82; 28%) were very preterm (<32 weeks). Half of them had a low birthweight (<2,500 grams; N = 145; 50%) and about one third had a very low birthweight (<1,500 grams; N = 99; 34%). Out of the 291 studied neonates, 119 (41%) were diagnosed with sepsis: 23 (19%) with EOS, and 96 (81%) with LOS. Among confirmed sepsis cases, the blood culture revealed the presence of Gram negative bacteria in 62 cases (51%), Gram positive spp. in 21 cases (18%), Candida in 12 cases (10%). Of the remaining sepsis cases, 10 neonates (8%) had positive CNS culture and 14 (12%) had positive urine culture. Among septic neonates 61 (51.3%) were diagnosed with septic shock and 18 (15%) died.

Model development

We sought to identify a multivariable model enabling the early diagnosis of sepsis in a population of neonates with suspected sepsis and developed and internally validated a preliminary diagnostic score using ROTEM parameters and clinical and biochemical characteristics. At the end of the selection process performed by applying 10-fold crossvalidation to a L1-penalized LASSO, 6 variables were identified and included in the final multivariable logit model used for building the Neonatal Sepsis Diagnostic (NeoSeD) Score: Clot firmness amplitude recorded at 10 min (A10, mm), gestational age (GA, weeks), C-reactive protein (CRP, log10 transformation of the value expressed in mg/L) as continuous variables, presence of considerable change in skin color, liver enlargement, and neutrofil left shift. The NeoSeD score was built on 287 neonates with suspected sepsis with complete data. Full model specifications are reported in Table 1.

TABLE 1

Table 1. Risk of sepsis in term and preterm neonates with suspected sepsis: model specifications of the final multivariable logit model selected by applying 10-fold cross-validation to a L1-penalized least absolute shrinkage and selection operator (LASSO) logit procedure.

We derived the NeoSeD score by converting the β coefficients and intercept into a weighted score. We provided detailed and easy-to-apply instructions to calculate the NeoSeD score for each patient given his/her values of relevant predictors (Table 2).

TABLE 2

Table 2. Neonatal Sepsis Diagnostic (NeoSeD) score: scoring system.

Example of the derivation of the NeoSeD score for one patient with the following characteristics

A10 = 50 mm (−10 × 50 = −500 points);

Presence of considerable change in skin color (+934 points);

Presence of liver enlargement (+476 points);

Absence of shift to the left (−383 points);

37 weeks of gestational age (−61 × 37 = −2,257 points)

100 ml/L of C-reactive protein (2 × 382 = +764 points);

Each patient starts with 1972 points

This patient score is +1,006 points which, according to the nomogram plot in Figure 3, equals to a very high risk of sepsis (>90%)

Internal validation and measures of model performance

The Hosmer-Lemeshow test indicated good model fit (p = 0.60). Internal validity was ensured by the 10-fold cross-validation procedure used during model development. The calibration belt showed optimal calibration (p = 0.95), meaning close to perfect agreement between the observed proportion of neonates with sepsis and the probability of sepsis across the entire spectrum of predicted probabilities (Figure 1).

FIGURE 1

Figure 1. The observed proportion of neonates with sepsis and the probability of sepsis across the entire spectrum of predicted probabilities.

The calibration in the large was equal to 0.00 (p = 0.99), meaning that there was no evidence of a significant difference between the mean predicted probability and the observed proportion of sepsis cases. The brier score and the McKelvey–Zavoina pseudo R² showed good predictive accuracy and a substantial proportion of the variation in the diagnosis of sepsis explained in our sample (Table 3).

TABLE 3

Table 3. Performance measures of the Neonatal Sepsis Diagnostic (NeoSeD) score in predicting sepsis (primary outcome),septic shock (secondary outcome) and blood culture-confirmed sepsis (sensitivity analysis) in the study population.

We plotted the area under the receiver operating characteristic (ROC) curve of the NeoSeD score (Figure 2). We obtained an area under curve (AUC) of 0.918 (95% CI, 0.884–0.952), indicating excellent discrimination capacity.

FIGURE 2

Figure 2. Area under receiver-operating characteristic (ROC) curve of the Neonatal Sepsis Diagnostic (NeoSeD) score.

Although the NeoSeD score was developed for the diagnosis of sepsis (primary outcome), it also showed excellent performances regarding septic shock (secondary outcome). The NeoSeD score showed good calibration (p = 0.48, Figure 3) and extremely high discrimination capacity (AUC: 0.974; 95% CI, 0.958–0.989, Figure 4).

FIGURE 3

Figure 3. Calibration plot of the Neonatal Sepsis Diagnostic (NeoSeD) score displaying the agreement between the observed proportion of neonates with septic shock (secondary event) and its predicted probability. The red line represents perfect calibration.

FIGURE 4

Figure 4. Area under receiver-operating characteristic (ROC) curve of the Neonatal Sepsis Diagnostic (NeoSeD) score: septic shock (secondary outcome).

As a sensitivity analysis, we tested the diagnostic value of the NeoSeD score when considering as septic patients only those with a positive blood culture, and it showed good calibration (p = 0.14, Figure 5) and high discrimination capacity (AUC: 0.868; 95% CI, 0.823–0.914, Figure 6).

FIGURE 5

Figure 5. Calibration plot of the Neonatal Sepsis Diagnostic (NeoSeD) score displaying the agreement between the observed proportion of neonates with blood culture-confirmed sepsis (sensitivity analysis) and its predicted probability. The red line represents perfect calibration.

FIGURE 6

Figure 6. Area under receiver-operating characteristic (ROC) curve of the Neonatal Sepsis Diagnostic (NeoSeD) score: blood culture-confirmed sepsis (sensitivity analysis).

The average cross-validated AUCs for sepsis, septic shock, and blood culture-confirmed sepsis showed a high discrimination capacity (Table 3).

The discrimination capacity of the NeoSeD score was significantly better (ie, higher AUC) than that of the Tollner score in our study population for both sepsis (AUC_Tollner for sepsis: 0.843; 95% CI, 0.793–0.893; p for the difference <0.001) and septic shock (AUC_Tollner for septic shock: 0.948; 95% CI, 0.912–0.984; p for the difference = 0.043). Additionally, the discrimination capacity of the NeoSeD score was higher than that of the Neonatal Sequential Organ Failure Assessment (nSOFA) for both sepsis (AUC_nSOFA for sepsis: 0.760; 95% CI, 0.703–0.817; p for the difference <0.001) and septic shock (AUC_nSOFA for septic shock: 0.912; 95% CI, 0.876–0.947; p for the difference <0.001).

Predicted probability of sepsis and septic shock and clinical utility

The nomogram plot displaying the predicted probability of sepsis and septic shock according to the NeoSeD score is presented in Figure 7. We examined the relationship between a range of threshold probabilities for predicting sepsis and the relative value of false-positive and false-negative results (i.e., the net benefit) by using decision curve analysis (28) (a guide on how to interpret decision-curve analysis is provided in Supplementary data-Appendix S1). When compared with the strategy of considering as septic all patients with a suspect of sepsis−and treating them accordingly−the ability of the NeoSeD score to predict sepsis provided a better net benefit and a larger net reduction in unnecessary interventions compared to a “treat all” strategy at most threshold probabilities (Figure 8). Importantly, the NeoSeD score was never worse than “treat all” and “treat none” strategies at any threshold probability. The NeoSeD score provided a better net benefit at all decision thresholds for sepsis and blood culture-confirmed sepsis, as compared with the predicted probabilities provided by applying the Tollner and the nSOFA scores to our cohort. In conclusion, the NeoSeD score provided the largest overall net benefit among all other strategies plotted in decision-curve analyses and the largest net reduction in unnecessary interventions (Figures 8, 9).

FIGURE 7

Figure 7. The predicted probability of sepsis and septic shock according to the Neonatal Sepsis Diagnostic (NeoSeD) score.

FIGURE 8

Figure 8. Decision curve analysis for the early detection (before the results of blood culture) of sepsis in critically-ill neonates admitted to the neonatal intensive care unit for the newly developed Neonatal Sepsis Diagnostic (NeoSed) score (red line), the Tollner score (orange line), and the neonatal Sequential Organ Failure Assessment (nSOFA, green line). (Panel A) The net benefit (y axis) is a metric representing the benefit of a certain intervention minus its harms, multiplied by an exchange rate. The unit of net benefit is true positives. A net benefit of 0.10, for instance, means “10 true positives for every 100 patients in the target population.” The net benefit is plotted over a range of possible decision thresholds/exchange rates (i.e., individual predicted probabilities given in Figure 3) for the early diagnosis of sepsis (x-axis). The net benefit also incorporates any consequence (i.e., clinical actions) of a diagnosis of sepsis. The net benefit of five different strategies is compared. The two extreme—default—strategies are “treat all” (diagonal dashed line) and “treat none” (orizontal dashed line) meaning enacting clinical actions as if all patients with suspected sepsis were septic (i.e., “treat all”), or as if they were not (i.e., “treat none”). The x-axis can be also renamed as preference: clinicians more worried about the harms of a missed diagnosis of sepsis will adopt thresholds closer to a predicted probability of zero (i.e., left side of the graph), while clinicians more worried about the harms of unnecessary interventions (i.e., on false positives) will adopt higher thresholds (i.e., right side of the graph). The x-axis is also called “exchange rate” which is an odds and represents how many false positives are worth one true positive (i.e., adopting a threshold probability of 10% means that a patient with a predicted probability over 10% will be considered septic—and treated accordingly—and by adopting this classification rule/threshold, one accepts that one true positive is worth nine false positives). Interventions associated with different harms may need the adoption of different thresholds/exchange rates for their applications. The strategy corresponding to the highest net benefit over the largest range of threshold probabilities, should be considered as the best one to adopt. (Panel B) In the second graph, net benefit is expressed in terms of “net reduction in unnecessary interventions” in respect to a strategy to “treat all” (i.e., this may be the default strategy for intravenous antibiotics in these patients). This graph is useful to understand whether using the NeoSeD score to identify patients with and without sepsis would help reduce unnecessary empiric antibiotic treatments. For example, in the current study population at a threshold of 2%, the NeoSeD score provides a net reduction of 10 per 100 patients. This should be interpreted as a net reduction of 10% in the number of unnecessary antibiotic treatments, without having missed any true septic patient. Panel A and B are derived from the same decision curve analysis. More complete guidance regarding interpreting decision-curve analysisis provided in Supplementary data Appendix S1.

FIGURE 9

Figure 9. Decision curve analysis for the early detection of blood culture-confirmed sepsis in critically-ill neonates admitted to the neonatal intensive care unit for the newly developed Neonatal Sepsis Diagnostic (NeoSed) score (red line), the Tollner score (orange line), and the neonatal Sequential Organ Failure Assessment (nSOFA, green line). The net benefit (y axis) is a metric representing the benefit of a certain intervention minus its harms, multiplied by an exchange rate. The unit of net benefit is true positives. A net benefit of 0.10, for instance, means “10 true positives for every 100 patients in the target population.” The net benefit is plotted over a range of possible decision thresholds/exchange rates (i.e. individual predicted probabilities given in Figure 3) for the early diagnosis of sepsis (x-axis). The net benefit also incorporates any consequence (i.e., clinical actions) of a diagnosis of sepsis. The net benefit of five different strategies is compared. The two extreme−default−strategies are “treat all” (diagonal dashed line) and “treat none” (orizontal dashed line) meaning enacting clinical actions as if all patients with suspected sepsis were septic (i.e., “treat all”), or as if they were not (i.e., “treat none”). The x-axis can be also renamed as preference: clinicians more worried about the harms of a missed diagnosis of sepsis will adopt thresholds closer to a predicted probability of zero (i.e., left side of the graph), while clinicians more worried about the harms of unnecessary interventions (i.e., on false positives) will adopt higher thresholds (i.e., right side of the graph). The x-axis is also called “exchange rate” which is an odds and represents how many false positives are worth one true positive (i.e., adopting a threshold probability of 10% means that a patient with a predicted probability over 10% will be considered septic—and treated accordingly—and by adopting this classification rule/threshold, one accepts that one true positive is worth nine false positives). Interventions associated with different harms may need the adoption of different thresholds/exchange rates for their applications. The strategy corresponding to the highest net benefit over the largest range of threshold probabilities, should be considered as the best one to adopt. For a more complete guidance regarding interpreting decision-curve analysis, please see Supplementary Appendix S1.

Discussion

We developed and internally validated a multivariable, practical diagnostic score for sepsis in ill neonates showing remarkable performances in the development cohort. The NeoSeD score showed a larger net benefit and better discrimination capacity in detecting neonatal sepsis and septic shock than the Tollner and nSOFA scores in our population of critically-ill patients admitted to NICU with suspected sepsis. A10, GA, CRP, the presence of considerable change in skin color, liver enlargement, and neutrofil left shift were identified as the most robust predictors and were included in the NeoSeD score. Importantly, the score retained good calibration and performance measures and offered the best net benefit compared to other available strategies for identification of neonates with confirmed sepsis.

Despite the advances in neonatal care, neonatal sepsis remains a major cause of morbidity and mortality worldwide, reflecting the increased survival rate of extremely preterm neonates with prolonged hospitalization, and the growing antibiotic resistance of bacteria (33, 34).

Sepsis in neonatal population, and especially in very preterm neonates, is not only a life-threatening situation but also a potential precursor for longterm morbidity and neurodevelopmental deficiencies. Various short-term and long-term complications of sepsis in the neonatal population have been recorded. Especially preterm neonates are at increased risk of severe adverse outcomes due to immaturity of their immune mechanisms and multiple invasive interventions. GA and birth weight are inversely and significantly correlated with the occurrence of sepsis (35–38). This correlation was corroborated by our results, as GA was found to be one of the predicting variables for sepsis in our model. Concomitant conditions, which are common in preterm neonates, including patent ductus arteriosus, respiratory distress syndrome, and necrotizing enterocolitis, may also increase the risk of neonatal sepsis. This is mainly due to more invasive therapeutic interventions (mechanical ventilation, central or peripheral vascular access, and prolonged parenteral nutrition) required in this patient subpopulation (37, 39).

The skin reflects early changes in circulation and perfusion attributed to systemic septic reaction (40). Alteration in skin color, a prolonged capillary refill time, and temperature instability, all markers of impaired peripheral vascular reactivity, are evident in septic neonates often before traditional indexes of sepsis (41). Skin color changes demonstrated predictive value in our model. Several biomarkers, including complete blood count components and peripheral smear, CRP, procalcitonin, interleukines, cytokines and chemokines, have been investigated for diagnosis and severity assessment of septic neonates (42). A parameter used in neonatal sepsis evaluation is the neutrophil left shift, a rapid and simple test with considerable positive predictive value and specificity (43, 44). CRP is a useful biomarker with a sensitivity ranging between 9% and 83%, higher specificity (>90%), and negative predictive value in serial measurements. In addition, monitoring CRP levels has been used to evaluate response to treatment (45, 46). Its combined use with other biomarkers has been suggested (46). Left shift of neutrophils and CRP levels emerged as predicting variables in our model. Multi-organ dysfunction often complicates the course of neonatal sepsis. In this context, palpable liver may be a sign of cardiac dysfunction or cardiogenic shock (4). Liver enlargement, observed in neonatal sepsis, also presented as a predictive variable in our model.

Severe sepsis is often complicated by systemic activation of coagulation cascade (4). Hemostasis and inflammation are significantly correlated. Activation of coagulation mechanism and impairment of anticoagulant pathways are both triggered by pro-inflammatory cells, cytokines and chemokines. Endothelial plasminogen activator inhibitor 1 (PAI-1) is induced by cytokine production, leading to inactivation of fibrinolytic system and a subsequent fibrin microvascular deposition. This is the fundamental pathophysiological process of DIC, which results in organ dysfunction. Consumption of coagulation factors and platelets, as well as reduced platelet functionality in septic patients is associated with increased risk of bleeding (4). Prompt diagnosis and management of hemostatic disorders in neonates with sepsis is essential for optimizing care of these patients (47). Conventional coagulation tests, including prothrombin time (PT) and activated partial thromboplastin time (aPTT), and platelet count, present limitations in the evaluation of the global hemostatic profile (48). However, viscoelastic tests assess the overall dynamics of clot formation and lysis, evaluating all stages of the coagulation process (48). Among the ROTEM parameters, EXTEM A10 emerged as the most potent variable in our model for sepsis diagnosis in neonates. This is a parameter which encloses platelet count, platelet function, fibrinogen concentration, factor XIII contribution in clot amplitude, and fibrin polymerization (49). This finding is in line with existing evidence of a hypocoagulable profile, reflected by reduced EXTEM A10, in septic neonates with hemorrhagic diathesis (11). Furthermore, EXTEM A10 has been independently and significantly associated with increased risk of mortality in critically ill neonates (50), while incorporating this variable in selected well- established neonatal disease severity scoring systems may improve their predictive performance for mortality (51). Finally, EXTEM A10 is included as a variable in the NeoBRis score for prediction of 24-hour bleeding risk in critically ill neonates (23, 52).

Sepsis complications could be devastating and life-threatening in neonates. A positive blood culture is the gold standard for diagnosis of sepsis. However, this is not feasible until 24–48 h following clinical presentation (53), while blood culture-negative cases account for a significant percentage of septic neonates (54). In addition, initial signs and symptoms of sepsis are subtle and often non-specific in neonates, and there is no established uniform definition for sepsis in this population (15). Thus, treatment with antibiotics is usually initiated early in the course of presumed sepsis, and as a consequence, a significant proportion of neonates receives antibiotics unnecessarily (15, 33). This practice is largely responsible for a high rate of antibiotic resistance in NICUs, which ultimately further aggravates the outcome for septic neonates. It has been estimated that 31% of neonatal deaths related to sepsis can be attributed to antimicrobial resistance, rendering it a major issue for global healthcare systems (55). Antimicrobial stewardship programs are necessary for avoidance of antibiotics overuse, while prompt management of suspected sepsis is also paramount (56). In this direction, several attempts have been made over the last few years to develop sepsis scores in neonatal population, either for EOS or for LOS (57–61). A free, online neonatal EOS-risk calculator (https://neonatalsepsiscalculator.kaiserpermanente.org) was developed for prediction of EOS incidence and for guiding decisions on treatment with antibiotics (62). Widespread application of this calculator has reduced antibiotics administration without a compromise in safety, especially in populations with relatively low rate of culture-proven EOS and uninhibited access to follow-up care (46). However, as half of the patients with culture-proven EOS are asymptomatic at birth, close monitoring and clinical observation is required to ensure safety. Critical limitations of this scoring system include the implementation only for EOS and in neonates ≥34 weeks GA. In 1982, Töllner et al. (21), developed a score system based on 7 clinical parameters (skin color, capillary refill time, muscular hypotonia, apnea, respiratory distress, hepatomegaly, gastrointestinal symptoms) and 3 hematologic parameters (WBC count, neutrophil left shift and thrombocytopenia). The discrimination capacity of the NeoSeD score was significantly higher compared to the Töllner score in our study population for both sepsis (p < 0.001) and septic shock (p = 0.043). This superiority could be due to a more comprehensive evaluation of coagulation status and the adoption of prematurity as a risk factor for neonatal sepsis. The NeoSeD score was able to predict the entire spectrum of individual probability of sepsis and sepsis shock, and showed optimal calibration of probabilities from the lowest to the highest-risk neonates. Although we performed extensive internal validation, we must recognize that without an external validation, model performance may be optimistic (63). In adults with suspected infection, SOFA score is used to estimate risk of Intensive Care Unit admission or mortality. The nSOFA, an adaptation of SOFA for neonates, was developed as an assessment tool for prediction of mortality risk of LOS in very low birth weight infants (64). Further to its notable performance, nSOFA was suggested to help build a consensus definition of sepsis in this population (22). In the present study, the discrimination capacity of NeoSeD score was significantly higher than that of nSOFA for both sepsis and septic shock. This could possibly be attributed to the fact that nSOFA score was designed for prediction of sepsis outcome, while NeoSeD score was specifically developed for the diagnosis of sepsis.

Although NeoSeD score was primarily designed to identify neonates with sepsis, it also demonstrated excellent performance for the detection of septic shock. As the presentation of sepsis is non-specific in neonates, advancement to septic shock is a possible, early complication, with an adverse impact on prognosis. Consequently, time is of the utmost essence for the reversion of neonatal shock.

Prompt evaluation and identification of sepsis symptoms and signs is crucial for the early and appropriate management of the neonate. To date, there is no single, optimal biomarker to diagnose sepsis, therefore, treatment decisions should be based on a combined approach taking into consideration both risk factors and neonatal clinical status. Among our critically ill neonates evaluated for suspected sepsis, the prevalence of confirmed sepsis was 41%. This finding is in agreement with previously reported rates, highlighting a high prevalence of non-septic neonates who receive antibiotics (65, 66). A scoring system with optimal performance for diagnosis of sepsis in ill neonates assessed for sepsis would be ideal. In the process of our model development, the most frequently used in clinical practice and existing scoring systems risk factors, demographic and laboratory data, and clinical signs, were assessed. Sepsis diagnostic scores based on clinical symptoms and signs are easily accessible and extremely helpful in limited-resource settings, while scores including primarily laboratory values provide a more accurate and objective tool. However, laboratory assessment is time- consuming and requires respective equipment. Therefore, the incorporation of clinical and readily available laboratory variables in a scoring model may be optimal (67).

Finally, the NeoSeD score allows for neonatal sepsis and septic shock risk stratification, providing clinicians with a more detailed therapeutic guidance. We demonstrated the clinical utility of the NeoSeD score in our study population by applying decision curve analysis (28), an approach whose application has been highly recommended recently (68–70). Irrespective of the threshold chosen for a diagnosis of sepsis, the NeoSeD score provided the highest overall net benefit among all strategies plotted in decision curve analyses. The analyses also suggested that even at low threshold probabilities (<4%) the NeoSeD score may offer a benefit in terms of net reduction of unnecessary interventions compared to a “treat all” strategy, which is the default strategy in most NICUs. This confirmatory step justifies the potential clinical applicability of the NeoSeD score.

The relatively small sample size and the single center origin are the main limitations of this study. On the other hand, a unique center study is characterized by homogeneity of methods, practices, and recording of data. External validation through multicenter studies with larger datasets from other NICUs is necessary to evaluate the generalizability of NeoSeD model prior to its clinical application.

In conclusion, the proposed model is simple, accurate, practical, and its application may contribute to the timely diagnosis and appropriate treatment of hospitalized neonates evaluated for sepsis, especially in case of delayed or negative blood culture results. By incorporating signs and symptoms, laboratory results, and risk factors, NeoSeD score could potentially improve early identification, optimal management, and short- and long-term outcome of septic neonates.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving human participants were reviewed and approved by the Institutional Review Board of Nikea General Hospital (July 15,2014, 32/3) and all methods were performed in accordance with the relevant guidelines and regulations. Written informed consent to participate in this study was provided by the participants’ legal guardian/next of kin.

Author contributions

RS, AET and GI conceptualized the project RS, DP, GI, AET and SB designed the methodology. RS, AK, GI, AGT, SP, ML, DH, NI, AV and SK were involved in data collection. DP, SB, GI, AET, RS, AK, SP, AG and PK were involved in analysis and data interpretation. AET, RS, SP AK, SB, DP and GI wrote the manuscript. All the co-authors critically revised and finally approved the manuscript. All authors agree to be held accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. All authors contributed to the article and approved the submitted version.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fped.2022.1004727/full#supplementary-material.

References

1. Fleischmann-Struzek C, Goldfarb DM, Schlattmann P, Schlapbach LJ, Reinhart K, Kissoon N. The global burden of paediatric and neonatal sepsis: a systematic review. Lancet Respir Med. (2018) 6(3):223–30. doi: 10.1016/S2213-2600(18)30063-8

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Seale AC, Mwaniki M, Newton CR, Berkley JA. Maternal and early onset neonatal bacterial sepsis: burden and strategies for prevention in sub-Saharan Africa. Lancet Infect Dis. (2009) 9(7):428–38. doi: 10.1016/S1473-3099(09)70172-0

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Zea-Vera A, Ochoa TJ. Challenges in the diagnosis and management of neonatal sepsis. J Trop Pediatr. (2015) 61(1):1–13. doi: 10.1093/tropej/fmu079

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Wynn JL, Wong HR. Pathophysiology and treatment of septic shock in neonates. Clin Perinatol. (2010) 37(2):439–79. doi: 10.1016/j.clp.2010.04.002

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Zeerleder S, Hack CE, Wuillemin WA. Disseminated intravascular coagulation in sepsis. Chest. (2005) 128(4):2864–75. doi: 10.1378/chest.128.4.2864

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Veldman A, Fischer D, Nold MF, Wong FY. Disseminated intravascular coagulation in term and preterm neonates. Semin Thromb Hemost. (2010) 36(4):419–28. doi: 10.1055/s-0030-1254050

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Sivula M, Pettilä V, Niemi TT, Varpula M, Kuitunen AH. Thromboelastometry in patients with severe sepsis and disseminated intravascular coagulation. Blood Coagul Fibrinolysis. (2009) 20(6):419–26. doi: 10.1097/MBC.0b013e32832a76e1

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Cortegiani A, Marino L, Montalto F, Milana M, Di Benedetto A, Vento L, et al. Use of thromboelastography in severe sepsis: a case-control study. Crit Care. (2011) 15(Suppl 1):P444-P. doi: 10.1186/cc9864

CrossRef Full Text | Google Scholar

9. Ostrowski SR, Windeløv NA, Ibsen M, Haase N, Perner A, Johansson PI. Consecutive thrombelastography clot strength profiles in patients with severe sepsis and their association with 28-day mortality: a prospective study. J Crit Care. (2013) 28(3):317.e1–e11. doi: 10.1016/j.jcrc.2012.09.003

CrossRef Full Text | Google Scholar

10. Lampridou M, Sokou R, Tsantes AG, Theodoraki M, Konstantinidi A, Ioakeimidis G, et al. ROTEM Diagnostic capacity for measuring fibrinolysis in neonatal sepsis. Thromb Res. (2020) 192:103–8. doi: 10.1016/j.thromres.2020.05.028

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Sokou R, Giallouros G, Konstantinidi A, Pantavou K, Nikolopoulos G, Bonovas S, et al. Thromboelastometry for diagnosis of neonatal sepsis-associated coagulopathy: an observational study. Eur J Pediatr. (2018) 177(3):355–62. doi: 10.1007/s00431-017-3072-z

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Grant HW, Hadley GP. Prediction of neonatal sepsis by thromboelastography. Pediatr Surg Int. (1997) 12(4):289–92. doi: 10.1007/BF01372152

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Katsaras GΝ, Sokou R, Tsantes AG, Piovani D, Bonovas S, Konstantinidi A, et al. The use of thromboelastography (TEG) and rotational thromboelastometry (ROTEM) in neonates: a systematic review. Eur J Pediatr. (2021) 180(12):3455–70. doi: 10.1007/s00431-021-04154-4

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Odabasi IO, Bulbul A. Neonatal sepsis. Sisli Etfal Hastanesi Tip Bulteni. (2020) 54(2):142–58. doi: 10.14744/SEMB.2020.00236

PubMed Abstract | CrossRef Full Text | Google Scholar

15. McGovern M, Giannoni E, Kuester H, Turner MA, van den Hoogen A, Bliss JM, et al. Challenges in developing a consensus definition of neonatal sepsis. Pediatr Res. (2020) 88(1):14–26. doi: 10.1038/s41390-020-0785-x

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Wynn JL, Wong HR, Shanley TP, Bizzarro MJ, Saiman L, Polin RA. Time for a neonatal-specific consensus definition for sepsis. Pediatr Crit Care Med. (2014) 15(6):523–8. doi: 10.1097/PCC.0000000000000157

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Levy MM, Fink MP, Marshall JC, Abraham E, Angus D, Cook D, et al. 2001 SCCM/ESICM/ACCP/ATS/SIS international sepsis definitions conference. Crit Care Med. (2003) 31(4):1250–6. doi: 10.1097/01.CCM.0000050454.01978.3B

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Aufieri R, Picone S, Paolillo P. Multiple organ failure in the newborn. J Pediatr Neonat IndMed. (2014) 3(2):e030254. doi: 10.7363/030254

CrossRef Full Text | Google Scholar

19. Moons KG, Altman DG, Reitsma JB, Ioannidis JP, Macaskill P, Steyerberg EW, et al. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): explanation and elaboration. Ann Intern Med. (2015) 162(1):W1–W73. doi: 10.7326/M14-0698

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Görlinger K, Dirkmann D, Hanke AA. Rotational thromboelastometry (ROTEM®). In: Gonzalez E, Moore HB, Moore EE, editors. Trauma induced coagulopathy. Cham: Springer International Publishing (2016). p. 267–98.

21. Töllner U. Early diagnosis of septicemia in the newborn. Eur J Pediatr. (1982) 138(4):331–7. doi: 10.1007/BF00442511

CrossRef Full Text | Google Scholar

22. Fleiss N, Coggins SA, Lewis AN, Zeigler A, Cooksey KE, Walker LA, et al. Evaluation of the neonatal sequential organ failure assessment and mortality risk in preterm infants with late-onset infection. JAMA Netw Open. (2021) 4(2):e2036518-e. doi: 10.1001/jamanetworkopen.2020.36518

CrossRef Full Text | Google Scholar

23. Sokou R, Piovani D, Konstantinidi A, Tsantes AG, Parastatidou S, Lampridou M, et al. A risk score for predicting the incidence of hemorrhage in critically ill neonates: development and validation study. Thromb Haemostasis. (2021) 121(2):131–9. doi: 10.1055/s-0040-1715832

CrossRef Full Text | Google Scholar

24. Tibshirani R. Regression shrinkage and selection via the lasso. J R Stat Soc Ser B Methodol. (1996) 58(1):267–88. doi: 10.1111/j.2517-6161.1996.tb02080.x

CrossRef Full Text | Google Scholar

25. Steyerberg E, Eijkemans M, Habbema J. Application of shrinkage techniques in logistic regression analysis: a case study. Stat Neerl. (2001) 55:76–88. doi: 10.1111/1467-9574.00157

CrossRef Full Text | Google Scholar

26. Dormann CF, Elith J, Bacher S, Buchmann C, Carl G, Carré G, et al. Collinearity: a review of methods to deal with it and a simulation study evaluating their performance. Ecography. (2013) 36(1):27–46. doi: 10.1111/j.1600-0587.2012.07348.x

CrossRef Full Text | Google Scholar

27. Hosmer DW Jr, Lemesow S, Sturdivant RX. Applied logistic regression. 3rd ed: Hoboken, NJ: John Wiley / Sons, Inc. (2013).

28. Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Making. (2006) 26(6):565–74. doi: 10.1177/0272989X06295361

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Nattino G, Lemeshow S, Phillips G, Finazzi S, Bertolini G. Assessing the calibration of dichotomous outcome models with the calibration belt. Stata J. (2017) 17(4):1003–14. doi: 10.1177/1536867X1801700414

CrossRef Full Text | Google Scholar

30. Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. (2010) 21(1):128–38. doi: 10.1097/EDE.0b013e3181c30fb2

PubMed Abstract | CrossRef Full Text | Google Scholar

31. McKelvey RD, Zavoina W. A statistical model for the analysis of ordinal level dependent variables. J Math Sociol. (1975) 4(1):103–20. doi: 10.1080/0022250X.1975.9989847

CrossRef Full Text | Google Scholar

32. Hanley JA, McNeil BJ. A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology. (1983) 148(3):839–43. doi: 10.1148/radiology.148.3.6878708

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Korang SK, Safi S, Gluud C, Lausten-Thomsen U, Jakobsen JC. Antibiotic regimens for neonatal sepsis—a protocol for a systematic review with meta-analysis. Syst Rev. (2019) 8(1):306. doi: 10.1186/s13643-019-1207-1

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Patel RM. Short- and long-term outcomes for extremely preterm infants. Am J Perinatol. (2016) 33(3):318–28. doi: 10.1055/s-0035-1571202

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Auriti C, Maccallini A, Di Liso G, Di Ciommo V, Ronchetti MP, Orzalesi M. Risk factors for nosocomial infections in a neonatal intensive-care unit. J Hosp Infect. (2003) 53(1):25–30. doi: 10.1053/jhin.2002.1341

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Brodie SB, Sands KE, Gray JE, Parker RA, Goldmann DA, Davis RB, et al. Occurrence of nosocomial bloodstream infections in six neonatal intensive care units. Pediatr Infect Dis J. (2000) 19(1):56–65. doi: 10.1097/00006454-200001000-00012

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Stoll BJ, Hansen N, Fanaroff AA, Wright LL, Carlo WA, Ehrenkranz RA, et al. Late-onset sepsis in very low birth weight neonates: the experience of the NICHD Neonatal Research Network. Pediatrics. (2002) 110(2 Pt 1):285–91. doi: 10.1542/peds.110.2.285

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Belachew A, Tewabe T. Neonatal sepsis and its association with birth weight and gestational age among admitted neonates in Ethiopia: systematic review and meta-analysis. BMC Pediatr. (2020) 20(1):55. -. doi: 10.1186/s12887-020-1949-x

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Chiang P-J, Hsu J-F, Tsai M-H, Lien R, Chiang M-C, Huang H-R, et al. The impact of patent ductus arteriosus in neonates with late onset sepsis: a retrospective matched-case control study. Pediatr Neonatol. (2012) 53(5):309–14. doi: 10.1016/j.pedneo.2012.07.006

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Hariri G, Joffre J, Leblanc G, Bonsey M, Lavillegrand J-R, Urbina T, et al. Narrative review: clinical assessment of peripheral tissue perfusion in septic shock. Ann Intensive Care. (2019) 9(1):37. doi: 10.1186/s13613-019-0511-1

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Martin H, Olander B, Norman M. Reactive hyperemia and interleukin 6, interleukin 8, and tumor necrosis factor-alpha in the diagnosis of early-onset neonatal sepsis. Pediatrics. (2001) 108(4):E61. doi: 10.1542/peds.108.4.e61

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Sharma D, Farahbakhsh N, Shastri S, Sharma P. Biomarkers for diagnosis of neonatal sepsis: a literature review. J Matern Fetal Neonatal Med. (2018) 31(12):1646–59. doi: 10.1080/14767058.2017.1322060

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Saied DA. Can we rely on the neutrophil left shift for the diagnosis of neonatal sepsis? Need for re-Evaluation. Egypt Pediatr Assoc Gaz. (2018) 66(1):22–7. doi: 10.1016/j.epag.2017.12.002

CrossRef Full Text | Google Scholar

44. Helguera-Repetto AC, Soto-Ramírez MD, Villavicencio-Carrisoza O, Yong-Mendoza S, Yong-Mendoza A, León-Juárez M, et al. Neonatal sepsis diagnosis decision-making based on artificial neural networks. Front Pediatr. (2020) 8:525. doi: 10.3389/fped.2020.00525

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Delanghe JR, Speeckaert MM. Translational research and biomarkers in neonatal sepsis. Clin Chim Acta. (2015) 451(Pt A):46–64. doi: 10.1016/j.cca.2015.01.031

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Gilfillan M, Bhandari V. Neonatal sepsis biomarkers: where are we now? Res Rep Neonatol. (2019) 9:9–20. doi: 10.2147/RRN.S163082

CrossRef Full Text | Google Scholar

47. Sokou R, Parastatidou S, Konstantinidi A, Tsantes AG, Iacovidou N, Doxani C, et al. Fresh frozen plasma transfusion in the neonatal population: a systematic review. Blood Rev. (2022):100951. doi: 10.1016/j.blre.2022.100951 [Epub ahead of print].35430103

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Konstantinidi A, Sokou R, Parastatidou S, Lampropoulou K, Katsaras G, Boutsikou T, et al. Clinical application of thromboelastography/thromboelastometry (TEG/TEM) in the neonatal population: a narrative review. Semin Thromb Hemost. (2019) 45(5):449–57. doi: 10.1055/s-0039-1692210

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Parastatidou S, Sokou R, Tsantes AG, Konstantinidi A, Lampridou M, Ioakeimidis G, et al. The role of ROTEM variables based on clot elasticity and platelet component in predicting bleeding risk in thrombocytopenic critically ill neonates. Eur J Haematol. (2021) 106(2):175–83. doi: 10.1111/ejh.13534

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Sokou R, Tsantes AG, Konstantinidi A, Ioakeimidis G, Lampridou M, Parastatidou S, et al. Rotational thromboelastometry in neonates admitted to a neonatal intensive care unit: a large cross-sectional study. Semin Thromb Hemost. (2021) 47(7):875–84. doi: 10.1055/s-0041-1729964

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Sokou R, Tritzali M, Piovani D, Konstantinidi A, Tsantes AG, Ioakeimidis G, et al. Comparative performance of four established neonatal disease scoring systems in predicting in-hospital mortality and the potential role of thromboelastometry. Diagnostics. (2021) 11(11):1955. doi: 10.3390/diagnostics11111955

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Sokou R, Piovani D, Konstantinidi A, Tsantes AG, Parastatidou S, Lampridou M, et al.. Prospective temporal validation of the neonatal bleeding risk (NeoBRis) index. Thromb Haemost. (2021) 121(9):1263–6. doi: 10.1055/a-1343-3342

PubMed Abstract | CrossRef Full Text | Google Scholar

53. Bromiker R, Elron E, Klinger G. Do neonatal infections require a positive blood culture? Am J Perinatol. (2020) 37(S 02):S18–S21. doi: 10.1055/s-0040-1714079

PubMed Abstract | CrossRef Full Text | Google Scholar

54. Klingenberg C, Kornelisse RF, Buonocore G, Maier RF, Stocker M. Culture-negative early-onset neonatal sepsis—at the crossroad between efficient sepsis care and antimicrobial stewardship. Front Pediatr. (2018) 6:285. doi: 10.3389/fped.2018.00285

PubMed Abstract | CrossRef Full Text | Google Scholar

55. Laxminarayan R, Matsoso P, Pant S, Brower C, Røttingen JA, Klugman K, et al. Access to effective antimicrobials: a worldwide challenge. Lancet. (2016) 387(10014):168–75. doi: 10.1016/S0140-6736(15)00474-2

PubMed Abstract | CrossRef Full Text | Google Scholar

56. Neal SR, Musorowegomo D, Gannon H, Cortina Borja M, Heys M, Chimhini G, et al. Clinical prediction models to diagnose neonatal sepsis: a scoping review protocol. BMJ Open. (2020) 10(8):e039712. doi: 10.1136/bmjopen-2020-039712

PubMed Abstract | CrossRef Full Text | Google Scholar

57. Husada D, Chanthavanich P, Chotigeat U, Sunttarattiwong P, Sirivichayakul C, Pengsaa K, et al. Predictive model for bacterial late-onset neonatal sepsis in a tertiary care hospital in Thailand. BMC Infect Dis. (2020) 20(1):151. doi: 10.1186/s12879-020-4875-5

PubMed Abstract | CrossRef Full Text | Google Scholar

58. Mahieu LM, De Muynck AO, De Dooy JJ, Laroche SM, Van Acker KJ. Prediction of nosocomial sepsis in neonates by means of a computer-weighted bedside scoring system (NOSEP score). Crit Care Med. (2000) 28(6):2026–33. doi: 10.1097/00003246-200006000-00058

PubMed Abstract | CrossRef Full Text | Google Scholar

59. Okascharoen C, Sirinavin S, Thakkinstian A, Kitayaporn D, Supapanachart S. A bedside prediction-scoring model for late-onset neonatal sepsis. J Perinatol. (2005) 25(12):778–83. doi: 10.1038/sj.jp.7211404

PubMed Abstract | CrossRef Full Text | Google Scholar

60. Gastmeier P, Geffers C, Schwab F, Fitzner J, Obladen M, Rüden H. Development of a surveillance system for nosocomial infections: the component for neonatal intensive care units in Germany. J Hosp Infect. (2004) 57(2):126–31. doi: 10.1016/j.jhin.2003.12.038

PubMed Abstract | CrossRef Full Text | Google Scholar

61. Rodwell RL, Leslie AL, Tudehope DI. Early diagnosis of neonatal sepsis using a hematologic scoring system. J Pediatr. (1988) 112(5):761–7. doi: 10.1016/S0022-3476(88)80699-1

PubMed Abstract | CrossRef Full Text | Google Scholar

62. Kuzniewicz MW, Walsh EM, Li S, Fischer A, Escobar GJ. Development and implementation of an early-onset sepsis calculator to guide antibiotic management in late preterm and term neonates. Jt Comm J Qual Patient Saf. (2016) 42(5):232–9. doi: 10.1016/S1553-7250(16)42030-1

PubMed Abstract | CrossRef Full Text | Google Scholar

63. James G, Witten D, Hastie T, Tibshirani R. An introduction to statistical learning. New York, NY: Springer (2013).

64. Wynn JL, Polin RA. A neonatal sequential organ failure assessment score predicts mortality to late-onset sepsis in preterm very low birth weight infants. Pediatr Res. (2020) 88(1):85–90. doi: 10.1038/s41390-019-0517-2

PubMed Abstract | CrossRef Full Text | Google Scholar

65. Modi N, Doré CJ, Saraswatula A, Richards M, Bamford KB, Coello R, et al. A case definition for national and international neonatal bloodstream infection surveillance. Arch Dis Child Fetal Neonatal Ed. (2009) 94(1):F8–F12. doi: 10.1136/adc.2007.126458

PubMed Abstract | CrossRef Full Text | Google Scholar

66. Lutsar I, Chazallon C, Carducci FI, Trafojer U, Abdelkader B, de Cabre VM, et al. Current management of late onset neonatal bacterial sepsis in five European countries. Eur J Pediatr. (2014) 173(8):997–1004. doi: 10.1007/s00431-014-2279-5

PubMed Abstract | CrossRef Full Text | Google Scholar

67. Sofouli GA, Kanellopoulou A, Vervenioti A, Dimitriou G, Gkentzi D. Predictive scores for late-onset neonatal sepsis as an early diagnostic and antimicrobial stewardship tool: what have we done so far? Antibiotics (Basel, Switzerland). (2022) 11(7):928. doi: 10.3390/antibiotics11070928

PubMed Abstract | CrossRef Full Text | Google Scholar

68. Fitzgerald M, Saville BR, Lewis RJ. Decision curve analysis. JAMA. (2015) 313(4):409–10. doi: 10.1001/jama.2015.37

PubMed Abstract | CrossRef Full Text | Google Scholar

69. Vickers AJ, Van Calster B, Steyerberg EW. Net benefit approaches to the evaluation of prediction models, molecular markers, and diagnostic tests. BMJ. (2016) 352:i6. doi: 10.1136/bmj.i6

PubMed Abstract | CrossRef Full Text | Google Scholar

70. Kerr KF, Brown MD, Zhu K, Janes H. Assessing the clinical impact of risk prediction models with decision curves: guidance for correct interpretation and appropriate use. J Clin Oncol. (2016) 34(21):2534–40. doi: 10.1200/JCO.2015.65.5654

PubMed Abstract | CrossRef Full Text | Google Scholar