National Early Warning Score Does Not Accurately Predict Mortality for Patients With Infection Outside the Intensive Care Unit: A Systematic Review and Meta-Analysis

Background: The prognostic value of the national early warning score (NEWS) in patients with infections remains controversial. We aimed to evaluate the prognostic accuracy of NEWS for prediction of in-hospital mortality in patients with infections outside the intensive care unit (ICU).

Methods: We searched PubMed, Embase, and Scopus for related articles from January 2012 to April 2021. Sensitivity, specificity, and likelihood ratios were pooled by using the bivariate random-effects model. Overall prognostic performance was summarized by using the area under the curve (AUC). We performed subgroup analyses to assess the prognostic accuracy of NEWS in selected populations.

Results: A total of 21 studies with 107,008 participants were included. The pooled sensitivity and specificity of NEWS were 0.71 and 0.60. The pooled AUC of NEWS was 0.70, which was similar to quick sequential organ failure assessment (qSOFA, AUC: 0.70) and better than systemic inflammatory response syndrome (SIRS, AUC: 0.60). However, the sensitivity (0.55) and AUC (0.63) of NEWS were poor in elder patients. The NEWS of 5 was more sensitive, which was a better threshold for activating urgent assessment and treatment.

Conclusions: The NEWS had good diagnostic accuracy for early prediction of mortality in patients with infections outside the ICU, and the sensitivity and specificity were more moderate when compared with qSOFA and SIRS. Insufficient sensitivity and poor performance in the elder population may have limitations as an early warning score for adverse outcomes. NEWS should be used for continuous monitoring rather than a single time point predictive tool.

Introduction

Sepsis, a life-threatening organ dysfunction due to a dysregulated host response to infection, is a major global health problem (1, 2). According to the latest Global Burden of Diseases study, despite declining incidence and mortality, approximately 48.9 million incident cases of sepsis were reported worldwide in 2017, and 11.0 million patients died from sepsis and its complications, accounting for 19.7% of all global deaths (3). Because rapid treatment could improve outcomes of sepsis patients, early identification and risk assessment are of vital importance in the management of sepsis (4, 5). Unfortunately, sepsis remains a complex syndrome with significant heterogeneity and diversity (6); both risk stratification and identification of high-risk patients are still difficult.

The National Early Warning Score (NEWS), first introduced in 2012 and updated in 2017 (National Early Warning Score 2, NEWS2), has received formal endorsement from the National Health Service to become the early warning system for identifying acutely ill patients in the United Kingdom (7, 8). The NEWS is a physiological composite score comprising respiratory rate, oxygen saturation, temperature, blood pressure, pulse rate, and level of consciousness. Each indicator is given a score: 0 is considered normal, and simple addition allows a total score between 0 and 18 to be calculated. A NEWS ≥5 represents the key threshold for urgent response, and patients with NEWS of 7 or more would be deemed to have high clinical risk and trigger a high-level clinical alert (Supplementary File 1). In the guidelines on the recognition, diagnosis and management of sepsis (9), NEWS was recommended for early detection of patients who need more urgent assessment. Moreover, in 2016, the quick sequential organ failure assessment (qSOFA) score was created with the sepsis-3 definitions as a screening tool to identify patients with suspected infection who are likely to have poor outcomes (1). Some studies found that the NEWS was superior to the qSOFA for predicting death or intensive care unit (ICU) admission for patients with infection (10). Although some systematic reviews suggest that NEWS could identify critically ill patients and predict clinically important outcomes (11, 12), the evidence relating to patients with infection is limited (13).

Therefore, we conducted the present study to investigate the prognostic value of NEWS for early prediction of in-hospital mortality in patients with infections outside the ICU. In addition, the performance of the NEWS was compared with that of the qSOFA and systemic inflammatory response syndrome (SIRS) (14) as well.

Methods

Study Selection

We followed the guidelines of PRISMA (Supplementary File 2) to structure the meta-analysis (15). A predefined protocol has been registered in PROSPERO (CRD42020164072). We searched PubMed, Embase, and Scopus from January 2012 to April 2021 for relevant articles. The detailed search strategies are reported in Supplementary File 3.

The basic inclusive criteria are as follows: (1) recruited adult patients with infection outside the ICU, (2) applied NEWS to predict in-hospital mortality or 28/30-day mortality, (3) provided sufficient data to estimate the prognostic accuracy. Detailed criteria are recorded in Supplementary File 3.

Data Extraction

Two authors (X.Z. and W.D.) independently retrieved and extracted studies according to inclusion criteria. We recorded the true positive, false positive, false negative, and true negative from articles directly or through converting of the sensitivity and specificity. Any disagreement in the process was resolved by discussion.

The primary outcome was to estimate the prognostic performance of NEWS for predicting in-hospital mortality. If a study did not report in-hospital mortality, we chose the 28- or 30-day mortality instead. If a study reported the prognostic data on multiple threshold values for the NEWS, we first chose the optimal threshold value (based on the Youden index) for analyses, and then we chose the data on threshold value of ≥5 and ≥7 for subgroup analyses. For included studies that also reported the prognostic performance of qSOFA and/or SIRS (the threshold value for qSOFA and SIRS was ≥2), we estimated the prognostic performance for qSOFA and SIRS to make a comparison with the NEWS. In addition, we also estimated the prognostic performance of NEWS for predicting ICU admission.

Quality Assessment

Two authors (X.Z. and W.D.) independently employed the PROBAST to assess the risk of bias and applicability concerns of included studies (16). The detailed quality assessment criteria are recorded in Supplementary File 3.

Statistical Synthesis and Analysis

The bivariate random-effects regression model was employed to pool the sensitivity, specificity, positive likelihood ratio, negative likelihood ratio, and area under the curve (AUC) as point estimate with 95% confidence interval (CI). We also constructed the hierarchical summary receiver operating characteristic (HSROC) curve to present the summary point estimates of sensitivity and specificity. We calculated the I² statistics to assess the statistical heterogeneity between included studies, where I² > 50% indicated significant heterogeneity (17). We performed subgroup analyses to evaluate the performance of NEWS in selected populations. Studies were stratified according to the age (<70 years old vs. ≥70 years old), disease (non-septic infection vs. sepsis), severity (mortality <10 vs. ≥10%), threshold (≥5 vs. ≥7), setting (emergency department vs. general hospital ward), and study location (United Kingdom vs. other countries). Sensitivity analyses were conducted by repeating the analyses within studies calculating the NEWS at admission.

Publication bias was evaluated by using Deek's test for funnel plot asymmetry (18). All analyses were performed using Stata 12.0 (StataCorp LP, College Station, TX, USA) and Review Manager 5.3 (The Cochrane Collaboration).

Results

Study Selection and Characteristics

A total of 578 published studies were initially identified. After removing duplicate articles and screening abstracts, we identified 49 studies, and 28 studies were excluded with reasons in the full-text assessments. Finally, we included 21 studies (19–39) in our meta-analyses (Figure 1).

FIGURE 1

Figure 1. Flow diagram of study inclusion.

Table 1 shows the characteristics of included studies. These studies recruited a total of 107,008 participants, and the mortality rate in each study ranged from 2.5 to 32.8%. Five studies (20, 24, 27, 31, 32) were relatively small in sample size (<400), and 10 studies (19, 23, 26, 28–30, 33, 36, 37, 40) enrolled more than 1,000 patients. Ten studies (19, 21, 22, 27–29, 32–35) included patients with suspected infection, and others focused on patients with suspected sepsis. Six studies investigated general ward patients (23, 29, 31–33, 39), and others evaluated emergency patients. Three studies (21, 23, 31) used the NEWS2, an updated version of NEWS, and others still used the original NEWS. In addition, 16 studies (20, 21, 24–29, 31–37, 39) estimated the prognostic performance of qSOFA, 10 studies (25–27, 29, 32, 34–37, 39) estimated the prognostic performance of SIRS, and five studies (26, 30, 34, 37, 38) also estimated the prognostic performance of NEWS for predicting the ICU admission. Other relevant information is recorded in Supplementary File 4.

TABLE 1

Table 1. Characteristics of included studies.

Quality Assessment

Table 2 shows the summary results of quality assessments by using PROBAST. Overall, 15 studies had high or unclear risk of bias, mainly because of the inappropriate handling method of missing data (nine studies excluded participants with missing values from analyses; four studies did not explicitly state the handling method of missing data). Seven studies had high or unclear concern regarding applicability because the time interval between the evaluation of the predictor and the determination of the outcome were not consistent with other studies. In addition, Fernando et al. (23) enrolled patients activating a rapid response team, and Pairattanakorn et al. (39) analyzed all hospitalized patients, including some ICU patients. These two studies were rated as high risk of bias and high concern regarding applicability in selection of participants. The details of quality assessment are reported in Supplementary File 5.

TABLE 2

Table 2. PROBAST results.

Furthermore, Deek's funnel plot indicated there was no significant publication bias (Supplementary File 6).

Results of Synthesis

Figure 2 shows the forest plot of sensitivity and specificity for NEWS, qSOFA, and SIRS. Figure 3 shows the HSROC curves for NEWS, qSOFA, and SIRS. The pooled sensitivity, specificity, and AUC of NEWS were 0.71 (95%CI 0.65, 0.76), 0.60 (95%CI 0.54, 0.66), and 0.70 (95%CI 0.65, 0.76), respectively. For qSOFA, the pooled sensitivity, specificity, and AUC were 0.48 (95%CI 0.38, 0.58), 0.80 (95%CI 0.73, 0.86), and 0.70 (95%CI 0.66, 0.74), respectively. For SIRS, the pooled sensitivity, specificity, and AUC were 0.85 (95%CI 0.76, 0.90), 0.25 (95%CI 0.17, 0.36), and 0.60 (95%CI 0.55, 0.64), respectively. In addition, the pooled sensitivity, specificity, and AUC of NEWS for predicting the ICU admission were 0.71 (95%CI 0.66, 0.76), 0.55 (95%CI 0.43, 0.65), and 0.71 (95%CI 0.67, 0.75).

FIGURE 2

Figure 2. Forest plots of sensitivity and specificity for NEWS, qSOFA and SIRS.

FIGURE 3

Figure 3. Hierarchical summary ROC curves for (A) NEWS, (B) qSOFA, (C) SIRS for predicting mortality for patients with infection outside the ICU.

Subgroup and Sensitivity Analyses

There was evidence that prognostic performance varied across different subgroups (Table 3). First, in the cohort of older patients (≥70 years old), the sensitivity was poor (0.55), and the AUC was significant lower than that in younger patients (AUC: 0.63 vs. 0.72, Z = 3.562, P < 0.001). Second, in patients with sepsis or more severe conditions (mortality rate ≥10%), the NEWS was more sensitive but relatively less specific in predicting mortality. For studies conducted in the United Kingdom, the NEWS was less sensitive (sensitivity: 0.62 vs. 0.75) and the AUC was lower although this was not statistically significant (AUC: 0.68 vs. 0.71, Z = 0.653, P = 0.514). Moreover, in the 12 studies that used the threshold of 5 or more, the pooled sensitivity, specificity, and AUC were 0.80 (95%CI 0.71, 0.86), 0.50 (95%CI 0.36, 0.63), and 0.73 (95%CI 0.69, 0.76). In nine studies that used the threshold of 7 or more, the pooled sensitivity, specificity, and AUC were 0.74 (95%CI 0.66, 0.81), 0.57 (95%CI 0.47, 0.66), and 0.71 (95%CI 0.67, 0.75).

TABLE 3

Table 3. Results of meta-analysis.

In addition, when we restricted analysis to studies (19–24, 26, 28, 31, 32, 34–38) that evaluated the NEWS at admission or excluded five studies with small sample sizes (20, 24, 27, 31, 32), the sensitivity analyses showed similar results to the primary result.

Discussion

In this study, we analyzed the prognostic accuracy of the NEWS for mortality in patients with infections outside the ICU and compared the performance with that of the qSOFA and SIRS. We found the NEWS had good diagnostic accuracy for early prediction of mortality in patients with infections outside the ICU, and the sensitivity and specificity were moderate when compared with qSOFA and SIRS. That the NEWS ≥5 was more sensitive for predicting mortality in patients with infections outside the ICU indicates that 5 points is an optimal threshold for activating urgent treatment or critical care input.

Estimates of pooled results showed considerable heterogeneity between studies. Investigating the source of heterogeneity and the prognostic performance of NEWS in selected populations are important objectives in our study. First, aging appears to decrease the predictive accuracy of NEWS. Our results indicate that the NEWS could not accurately predict mortality in the elder population. Due to the relatively low sensitivity and AUC, the NEWS was of limited prognostic value in elder patients with infection. Second, the severity of disease might affect the prognostic accuracy. Our study population was composed of two groups: patients with non-septic infection and patients with sepsis. Applying the NEWS in septic patients who have higher mortality rates might result in greater predictive probabilities of death as opposed to employing it among patients with non-septic infection. The subgroup analysis indicated that the sensitivity of NEWS in the sepsis subgroup was higher than the non-septic infection subgroup. Besides this, different ways to identify infection (e.g., patients with positive blood cultures or who received intravenous antibiotics) or sepsis (e.g., sepsis-1 or sepsis-3 criteria) may have been responsible for heterogeneity. Furthermore, study location might be a source of heterogeneity because differences in the health care systems of each country could affect clinical outcomes. Specifically, early warning score systems have been introduced and linked to effective clinical responses in many UK hospitals (41). It might introduce the treatment paradox, by which some deteriorating patients were likely to receive rapid medical interventions after triggering the alert. Hence, the actual mortality tends to be lower than predicted and biases our estimate of accuracy. In addition, our study included patients with different infection types, and variations in outcome measures (in-hospital or 28/30-day mortality) could also account for heterogeneity.

Comparisons With Previous Literature

Sepsis is a common cause of adverse clinical outcomes among patients with infections, and acute management is the foundation of improved outcomes for these patients (5). The main problem in the management of sepsis outside the ICU is identification of high-risk patients since the first assessment. This way, early treatments can be started, which would alter the septic patient's prognosis. Clinicians previously used the SIRS criteria to diagnose sepsis for patients with infection, which is criticized because the criteria were too non-specific (42). The sepsis-3 definitions task force developed the qSOFA score as an early warning risk stratification tool for identification and escalation of care in septic patients outside the ICU (43). However, Song et al. investigate the prognostic value of qSOFA and SIRS in patients with infection outside the ICU and indicate that qSOFA could not serve as a predictive tool for adverse outcomes because of its low sensitivity (0.51) (44). Similarly, Fernando et al. comprehensively investigate the prognostic accuracy of qSOFA in different populations and find that qSOFA had significant lower sensitivity in patients outside the ICU than others in the ICU (0.46 vs. 0.87) (45). In our study, we prove that the qSOFA score had the highest specificity but low sensitivity in predicted mortality, which was consistent with previous studies (44, 45). Compared with qSOFA and SIRS, the NEWS had moderate sensitivity and specificity and was superior to the qSOFA for predicting mortality.

In addition to the NEWS, qSOFA, and SIRS, some severity scoring systems have been widely used outside the ICU (12). The mortality in emergency department sepsis (MEDS) score was specially developed for emergency patients with suspected infections to predict the 28-day mortality rate (46). Our current study also reveals that MEDS has good discrimination (AUC 0.83) and moderate sensitivity (0.79) and specificity (0.74) (47). Some of the score components rely on a clinician's subjective assessment (e.g., terminal illness, altered mental status), and the neutrophil bands were not routinely measured in some departments. These weaknesses limit the scope of its implementation. The modified early warning score (MEWS) is another frequently used prognostic score system, which is considered the most reliable method to assess in-hospital mortality in the general population (12). However, when applied to patients with sepsis, Hamilton et al. analyzes five trials and suggests the MEWS has poor prognostic value in predicting sepsis mortality (48). In addition, we further searched relevant studies and performed an updated meta-analysis, and our results reconfirmed that MEWS could not accurately identify patients at high risk of death (49).

Furthermore, it is possible that the prognostic accuracy of the NEWS could be improved by combining some important clinical parameters. For instance, research shows that older patients with infection have a higher death rate (3), and increased age is independently associated with poor prognosis in septic patients (50). Thus, a Chinese group put forward a modified version of the NEWS with the addition of age >65 years as an independent component, termed NEWS-C (51). An external validation study found the NEWS-C has the best predictive accuracy among common scoring systems for predicting deterioration of respiratory function in patients with COVID-19 (52). Moreover, lactate is a strong and independent predictor of mortality for patients with infection (53). A modified early warning score combining the NEWS and initial serum lactate level, called NEWS-L, is proved to have good discriminant value for identifying high-risk patients in the emergency department (54).

Implications for Practice

Although our research suggests that NEWS has moderate prognostic performance, it is worth highlighting some potential pitfalls in clinical practice. First of all, the qSOFA only contains three components and three boundary levels, whereas NEWS has seven components with more than 20 boundaries. Certainly, the application of NEWS requires additional resources and could become a burden on clinicians, especially in situations of overcrowding in the emergency wards. Second, the NEWS is not an ideal screening tool to identify high-risk patients because of its limited sensitivity. It means that some critically ill patients may be improperly classified as non-severe, even delayed treatment, which is devastating to the patients. Therefore, the NEWS is not an alternative to the clinical judgment by experienced clinicians, and it should be utilized to help clinical decision making by providing objective data. Even negative NEWS should not prevent clinicians from conducting further assessment and management of patients with suspected infection, indicating that experienced clinical judgment remains vital (13). Finally, in addition to the initial assessment of illness severity, the NEWS is also recommended to be used as a track and trigger tool to identify acute clinical deterioration and guide the clinical response for patients. By recording the NEWS on a regular basis, the trends in the patient's clinical response can be tracked (7). Therefore, the score should be calculated not only at patient admission but also throughout the hospital stay as part of the standard clinical observation chart to evaluate a possible deterioration in the clinical situation (48).

Strength and Limitations

Strengths of this meta-analysis include a standard protocol and comprehensive search strategies across multiple databases. Thus, we believe that we did not miss any relevant studies. Second, a statistically robust hierarchical model was employed to estimate pooled results and to construct HSROC plots. This approach allows for both between-study variability in sensitivity and specificity and flexibility in the estimation of summary statistics (55). Our findings can contribute to a better understanding of NEWS in patients with infections outside the ICU and could be useful for implementing NEWS in clinical practice.

Meanwhile, there are some important limitations in the meta-analysis. First, previous research suggests heterogeneities are widely observed in systematic reviews of diagnostic test accuracy (44, 56). We also identify significant heterogeneity among included studies, which might affect the credibility of the pooled estimates. Although we directly compared the NEWS with qSOFA using the same cohort of patients to minimize heterogeneity, the results should be interpreted prudently. Second, both the NEWS and qSOFA were developed for detecting patients with high risk of clinical deterioration rather than predicting in-hospital mortality. This may be the main reason why NEWS shows poor performance in our meta-analysis. Further research should focus on the prognostic accuracy of NEWS for predicting clinical deterioration. Third, as only three studies evaluated the NEWS2, the insufficient data could not develop reliable conclusions regarding the potential benefits of the updated score over the original NEWS. Furthermore, the NEWS was not designed as a single time point predictive tool. Because existing research only shows the prognostic accuracy of NEWS in predicting mortality at a single time point (mostly at the time of admission), we could not evaluate NEWS in any other context. On the other hand, the timing of NEWS measurement was not entirely consistent in included studies. Given the dynamic nature of sepsis, we assume that the accuracy might be improved if multiple time points were considered. The change trend of NEWS with time has potential application value of predicting mortality, just like the delta SOFA (57).

Conclusion

The NEWS has good diagnostic accuracy for early prediction of mortality in patients with infections outside the ICU, the sensitivity and specificity were more moderate when compared with qSOFA and SIRS. NEWS of 5 or more was an optimal trigger threshold for activating a rapid response. However, as an early warning score, both NEWS and qSOFA had a significant weakness that insufficient sensitivity could delay lifesaving treatment for critical patients. The NEWS should be used for continuous monitoring of patients' condition and guide clinical response, not solely for initial assessment of illness severity. We suggest that developing enhanced or modified scoring systems is quite necessary, and future early warning scores could be devised by using machine learning algorithms.

Data Availability Statement

The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author/s.

Author Contributions

KZ conceived the idea, performed the analysis, and drafted the manuscript. XZ and WD contributed to the study design, data acquisition, and interpretation. NX, BT, and TH helped in the statistical analysis. ZZ and WC critically revised the manuscript for important intellectual content. GZ and HH helped to frame the idea of the study and provided technical support. All authors contributed to the article and approved the submitted version.

Funding

This work was supported in part by grants from the National Natural Science Foundation of China (Nos. 81971871, GZ; 81772110, ZZ). The sponsors of this study had no role in study design, data collection, data analysis, data interpretation, or writing of the report.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fmed.2021.704358/full#supplementary-material

Abbreviations

ICU, Intensive care unit; qSOFA, quick Sequential Organ Failure Assessment; SOFA, Sequential Organ Failure Assessment; AUC, Area Under the Curve; NEWS, National Early Warning Score; CI, confidence interval; PLR, positive likelihood ratio; NLR, negative likelihood ratio; DOR, diagnostic odds ratio; HSROC, hierarchical summary receiver operating characteristic; QUADAS, Quality Assessment of Diagnostic Accuracy Studies.

References

1. Singer M, Deutschman CS, Seymour CW, Shankar-Hari M, Annane D, Bauer M, et al. The Third International Consensus definitions for sepsis and septic shock (Sepsis-3). JAMA. (2016) 315:801–10. doi: 10.1001/jama.2016.0287

CrossRef Full Text | Google Scholar

2. Fleischmann C, Scherag A, Adhikari NK, Hartog CS, Tsaganos T, Schlattmann P, et al. Assessment of global incidence and mortality of hospital-treated sepsis. Current estimates and limitations. Am J Respir Crit Care Med. (2016) 193:259–72. doi: 10.1164/rccm.201504-0781OC

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Rudd KE, Johnson SC, Agesa KM, Shackelford KA, Tsoi D, Kievlan DR, et al. Global, regional, and national sepsis incidence and mortality, 1990–2017: analysis for the Global Burden of Disease Study. Lancet. (2020) 395:200–11. doi: 10.1016/S0140-6736(19)32989-7

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Seymour CW, Gesten F, Prescott HC, Friedrich ME, Iwashyna TJ, Phillips GS, et al. Time to treatment and mortality during mandated emergency care for sepsis. New Engl J Med. (2017) 376:2235–44. doi: 10.1056/NEJMoa1703058

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Rhodes A, Evans LE, Alhazzani W, Levy MM, Antonelli M, Ferrer R, et al. Surviving sepsis campaign: international guidelines for management of sepsis and septic shock: 2016. Intensive Care Med. (2017) 43:304–77. doi: 10.1007/s00134-017-4683-6

PubMed Abstract | CrossRef Full Text | Google Scholar

6. de Grooth H-J, Postema J, Loer SA, Parienti J-J, Oudemans-van Straaten HM, Girbes AR. Unexplained mortality differences between septic shock trials: a systematic analysis of population characteristics and control-group mortality rates. Intens Care Med. (2018) 44:311–22. doi: 10.1007/s00134-018-5134-8

PubMed Abstract | CrossRef Full Text | Google Scholar

7. National Early Warning Score (NEWS). Standardising the Assessment of Acute-Illness Severity in the NHS. Report of a Working Party. London: RCP: Royal College of Physicians (2012).

8. National Early Warning Score (NEWS) 2. Standardising the Assessment of Acute-Illness Severity in the NHS. Updated Report of a Working Party. London: Royal College of Physicians (2017).

9. NICE. Sepsis: Recognition, Diagnosis Early Management | Recommendations | Guidance Guidelines. London: NICE (2016).

10. Steele L, Hill S. Using sepsis scores in emergency department and ward patients. Br J Hosp Med. (2019) 80:C120–3. doi: 10.12968/hmed.2019.80.8.C120

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Williams TA, Tohira H, Finn J, Perkins GD, Ho KM. The ability of early warning scores (EWS) to detect critical illness in the prehospital setting: a systematic review. Resuscitation. (2016) 102:35–43. doi: 10.1016/j.resuscitation.2016.02.011

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Nannan Panday RS, Minderhoud TC, Alam N, Nanayakkara PWB. Prognostic value of early warning scores in the emergency department (ED) and acute medical unit (AMU): a narrative review. Eur J Intern Med. (2017) 45:20–31. doi: 10.1016/j.ejim.2017.09.027

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Postelnicu R, Pastores SM, Chong DH, Evans L. Sepsis early warning scoring systems: the ideal tool remains elusive! J Crit Care. (2019) 52:251–3. doi: 10.1016/j.jcrc.2018.07.009

CrossRef Full Text | Google Scholar

14. Bone RC, Balk RA, Cerra FB, Dellinger RP, Fein AM, Knaus WA, et al. Definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis. Chest. (1992) 101:1644–55. doi: 10.1378/chest.101.6.1644

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. (2009) 6:e1000097. doi: 10.1371/journal.pmed.1000097

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Wolff RF, Moons KGM, Riley RD, Whiting PF, Westwood M, Collins GS, et al. PROBAST: a tool to assess the risk of bias and applicability of prediction model studies. Ann Intern Med. (2019) 170:51–58. doi: 10.7326/M18-1376

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Higgins JP, Thompson SG. Quantifying heterogeneity in a meta-analysis. Stat Med. (2002) 21:1539–58. doi: 10.1002/sim.1186

CrossRef Full Text | Google Scholar

18. Deeks JJ, Macaskill P, Irwig L. The performance of tests of publication bias and other sample size effects in systematic reviews of diagnostic test accuracy was assessed. J Clin Epidemiol. (2005) 58:882–93. doi: 10.1016/j.jclinepi.2005.01.016

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Saeed K, Wilson DC, Bloos F, Schuetz P, van der Does Y, Melander O, et al. The early identification of disease progression in patients with suspected infection presenting to the emergency department: a multi-centre derivation and validation study. Crit Care. (2019) 23:40. doi: 10.1186/s13054-019-2536-0

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Pong JZ, Fook-Chong S, Koh ZX, Samsudin MI, Tagami T, Chiew CJ, et al. Combining heart rate variability with disease severity score variables for mortality risk stratification in septic patients presenting at the emergency department. Int J Environ Res Public Health. (2019) 16:1725. doi: 10.3390/ijerph16101725

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Mellhammar L, Linder A, Tverring J, Christensson B, Boyd JH, Sendi P, et al. NEWS2 is superior to qSOFA in detecting sepsis with organ dysfunction in the emergency department. J Clin Med. (2019) 8:1128. doi: 10.3390/jcm8081128

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Gonzalez Del Castillo J, Wilson DC, Clemente-Callejo C, Roman F, Bardes-Robles I, Jimenez I, et al. Biomarkers and clinical scores to identify patient populations at risk of delayed antibiotic administration or intensive care admission. Crit Care. (2019) 23:335. doi: 10.1186/s13054-019-2613-4

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Fernando SM, Fox-Robichaud AE, Rochwerg B, Cardinal P, Seely AJE, Perry JJ, et al. Prognostic accuracy of the Hamilton Early Warning Score (HEWS) and the National Early Warning Score 2 (NEWS2) among hospitalized patients assessed by a rapid response team. Crit Care. (2019) 23:60. doi: 10.1186/s13054-019-2355-3

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Chiew CJ, Liu N, Tagami T, Wong TH, Koh ZX, Ong MEH. Heart rate variability based machine learning models for risk prediction of suspected sepsis patients in the emergency department. Medicine. (2019) 98:e14197. doi: 10.1097/MD.0000000000014197

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Brink A, Alsma J, Verdonschot R, Rood PPM, Zietse R, Lingsma HF, et al. Predicting mortality in patients with suspected sepsis at the Emergency Department; A retrospective cohort study comparing qSOFA, SIRS and National Early Warning Score. PLoS ONE. (2019) 14:e0211133. doi: 10.1371/journal.pone.0211133

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Goulden R, Hoyle MC, Monis J, Railton D, Riley V, Martin P, et al. qSOFA, SIRS and NEWS for predicting inhospital mortality and ICU admission in emergency admissions treated as sepsis. Emerg Med J. (2018) 35:345–9. doi: 10.1136/emermed-2017-207120

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Camm CF, Hayward G, Elias TCN, Bowen JST, Hassanzadeh R, Fanshawe T, et al. Sepsis recognition tools in acute ambulatory care: associations with process of care and clinical outcomes in a service evaluation of an Emergency Multidisciplinary Unit in Oxfordshire. BMJ Open. (2018) 8:e020497. doi: 10.1136/bmjopen-2017-020497

PubMed Abstract | CrossRef Full Text | Google Scholar

28. de Groot B, Stolwijk F, Warmerdam M, Lucke JA, Singh GK, Abbas M, et al. The most commonly used disease severity scores are inappropriate for risk stratification of older emergency department sepsis patients: an observational multi-centre study. Scand J Trauma Resusc Emerg Med. (2017) 25:91. doi: 10.1186/s13049-017-0436-3

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Churpek MM, Snyder A, Han X, Sokol S, Pettit N, Howell MD, et al. Quick sepsis-related organ failure assessment, systemic inflammatory response syndrome, and early warning scores for detecting clinical deterioration in infected patients outside the intensive care unit. Am J Respir Crit Care Med. (2017) 195:906–11. doi: 10.1164/rccm.201604-0854OC

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Corfield AR, Lees F, Zealley I, Houston G, Dickie S, Ward K, et al. Utility of a single early warning score in patients with sepsis in the emergency department. Emerg Med J. (2014) 31:482–7. doi: 10.1136/emermed-2012-202186

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Ye Lynn KL, Hanson J, Mon NCN, Yin KN, Nyein ML, Thant KZ, et al. The clinical characteristics of patients with sepsis in a tertiary referral hospital in Yangon, Myanmar. Transact R Soc Trop Med Hyg. (2019) 113:81–90. doi: 10.1093/trstmh/try115

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Szakmany T, Pugh R, Kopczynska M, Lundin RM, Sharif B, Morgan P, et al. Defining sepsis on the wards: results of a multi-centre point-prevalence study comparing two sepsis definitions. Anaesthesia. (2018) 73:195–204. doi: 10.1111/anae.14062

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Redfern OC, Smith GB, Prytherch DR, Meredith P, Inada-Kim M, Schmidt PE. A comparison of the quick sequential (sepsis-related) organ failure assessment score and the national early warning score in non-ICU patients with/without infection. Crit Care Med. (2018) 46:1923–33. doi: 10.1097/CCM.0000000000003359

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Oduncu AF, Kiyan GS, Yalçinli S. Comparison of qSOFA, SIRS, and NEWS scoring systems for diagnosis, mortality, and morbidity of sepsis in emergency department. Am J Emerg Med. (2021) 48:54–9. doi: 10.1016/j.ajem.2021.04.006

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Wattanasit P, Khwannimit B. Comparison the accuracy of early warning scores with qSOFA and SIRS for predicting sepsis in the emergency department. Am J Emerg Med. (2020) S0735-6757(20)30672-0. doi: 10.1016/j.ajem.2020.07.077

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Ruangsomboon O, Boonmee P, Limsuwat C, Chakorn T, Monsomboon A. The utility of the rapid emergency medicine score (REMS) compared with SIRS, qSOFA and NEWS for predicting in-hospital mortality among patients with suspicion of sepsis in an emergency department. BMC Emerg Med. (2021) 21:2. doi: 10.1186/s12873-020-00396-x

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Phungoen P, Khemtong S, Apiratwarakul K, Ienghong K, Kotruchin P. Emergency severity index as a predictor of in-hospital mortality in suspected sepsis patients in the emergency department. Am J Emerg Med. (2020) 38:1854–9. doi: 10.1016/j.ajem.2020.06.005

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Almutary A, Althunayyan S, Alenazi K, Alqahtani A, Alotaibi B, Ahmed M, et al. National early warning score (NEWS) as prognostic triage tool for septic patients. Infect Drug Resist. (2020) 13:3843–51. doi: 10.2147/IDR.S275390

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Pairattanakorn P, Angkasekwinai N, Sirijatuphat R, Wangchinda W, Tancharoen L, Thamlikitkul V. Diagnostic and prognostic utility compared among different sepsis scoring systems in adult patients with sepsis in thailand: a prospective cohort study. Open Forum Infect Dis. (2021) 8:ofaa573. doi: 10.1093/ofid/ofaa573

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Grudzinska FS, Aldridge K, Hughes S, Nightingale P, Parekh D, Bangash M, et al. Early identification of severe community-acquired pneumonia: a retrospective observational study. BMJ Open Respir Res. (2019) 6:e000438. doi: 10.1136/bmjresp-2019-000438

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Scott LJ, Redmond NM, Garrett J, Whiting P, Northstone K, Pullyblank A. Distributions of the National Early Warning Score (NEWS) across a healthcare system following a large-scale roll-out. Emerg Med J. (2019) 36:287–92. doi: 10.1136/emermed-2018-208140

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Sprung CL, Sakr Y, Vincent JL, Le Gall JR, Reinhart K, Ranieri VM, et al. An evaluation of systemic inflammatory response syndrome signs in the sepsis occurrence in acutely Ill patients (SOAP) study. Intens Care Med. (2006) 32:421–7. doi: 10.1007/s00134-005-0039-8

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Vincent JL, Martin GS, Levy MM. qSOFA does not replace SIRS in the definition of sepsis. Crit Care. (2016) 20:210. doi: 10.1186/s13054-016-1389-z

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Song JU, Sin CK, Park HK, Shim SR, Lee J. Performance of the quick sequential (sepsis-related) organ failure assessment score as a prognostic tool in infected patients outside the intensive care unit: a systematic review and meta-analysis. Crit Care. (2018) 22:28. doi: 10.1186/s13054-018-1952-x

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Fernando SM, Tran A, Taljaard M, Cheng W, Rochwerg B, Seely AJE, et al. Prognostic accuracy of the quick sequential organ failure assessment for mortality in patients with suspected infection: a systematic review and meta-analysis. Ann Intern Med. (2018) 168:266–75. doi: 10.7326/M17-2820

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Shapiro NI, Wolfe RE, Moore RB, Smith E, Burdick E, Bates DW. Mortality in emergency department sepsis (MEDS) score: a prospectively derived and validated clinical prediction rule. Crit Care Med. (2003) 31:670–5. doi: 10.1097/01.CCM.0000054867.01688.D1

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Zhang G, Zhang K, Zheng X, Cui W, Hong Y, Zhang Z. Performance of the MEDS score in predicting mortality among emergency department patients with a suspected infection: a meta-analysis. Emerg Med J. (2020) 37:232–9. doi: 10.1136/emermed-2019-208901

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Hamilton F, Arnold D, Baird A, Albur M, Whiting P. Early warning scores do not accurately predict mortality in sepsis: a meta-analysis and systematic review of the literature. J Infect. (2020) 81:e100–e101. doi: 10.1016/j.jinf.2018.01.002

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Zhang K, Zhang G. An updated meta-analysis of modified early warning scores in patients with sepsis outside intensive care unit. J Infec. (2020). doi: 10.1016/j.jinf.2020.04.015

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Herrán-Monge R, Muriel-Bombín A, García-García MM, Merino-García PA, Martínez-Barrios M, Andaluz D, et al. Epidemiology and changes in mortality of sepsis after the implementation of surviving sepsis campaign guidelines. J Intens Care Med. (2019) 34:740–50. doi: 10.1177/0885066617711882

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Liao X, Wang B, Kang Y. Novel coronavirus infection during the 2019-2020 epidemic: preparing intensive care units-the experience in Sichuan Province, China. Intens Care Med. (2020) 46:357–60. doi: 10.1007/s00134-020-05954-2

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Su Y, Ju MJ, Xie RC, Yu SJ, Zheng JL, Ma GG, et al. Prognostic accuracy of early warning scores for clinical deterioration in patients with COVID-19. Front Med. (2020) 7:624255. doi: 10.3389/fmed.2020.624255

PubMed Abstract | CrossRef Full Text | Google Scholar

53. Trzeciak S, Dellinger RP, Chansky ME, Arnold RC, Schorr C, Milcarek B, et al. Serum lactate as a predictor of mortality in patients with infection. Intens Care Med. (2007) 33:970–7. doi: 10.1007/s00134-007-0563-9

PubMed Abstract | CrossRef Full Text | Google Scholar

54. Jo S, Yoon J, Lee JB, Jin Y, Jeong T, Park B. Predictive value of the National Early Warning Score-Lactate for mortality and the need for critical care among general emergency department patients. J Crit Care. (2016) 36:60–8. doi: 10.1016/j.jcrc.2016.06.016

PubMed Abstract | CrossRef Full Text | Google Scholar

55. Rutter CM, Gatsonis CA. A hierarchical regression approach to meta-analysis of diagnostic test accuracy evaluations. Stat Med. (2001) 20:2865–84. doi: 10.1002/sim.942

PubMed Abstract | CrossRef Full Text | Google Scholar

56. Dinnes J, Deeks J, Kirby J, Roderick P. A methodological review of how heterogeneity has been examined in systematic reviews of diagnostic test accuracy. Health Technol Assess. (2005) 9:1–113, iii. doi: 10.3310/hta9120

PubMed Abstract | CrossRef Full Text | Google Scholar

57. de Grooth HJ, Geenen IL, Girbes AR, Vincent JL, Parienti JJ, Oudemans-van Straaten HM. SOFA and mortality endpoints in randomized controlled trials: a systematic review and meta-regression analysis. Crit Care. (2017) 21:38. doi: 10.1186/s13054-017-1609-1

PubMed Abstract | CrossRef Full Text | Google Scholar