Low Accuracy of Positive qSOFA Criteria for Predicting 28-Day Mortality in Critically Ill Septic Patients During the Early Period After Emergency Department Presentation

doi:10.1016/j.annemergmed.2017.05.022

Annals of Emergency Medicine

Volume 71, Issue 1, January 2018, Pages 1-9.e2

https://doi.org/10.1016/j.annemergmed.2017.05.022 Get rights and content

Study objective

We determine the diagnostic performance of positive Quick Sequential Organ Failure Assessment (qSOFA) scores for predicting 28-day mortality among critically ill septic patients during the early period after emergency department (ED) presentation.

Methods

This was a retrospective cohort study at a tertiary care academic center. We reviewed a registry of adult (≥18 years) patients who received a diagnosis of severe sepsis or septic shock during an ED stay from August 2008 through September 2014. We identified the point at which patients met 2 or more of the 3 qSOFA criteria (indicating a positive qSOFA score) simultaneously during the initial 24 hours. The diagnostic performance of positive qSOFA score for predicting 28-day mortality was assessed (on ED arrival and within 3, 6, and 24 hours after ED presentation).

Results

A total of 1,395 patients were included, and the overall 28-day mortality was 15%. For patients with positive qSOFA score, 28-day mortality was 23% (95% confidence interval [CI] 19% to 28%) on ED arrival, 20% (95% CI 17% to 23%) at 3 hours, 20% (95% CI 17% to 22%) at 6 hours, and 17% (95% CI 15% to 20%) at 24 hours. Positive qSOFA score for predicting 28-day mortality had a sensitivity, specificity, and area under the receiver operating curve, respectively, of 39% (95% CI 32% to 46%), 77% (95% CI 75% to 80%), and 0.58 (95% CI 0.55 to 0.62) on ED arrival; 68% (95% CI 62% to 75%), 52% (95% CI 49% to 55%), and 0.60 (95% CI 0.57 to 0.63) within 3 hours; 82% (95% CI 76% to 87%), 41% (95% CI 38% to 44%), and 0.61 (95% CI 0.58 to 0.64) within 6 hours; and 91% (95% CI 86% to 94%), 23% (95% CI 21% to 25%), and 0.57 (95% CI 0.54 to 0.59) within 24 hours.

Conclusion

The diagnostic performance of positive qSOFA score for predicting 28-day mortality was low in critically ill septic patients, particularly during the early period after ED presentation. The study requires further prospective validation because of limitations with its retrospective design and use of single-center data.

Introduction

Sepsis is a public health concern affecting millions of people each year, and it is a leading cause of morbidity and mortality among hospitalized patients.¹ However, the complexities and heterogeneity of the disease make it challenging for health care providers to identify patients with sepsis. In the past 2 decades, a consensus definition based on systemic inflammatory response syndrome has been used to diagnose sepsis.²

Editor’s Capsule Summary

What is already known on this topic

The Quick Sequential Organ Failure Assessment (qSOFA) was recently introduced as an easy tool to identify infected patients with high risk of subsequent deterioration.

What question this study addressed

The prognostic performance of qSOFA scores during the initial 24 hours of hospitalization to predict 28-day mortality among 1,395 patients identified in the emergency department (ED) with sepsis.

What this study adds to our knowledge

Positive qSOFA score (2 or more points) at ED presentation and 3, 6, and 24 hours had poor sensitivity and specificity for predicting 28-day mortality.

How this is relevant to clinical practice

This informs clinical practice by suggesting that qSOFA may need more investigation and refinement before widespread use as a method of risk stratification of ED patients with suspected infection.

Recently, the definitions of sepsis and septic shock were updated by the Third International Consensus Definitions Task Force.³ In accordance with the new definitions, the quick Sequential [Sepsis-related] Organ Failure Assessment (qSOFA) score was introduced as a novel risk-stratification tool outside the ICU. The qSOFA score consists of 3 clinical variables: systolic blood pressure less than or equal to 100 mm Hg, respiratory rate greater than or equal to 22 breaths/min, and altered mentation.⁴ The presence of 2 or more of these criteria, which is considered a positive qSOFA score, is an initial way to identify patients with suspected infection who are at high risk for poor outcomes (inhospital mortality or ICU stay ≥3 days) and to prompt further evaluation for organ damage and initiate a higher level of care.

qSOFA is simple and can be performed quickly and repeatedly at the bedside according to clinical parameters, without any laboratory tests. In addition, the predictive validity of the qSOFA score for inhospital mortality among patients with suspected infection outside the ICU was greater than that of the full Sequential [Sepsis-related] Organ Failure Assessment (SOFA) score or the systemic inflammatory response syndrome criteria.⁴ However, the illness severity of the cohort of the original qSOFA study was relatively low, and emergency department (ED) populations were not analyzed separately, which makes the role of the qSOFA in the ED unclear. qSOFA requires further assessment for appropriate ED application.

The aim of this study was to investigate the diagnostic performance of positive qSOFA score for predicting 28-day mortality among critically ill septic patients from ED arrival to 24 hours after ED presentation. We also evaluated the cumulative proportions of positive qSOFA score and the positive components of each qSOFA criterion during the first 24 hours after ED arrival.

Section snippets

Study Design and Setting

This was a single-center, retrospective study of patients who presented to the ED at Samsung Medical Center (a 1,960-bed, university-affiliated, tertiary care referral hospital located in a metropolitan city with an annual census of 70,000). The study period was from August 2008 through September 2014.

Adult patients who met the criteria for severe sepsis or septic shock during their ED stay were registered for data collection and quality improvement activity in our institution’s registry, under

Characteristics of Study Subjects

A total of 1,728 patients with severe sepsis or septic shock were identified during the study period. Of these, 333 patients were excluded, and the remaining 1,395 patients were included in the final analysis. Of the eligible patients, 717 (51%) showed positive qSOFA scores within 3 hours (Figure 1). The baseline characteristics of all patients and comparisons based on a positive qSOFA score within 3 hours of ED arrival are shown in Table 1. We present the baseline characteristics according to

Limitations

There are some limitations to this study that should be considered. First, our study population consisted of critically ill patients with severe sepsis and septic shock, half of whom required ICU admission. We did not include patients with uncomplicated infections or patients receiving end-of-life care who might decline initial diagnostic tests or treatments. Originally, our study data were collected primarily for performance improvement of initial management of severe sepsis and septic shock

Discussion

In this study, we investigated the diagnostic performance of qSOFA for predicting 28-day mortality in critically ill septic patients who presented to the ED. We found that a considerable number of patients did not meet the criteria for positive qSOFA score, resulting in low sensitivity, particularly during the early period after ED presentation. The sensitivity of qSOFA was lowest on patient ED arrival and gradually increased. Overall, qSOFA criteria for predicting mortality had low accuracy in

References (29)

R.C. Bone et al.
Definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis. The ACCP/SCCM Consensus Conference Committee. American College of Chest Physicians/Society of Critical Care Medicine
Chest
(1992)
A.H. Kaji et al.
Looking through the retrospectoscope: reducing bias in emergency medicine chart review studies
Ann Emerg Med
(2014)
A.J. Singer et al.
Quick SOFA scores predict mortality in adult emergency department patients with and without suspected infection
Ann Emerg Med
(2017)
J.M. Williams et al.
SIRS, qSOFA and organ dysfunction: insights from a prospective database of emergency department patients with infection
Chest
(2017)
J.Y. Wang et al.
Predictive performance of quick Sepsis-related Organ Failure Assessment for mortality and ICU admission in patients with infection at the ED
Am J Emerg Med
(2016)
M.D. April et al.
Sepsis clinical criteria in emergency department patients admitted to an intensive care unit: an external validation study of quick Sequential Organ Failure Assessment
J Emerg Med
(2017)
M.A. Puskarich et al.
Outcomes of patients undergoing early sepsis resuscitation for cryptic shock compared with overt shock
Resuscitation
(2011)
D.C. Angus et al.
Epidemiology of severe sepsis in the United States: analysis of incidence, outcome, and associated costs of care
Crit Care Med
(2001)
M. Singer et al.
The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3)
JAMA
(2016)
C.W. Seymour et al.
Assessment of clinical criteria for sepsis: for the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3)
JAMA
(2016)

T.G. Shin et al.

Comprehensive interpretation of central venous oxygen saturation and blood lactate levels during resuscitation of patients with severe sepsis and septic shock in the emergency department

Shock

(2016)

Y.M. Joo et al.

Impact of timely antibiotic administration on outcomes in patients with severe sepsis and septic shock in the emergency department

Clin Exp Emerg Med

(2014)

S.Y. Hwang et al.

Association between hemodynamic presentation and outcome in sepsis patients

Shock

(2014)

K. Jeon et al.

Improvements in compliance with resuscitation bundles and achievement of end points after an educational program on the management of severe sepsis and septic shock

Shock

(2012)

Cited by (84)

The prognostic value of sepsis scores and dichotomized triage score in patients presenting to the emergency department with fever: A prospective, observational study
2022, International Emergency Nursing
The performance of the Quick Sequential Organ Failure Assessment (qSOFA) score needs to be explored further in the emergency triage room. This study aims to explore the performance of triage (tqSOFA) versus the dichotomized triage score (DTS) in patients admitted to the emergency room triage with fever.
This research was designed as a prospective, observational study within a six-month period, including patients who presented to the emergency room triage with infrared fever ≥ 37.5 °C.
771 patients were analyzed.The highest sensitivity for predicting overall hospitalization and intensive care admission was seen for DTS (95.4 %, 100 %; p < 0.0001, p < 0.0001, respectively) (AUC:0.697, 95 % CI 0.663 to 0.730; AUC:0.684, 95 % CI 0.650 to 0.717, respectively). The highest sensitivity for predicting 1st week and 1st month mortality was found for DTS (100 %, 96.3 %; p < 0.0001, p < 0.0001, respectively). However, the highest specificity for predicting 1st week and 1st month mortality was observed in tqSOFA (94.1 %, 95.16; p = 0.0845, p < 0.0001, respectively) (AUC:0.658, 95 % CI 0.623 to 0.691; AUC:0.698, 95 % CI 0.664 to 0.730, respectively).
We found DTS to be as effective as tqSOFA and SIRS in determining all hospitalization times and mortality.
Comparison of qSOFA, SIRS, and NEWS scoring systems for diagnosis, mortality, and morbidity of sepsis in emergency department
2021, American Journal of Emergency Medicine
This study was aimed to compare the quick Sequential Organ Failure Assessment (qSOFA), Systemic Inflammatory Response Syndrome (SIRS), and National Early Warning Score (NEWS) scoring systems for diagnosing sepsis and predicting mortality and morbidity.
A prospective study was designed. qSOFA, SIRS, and NEWS scores were calculated at the admission. The diagnosis of sepsis was made with SOFA scoring initially. The morbidity and mortality of the patients were identified during follow-up. Also, the sensitivity, specificity, negative predictive value, and positive predictive value of three scoring systems were calculated. The scoring systems were compared with ROC analysis.
A total of 463 patients were evaluated. There were 287 (62.0%) patients diagnosed with sepsis, and septic shock occurred in 64 (13.8%) of patients. Seven-day mortality rate was 8.4% (n = 39), 30-day mortality rate was 18.1% (n = 84). The sensitivity for qSOFA, SIRS, and NEWS for diagnosis of sepsis was 23%, 77%, 58%, and specificity was 99%, 35%, 81% respectively. The sensitivity of the qSOFA, SIRS and NEWS scoring systems for mortality was 39%, 82%, 77% and specificity 91%, 29%, and 64%, respectively. AUROC values for mortality detected as NEWS = 0.772, qSOFA = 0.758, SIRS = 0.542. According to the ROC analysis, the SIRS system was significantly less useful than the qSOFA and NEWS system in the diagnosis of sepsis and mortality (p < 0.0001).
NEWS and qSOFA scoring systems have similar prognosis in both diagnosing sepsis and predicting mortality and both are superior to SIRS.
Clinical prediction rule is more useful than qSOFA and the Sepsis-3 definition of sepsis for screening bacteremia
2021, American Journal of Emergency Medicine
Citation Excerpt :
Early identification of sepsis and bacteremia, i.e., associated with mortality, is important for the medical decision-making of bacterial infection, and obtaining blood cultures is recommended before the initial administration of antibiotics [1]; however, identifying which patients require blood cultures is difficult. Quick-SOFA, a screening tool for patients likely to have sepsis that was introduced in 2016, has been reported to have high specificity in the early detection of sepsis and in-hospital mortality [5,16,17]; however, qSOFA was criticized for its low sensitivity in identifying sepsis and in-hospital mortality [18-21]. Furthermore, relationships between qSOFA and Sepsis-3 definition of sepsis, and bacteremia are not well discussed [11,17].
Clinical guidelines recommend blood cultures for patients suspected with sepsis and bacteremia. Sepsis-3 task force introduced the new definition of sepsis in 2016; however, the relationship between the Sepsis-3 definition of sepsis and bacteremia remains unclear. This study aimed to investigate how to detect patients who need blood cultures.
Consecutive patients who visited the emergency department in our hospital with suspected symptoms of bacterial infection and with collected blood culture were retrospectively examined between April and September 2019. The relationship between bacteremia and Sepsis-3 definition of sepsis, and the relationship between bacteremia and clinical scores (quick-Sequential Organ Failure Assessment [qSOFA], systematic inflammatory response syndrome [SIRS], and Shapiro's clinical prediction rule) were investigated. In any scores used, ≥2 points were considered positive.
Among the 986 patients who met the inclusion criteria, 171 (17%) were complicated with bacteremia and 270 (27%) were patients with sepsis. Sepsis was more frequent (61% vs. 20%, P < 0.001) and all clinical scores were more frequently positive in patients with bacteremia than in those without (qSOFA, 23% vs. 9%; SIRS, 72% vs. 58%; Shapiro's clinical prediction rule, 88% vs. 49%; P < 0.001). Specificity to predict bacteremia was high in sepsis and positive qSOFA (0.80 and 0.91, respectively), whereas sensitivity was high in positive SIRS and Shapiro's clinical prediction rule (0.72 and 0.88, respectively); however, no clinical definitions and scores had both high sensitivity and specificity. The area under the receiver operating characteristic curves were 0.59 (95% confidence interval, 0.55–0.64), 0.60 (0.56–0.65), and 0.78 (0.74–0.82) in qSOFA, SIRS, and Shapiro's clinical prediction rule, respectively.
Blood cultures should be obtained for patients with sepsis and positive qSOFA because of its high specificities to predict bacteremia; however, because of low sensitivities, Shapiro's clinical prediction rule can be more efficiently used for screening bacteremia.
Supervised classification techniques for prediction of mortality in adult patients with sepsis
2021, American Journal of Emergency Medicine
Sepsis mortality is still unacceptably high and an appropriate prognostic tool may increase the accuracy for clinical decisions.
To evaluate several supervised techniques of Artificial Intelligence (AI) for classification and prediction of mortality, in adult patients hospitalized by emergency services with sepsis diagnosis.
Secondary data analysis of a prospective cohort in three university hospitals in Medellín, Colombia. We included patients >18 years hospitalized for suspected or confirmed infection and any organ dysfunction according to the Sepsis-related Organ Failure Assessment. The outcome variable was hospital mortality and the prediction variables were grouped into those related to the initial clinical treatment and care or to the direct measurement of physiological disturbances. Four supervised classification techniques were analyzed: the C4.5 Decision Tree, Random Forest, artificial neural networks (ANN) and support vector machine (SVM) models. Their performance was evaluated by the concordance between the observed and predicted outcomes and by the discrimination according to AUC-ROC.
A total of 2510 patients with a median age of 62 years (IQR = 46–74) and an overall hospital mortality rate of 11.5% (n = 289). The best discrimination was provided by the SVM and ANN using physiological variables, with an AUC-ROC of 0.69 (95%CI: 0.62; 0.76) and AUC-ROC of 0.69 (95%CI: 0.61; 0.76) respectively.
Deep learning and AI are increasingly used as support tools in clinical medicine. Their performance in a syndrome as complex and heterogeneous as sepsis may be a new horizon in clinical research. SVM and ANN seem promising for improving sepsis classification and prognosis.
Failure of vital sign normalization is more strongly associated than single measures with mortality and outcomes
2020, American Journal of Emergency Medicine
Modified Early Warning Systems (MEWS) scores offer proxies for morbidity and mortality that are easily acquired, but there are limited data on what changing MEWS scores within the ED indicate. We examined the correlation of changing MEWS scores during resuscitation in the ED and in-hospital morbidity and mortality.
We conducted a retrospective analysis on medical ED patients with simplified MEWS scores (without urine output or mental status) admitted to a single academic tertiary care center over one year. Triage-to-Last delta MEWS score and Triage-to-Max delta MEWS scores were calculated and correlated to in-hospital mortality, ICU admission, length of stay (LOS) and diagnosis of sepsis.
Our analysis included 8322 ED patients with an ICU admission rate of 17% and a mortality rate of 2%. Every point of worsened MEWS after triage was more strongly associated with all-cause mortality (OR 2.41, 95% CI 1.96–2.97) than triage MEWS alone (OR 1.33, 95% CI 1.23–1.44; p < 0.001). Likewise, each point of worsened MEWS was associated with increased odds of ICU admission (Triage-to-Last: OR 2.12, 95% CI 1.92–2.33 and Triage-to-Max: OR 1.52, 95% CI 1.45–1.60, respectively). Among patients with suspected infection, similar associations are found.
Dynamic vital signs in the emergency department, as categorized by delta MEWS, and failure to normalize abnormalities, were associated with increased mortality, ICU admission, LOS, and the diagnosis of sepsis. Our results suggest that MEWS scores that do not normalize, from triage onward, are more strongly associated with outcome than any single score.
Prognostic accuracy of qSOFA score, SIRS criteria, and EWSs for in-hospital mortality among adult patients presenting with suspected infection to the emergency department (PASSEM) Multicenter prospective external validation cohort study protocol
2024, PLoS ONE

View all citing articles on Scopus

: Please see page 2 for the Editor’s Capsule Summary of this article.

: Supervising editor: Alan E. Jones, MD

: Author contributions: TGS and SYH conceived the study, designed the data collection instrument, and managed the database. IJJ, WCC, and MSS performed statistical analysis in the study. SYH, HY, SUL, and TRL contributed to the drafting of the article. TGS, SYH, and HY contributed to revisions of the article. TGS takes responsibility for the paper as a whole.

: All authors attest to meeting the four ICMJE.org authorship criteria: (1) Substantial contributions to the conception or design of the work; or the acquisition, analysis, or interpretation of data for the work; AND (2) Drafting the work or revising it critically for important intellectual content; AND (3) Final approval of the version to be published; AND (4) Agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

: Funding and support: By Annals policy, all authors are required to disclose any and all commercial, financial, and other relationships in any way related to the subject of this article as per ICMJE conflict of interest guidelines (see www.icmje.org). The authors have stated that no such relationships exist.

: Readers: click on the link to go directly to a survey in which you can provide feedback to Annals on this particular article.

: A podcast for this article is available at www.annemergmed.com.

: Continuing Medical Education exam for this article is available at http://www.acep.org/ACEPeCME/.

View full text

Infectious disease/Original researchLow Accuracy of Positive qSOFA Criteria for Predicting 28-Day Mortality in Critically Ill Septic Patients During the Early Period After Emergency Department Presentation

Study objective

Methods

Results

Conclusion

Introduction

Editor’s Capsule Summary

Section snippets

Study Design and Setting

Characteristics of Study Subjects

Limitations

Discussion

Chest

Ann Emerg Med

Ann Emerg Med

Chest

Am J Emerg Med

J Emerg Med

Resuscitation

Epidemiology of severe sepsis in the United States: analysis of incidence, outcome, and associated costs of care

Crit Care Med

The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3)

JAMA

Assessment of clinical criteria for sepsis: for the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3)

JAMA

Comprehensive interpretation of central venous oxygen saturation and blood lactate levels during resuscitation of patients with severe sepsis and septic shock in the emergency department

Shock

Impact of timely antibiotic administration on outcomes in patients with severe sepsis and septic shock in the emergency department

Clin Exp Emerg Med

Association between hemodynamic presentation and outcome in sepsis patients

Shock

Improvements in compliance with resuscitation bundles and achievement of end points after an educational program on the management of severe sepsis and septic shock

Shock

Infectious disease/Original research
Low Accuracy of Positive qSOFA Criteria for Predicting 28-Day Mortality in Critically Ill Septic Patients During the Early Period After Emergency Department Presentation