An Electronic Emergency Triage System to Improve Patient Distribution by Critical Outcomes

doi:10.1016/j.jemermed.2016.02.026

The Journal of Emergency Medicine

Volume 50, Issue 6, June 2016, Pages 910-918

https://doi.org/10.1016/j.jemermed.2016.02.026 Get rights and content

Abstract

Background

Patient triage is necessary to manage excessive patient volumes and identify those with critical conditions. The most common triage system used today, Emergency Severity Index (ESI), focuses on resources utilized and critical outcomes.

Objective

This study derives and validates a computer-based electronic triage system (ETS) to improve patient acuity distribution based on serious patient outcomes.

Methods

This cross-sectional study of 25,198 (97 million weighted) adult emergency department visits from the 2009 National Hospital Ambulatory Medical Care Survey. The ETS distributes patients by using a composite outcome based on the estimated probability of mortality, intensive care unit admission, or transfer to operating room or catheterization suite. We compared the ETS with the ESI based on the differentiation of patients, outcomes, inpatient hospitalization, and resource utilization.

Results

Of the patients included, 3.3% had the composite outcome and 14% were admitted, and 2.52 resources/patient were used. Of the 90% triaged to low-acuity levels, ETS distributed patients evenly (Level 3: 30%; Level 4: 30%, and Level 5: 29%) compared to ESI (46%, 34%, and 7%, respectively). The ETS better-identified patients with the composite outcome present in 40% of ETS Level 1 vs. 17% for ESI and the ETS area under the receiver operating characteristic curve (AUC) was 0.83 vs. ESI 0.73. Similar results were found for hospital admission (ETS AUC = 0.83 vs. ESI AUC = 0.72). The ETS demonstrated slight improvements in discriminating patient resource utilization.

Conclusions

The ETS is a triage system based on the frequency of critical outcomes that demonstrate improved differentiation of patients compared to the current standard ESI.

Introduction

Increasing patient visits and decreasing capacity due to closing facilities has led a growing number of patients to experience significant wait times to receive medical evaluation and treatment (1). Despite the move to early time to a provider, patient triage is necessary to effectively manage excessive volumes of patients and identify patients with critical and time-sensitive conditions (e.g., myocardial ischemia or sepsis) from those with less-urgent needs (e.g., indigestion or minor infections). Although triage decisions are straightforward for very high- or low-severity cases; the projected clinical course for the majority of patients is not obvious. Inability to quickly distinguish significantly ill patients (i.e., under-triage) can delay time-sensitive treatment and lead to deterioration, morbidity, and mortality 2, 3, 4, 5. Over-triaged patients consume limited resources that might be directed more appropriately to those with higher-acuity illness 6, 7. As emergency care demands higher efficiency to manage growing patient volumes, an accurate and evidence-based triage system is required to provide safe and optimal care.

Triage has been a long-standing principle in emergency medicine, but standardized triage tools are relatively new. Canada, Australia, and the United Kingdom have created their own triage instruments and, in the United States, 72% of emergency department (ED) patient visits are assessed using the Emergency Severity Index (ESI) 6, 8, 9, 10, 11. ESI is composed of a series of 3 questions used to assign patients to one of five acuity levels. The ESI triage process relies on experienced nurses' judgments to assess patients according to the following questions: 1) Is the patient dying? 2) Should the patient wait? and 3) How many resources will this patient require? (6) Patients dying are categorized to Level 1 (immediate treatment); patients who should not wait are categorized to Level 2 (emergent treatment); and patients deemed safe to wait are stratified to Levels 3 (urgent treatment) through 5 (nonurgent treatment) based on anticipated resource utilization. Level 3 patients are expected to use the most resources (more than two resources) followed by Level 4 (one resource) and Level 5 (no resources). The triage provider can also re-categorize Level 3 patients up to Level 2, based on abnormal vital signs. ESI has been validated by its developers to outcomes of hospital admission and ED resource use 12, 13, 14, 15. Overall, the ESI tool stratifies patients based on nurses' experience and “sixth sense” for immediacy of medical need and resource utilization (6). Including resource utilization to determine a triage level makes the system unique among modern triage systems.

Although currently in widespread use within the United States, ESI has several shortcomings 14, 16, 17. Foremost, it does not sufficiently discriminate and distribute patients across its five triage levels, with almost half of all ED patients nationally assigned to acuity Level 3 18, 19. This results in patients with a wide range of illness severity clustered to one large group, potentially delaying care to those most severely ill. This seems to counter the true objective of triage by not differentiating a majority of patient visits and, therefore, creates challenges in efficient resource distribution. Next, ESI has not been adequately validated against time-sensitive or critical care outcomes, which are important for an effective triage system. Finally, ESI relies heavily on subjectivity at triage and might be limited by untoward variability that can adversely affect patients through nurse inexperience, human error, or even systemic flaws in triage assessment.

To address these deficiencies, we utilized a large nationally representative sample of ED patient visits to develop an automated, computer-based, electronic triage system (i.e., ETS) designed to improve patient differentiation objectively based on the risk of critical patient outcomes. The ETS uses simple standardized patient information routinely collected at triage to predict risk for critical outcomes and distributes patients among five triage levels based on estimated risk. The ETS and ESI are then evaluated using measures targeted by both the ETS (distribution of patients and risk of critical outcomes) and ESI (risk of inpatient hospitalization and ED resource utilization).

Section snippets

Study Design and Setting

This is a retrospective cohort study of patient visits included in the 2009 National Hospital Ambulatory Medical Care Survey (NHAMCS). NHAMCS is an annually collected, nationally representative probability sample survey of ED visits conducted by the Centers for Disease Control and Prevention's (CDC) National Center for Health Statistics (20). This study utilizes a pre-existing, publicly available, de-identified database; therefore, no additional participant consent or Institutional Review Board

Results

The ETS predicts an ED patient's probability of the composite outcome (mortality, admission to the ICU, or direct transport to the OR or cardiovascular catheterization suite) using routinely available information obtained at ED triage. Of the 97 million ED visits included, 3.3% had the composite outcome (0.5% in-hospital mortality, 2.2% admitted to an ICU, 0.5% transferred to an OR, and 0.1% transferred to a cardiovascular catheterization suite). Factors associated with increased likelihood of

Discussion

The ETS is a newly proposed electronic triage instrument that differentiates patients based on critical and time-sensitive outcomes. It can be easily incorporated into any electronic medical record system and immediately calculate a score once common patient information has been put into the system. The ETS was developed using a nationally representative sample of 97 million ED visits accounting for geographic and facility variability, maximizing its potential for generalized application across

Conclusions

The ETS is a novel electronic triage system that uses commonly obtained triage information to automatically distribute adult ED patients across five severity levels based on critical and time-sensitive outcomes (mortality, admission to the ICU, or direct transport to the OR or cardiovascular catheterization suite). The algorithm was derived and validated using a nationally representative sample of 25,198 adult patient visits (97 million visits weighted) collected by the CDC. Compared to ESI,

Acknowledgments

This work was supported by the Department of Homeland Security (PACER: National Center for Study of Preparedness and Response [2010-ST-061-PA0001]).

References (30)

B. Haas et al.
Survival of the fittest: the hidden cost of undertriage of major trauma
J Am Coll Surg
(2010)
I. Yurkova et al.
Under-triage as a significant factor affecting transfer time between the emergency department and the intensive care unit
J Emerg Nurse
(2011)
P. Tanabe et al.
The Emergency Severity Index (version 3) 5-level triage system scores predict ED resource consumption
J Emerg Nurs
(2004)
C. Fernandes et al.
Five-level triage: a report from the ACEP/ENA Five-Level Triage Task Force
J Emerg Nurs
(2005)
The Future of Emergency Care in the United States Health System
Ann Emerg Med
(2006)
N. Seiger et al.
Undertriage in the Manchester triage system: an assessment of severity and options for improvement
Arch Dis Child
(2011)
M. Hitchcock et al.
Triage: an investigation of the process and potential vulenerabilities
J Adv Nurs
(2013)
Emergency Severity Index, Version 4: implementation handbook. AHRQ Publication No. 12–0014. Agency for Healthcare...
S.L. Dong et al.
Comparing a novel computer triage program with standard triage
Acad Emerg Med
(2005)
M. McHugh et al.
More patients are triaged using the Emergency Severity Index than any other triage acuity system in the United States
Acad Emerg Med
(2012)

R. Meek et al.

Australian Triage Scale: consumer perspective

Emerg Med Australas

(2005)

Emergency triage: Manchester Triage Group

(1997)

R. Beveridge et al.

Canadian emergency department triage and acuity scale; implementation guidelines

Can J Emerg Med

(1999)

D.R. Eitel et al.

The emergency severity index triage algorithm version 2 is reliable and valid

Acad Emerg Med

(2003)

R.C. Wuerz et al.

Reliability and validity of a new five-level triage instrument

Acad Emerg Med

(2000)

Cited by (91)

Developing a decision model to early predict ICU admission for COVID-19 patients: A machine learning approach
2024, Intelligence-Based Medicine
Emergency department (ED) overcrowding is a significant problem in the US. This paper develops a decision model to mitigate ED overcrowding by helping hospitals proactively plan patient boarding processes. The information obtained after the initial assessment of COVID-19 patients in the ED, including patient demographics and medical history, is utilized to predict ICU admission earlier. The predicted information can be communicated with the inpatient unit to prepare an ICU bed for the patients who need ICU care. As a result, the boarding time when patients wait for an ICU bed to be ready can be reduced. The data used in this study included 100 features and 19,155 COVID-19 patients from an academic medical center located in the Southeast United States. Multiple feature selection methods along with Extreme Gradient Boosting (XGBoost) were utilized to develop the models. The parameters of the XGBoost models are optimized using simulated annealing (SA). Among the proposed models, the best model included ten features and resulted in an area under the curve (AUC) of 89.2%, which is the highest among the models proposed in the literature. The proposed prediction model allows hospital administrators to allocate ICU beds more efficiently, enhance patient flow, and mitigate ED overcrowding.
Delays in treatment and disposition attributable to undertriage of pediatric emergency medicine patients
2023, American Journal of Emergency Medicine
Triage, the initial assessment and sorting of patients in the Emergency Department (ED), determines priority of evaluation and treatment. Little is known about the impact of undertriage, the underestimation of disease severity at triage, on clinical care in pediatric ED patients. We evaluate the impact of undertriage on time to disposition and treatment decisions in pediatric ED patients.
This was a case control study of ED visits for patients <22 years of age, with an assigned Emergency Severity Index (ESI) score of 4 or 5, and associated hospital admission, nebulized treatment, supplemental oxygen, and/or intravenous (IV) line placement, between January 1, 2018, to June 30, 2022. Controls were sampled from a pool of patient visits with an ESI score of 3, matched by intervention, disposition, and date and hour of arrival.
Primary outcome measures were time to order of intervention (nebulized treatment, oxygen administration, or IV placement) and time to disposition decision. A secondary outcome measure was return visits requiring admission or emergency intervention within 14 days of the index visit. Continuous variables (time to orders) were analyzed using Wilcoxon rank sum test and dichotomous outcomes (return visits) were compared using odds ratios with 95% confidence intervals. Analysis was performed with Python v3.10.
The final analysis included 7245 undertriaged patients. Undertriaged patients had longer times to orders for nebulized treatments, (p < 0.001) IV placement, (p < 0.001) and admission (p < 0.001) when compared to controls. There were no significant differences in time to supplemental oxygen delivery and time to discharge compared to controls. Undertriaged patients were more likely to experience a return visit requiring admission or emergency intervention (OR 3.74, 95% CI 3.32,4.22).
Undertriage in the pediatric ED is associated with delays in care and disposition decisions and increases likelihood of return visits.
Criticality and clinical department prediction of ED patients using machine learning based on heterogeneous medical data
2023, Computers in Biology and Medicine
Emergency triage faces multiple challenges, including limited medical resources and inadequate manual triage nurses, which cause incorrect triage, overcrowding in the emergency department (ED), and long patient waiting time.
This paper aims to propose and validate an accurate and efficient artificial intelligence-based method for effectively ED triage and alleviating the pressure on medical resources.
We propose two novel machine learning models, TransNet and TextRNN, for predicting patient severity levels and clinical departments using heterogeneous medical data in ED triage. Our models employ a parallel structure for feature extraction and incorporate an attention mechanism to extract essential information from the fused features, enabling accurate predictions. The models analyze the triage data (2020–2022) from the ED of Beijing University People's Hospital, incorporating variables (demographics, triage vital signs, and chief complaints) to identify patient severity levels and clinical departments. We performed data cleaning, categorization, and encoding first. Then, we divided the available data into a training set (56%), a validation set (24%), and a test set (20%) by random sampling. Finally, our models underwent 5-fold cross-validation and were compared with other state-of-the-art models.
We comprehensively evaluated the proposed models against various Recurrent Neural Networks (RNN), Convolutional Neural Networks (CNN), Traditional Machine Learning (TML), and Transformer-based (TF) models, achieving excellent performance in predicting triage outcomes. Specifically, TextRNN achieved a prediction success rate of 86.23% [85.86–86.70] for severity levels and 94.30% [94.00–94.46] for clinical departments among 161,198 ED visits. Moreover, TransNet demonstrated higher sensitivities of 84.08% and 90.05% for severity levels and clinical departments, respectively, with specificities of 76.48% and 95.16%. The accuracy of our model is 0.87%, 0.18%, 4.29%, and 1.96%, higher than that of the above four family models on average. Furthermore, our method significantly reduced under-triage by 12.06% and over-triage by 17.92% compared to manual triage.
Experimental results demonstrated that the proposed models fuse heterogeneous medical data in the triage process, successfully predicting patients’ triage outcomes. Our models can improve triage efficiency, reduce the under/over-triage rate, and provide physicians with valuable decision-making support.
Effect of triage training on nurses' practice and triage outcomes of patients with acute coronary syndrome
2023, International Emergency Nursing
Accurate assessment and prompt management of patients with acute coronary syndrome (ACS) is a complex process for emergency department (ED) nurses and has variable clinical outcomes. The aim of the present study was to determine the effectiveness of an educational intervention on nurses' practice during the triage of patients with ACS and the triage outcomes in this group of patients.
In this quasi-experimental study, a pretest-posttest group of 24 nurses were included by convenience sampling method and 960 patients with ACS were selected by sequential sampling during the pre-intervention (n = 480) and post-intervention (n = 480) phases. A case-based learning (CBL) intervention was performed for nurses for one month considering the role of the triage nurse according to the American College of Cardiology (ACC) and the American Heart Association (AHA) recommendations as well as the factors affecting the proper identification and management of patients with ACS. During patient triage in the pre- and post-intervention phases, the “Triage Nurse Practice Checklist” and the “Medical Electronic Records” were used to assess nurses' practice and the triage outcomes in patients, respectively.
The overall mean score of the triage nurses' practice and its subscales, including Primary monitoring and assessment, cardiovascular risk factors assessment, evaluation of coronary heart disease (CHD) symptoms, chest pain management, and adherence to the ACC/AHA practice guidelines were significantly improved in the post-intervention phase compared with the pre-intervention phase (p < 0.001). There was no significant difference between the triage outcomes, including in-hospital mortality within 24 hours, death in ED, hospitalization in other wards, and discharge from ED in the pre and post-intervention phases (P = 0.723).
The development of a cardiac triage-specific educational program could improve the performance of nurses in the evaluation and management of patients with ACS, but had no effect on the triage outcomes in this group of patients. We recommend a quality improvement project or a critical outcomes-based triage system to assess ACS patients’ care needs in the ED.
Factors affecting the triage decision-making ability of emergency nurses in Northern China: A multi-center descriptive survey
2023, International Emergency Nursing
The quality of triage decision-making is a prerequisite for priority treatment of critically ill patients and effective utilization of medical resources. Figuring out how to improve triage decision-making is still a topic around the global emergency department. Hence, this study aims to promote an understanding of triage priority care and clarify the elements influencing triage decision-making ability, offering reference for the future to improve the quality of triage decision-making.
A total of 404 emergency nurses from 11 tertiary hospitals in northern China were surveyed by questionnaire, of which 371 valid questionnaires were submitted (effective rate = 91.83 %). One hospital distributed the questionnaire face-to-face, and the other ten used online form.
Prior to occupying triage jobs, only a quarter of participants(25.30 %)were qualified. Less than half of emergency nurses (46.60 %) reported taking part in the triage training program. The emergency nurses' triage decision-making ability score was 166.50 ± 26.90(95 %CI 163.75,169.24) in northern China. Gender(P = 0.003), case discussion(P = 0.024), secondary assessment(P = 0.020)and knowledge of triage consensus(P = 0.027) are independent factors influencing triage decision-making ability.
Emergency triage practices are less implemented in northern China. The triage decision-making ability of emergency nurses in northern China is at a low level. Providing emergency nurses with diverse opportunities to develop their triage skills, finding effective triage training content, form, and frequency, strengthening implement triage consensus, and wisely managing triage nurse resources would improve triage decision-making.
A comparative study of supervised machine learning approaches to predict patient triage outcomes in hospital emergency departments
2023, Array
The inconsistency in triage evaluation in emergency departments (EDs) and the limitations in practice within the standard triage tools among triage nurses have led researchers to seek more accurate and robust triage evaluation that provides better patient prioritization based on their medical conditions. This study aspires to establish the best methodological practices for applying machine learning (ML) techniques to build an automated triage model for more accurate evaluation.
A comparative study of selected supervised ML models was conducted to determine the best-performing approach to evaluate patient triage outcomes in hospital emergency departments. A retrospective dataset of 2688 patients who visited the ED between April 1, 2020 and June 9, 2020 was collected. Data included patient demographics (age and gender), Vital signs (body temperature, respiratory rate, heart rate, blood pressure and oxygen saturation), chief complaints, and chronic illness. Nine supervised ML techniques were investigated in this study. Models were trained based on patient disposition outcomes and then validated to evaluate their performance.
ML models show high capabilities in predicting patient disposition outcomes in ED settings. Four models (KNN, GBDT, XGBoost, and RF) performed better than the rest. RF was selected as the optimal model as it demonstrated a slight advantage over the other models with 89.1% micro accuracy, 89.0% precision, 89.1% recall, and 89.0% F1-score, exhibiting outstanding performance in differentiation between patients with critical outcomes (e.g., Mortality and ICU admission) from those patients with less critical outcomes (e.g., discharged and hospitalized) in ED settings.
Machine learning techniques demonstrate high promise in improving predictive abilities in emergency medicine and providing robust decision-making tools that can enhance the patient triage process, assist triage personnel in their decision and thus reduce the effects of ED overcrowding and enhance patient outcomes.

View all citing articles on Scopus

View full text

Computers in Emergency MedicineAn Electronic Emergency Triage System to Improve Patient Distribution by Critical Outcomes

Abstract

Background

Objective

Methods

Results

Conclusions

Introduction

Section snippets

Study Design and Setting

Results

Discussion

Conclusions

Acknowledgments

J Am Coll Surg

J Emerg Nurse

J Emerg Nurs

J Emerg Nurs

The Future of Emergency Care in the United States Health System

Ann Emerg Med

Undertriage in the Manchester triage system: an assessment of severity and options for improvement

Arch Dis Child

Triage: an investigation of the process and potential vulenerabilities

J Adv Nurs

Comparing a novel computer triage program with standard triage

Acad Emerg Med

More patients are triaged using the Emergency Severity Index than any other triage acuity system in the United States

Acad Emerg Med

Australian Triage Scale: consumer perspective

Emerg Med Australas

Emergency triage: Manchester Triage Group

Canadian emergency department triage and acuity scale; implementation guidelines

Can J Emerg Med

The emergency severity index triage algorithm version 2 is reliable and valid

Acad Emerg Med

Reliability and validity of a new five-level triage instrument

Acad Emerg Med

Computers in Emergency Medicine
An Electronic Emergency Triage System to Improve Patient Distribution by Critical Outcomes