Computers in emergency medicine
The Soterion Rapid Triage System: Evaluation of inter-rater reliability and validity

https://doi.org/10.1016/j.jemermed.2005.05.037Get rights and content

Abstract

The Soterion Rapid Triage System is a new, computerized, five-level triage acuity system. The purpose of this study was to evaluate the system’s inter-rater reliability and validity for stratifying patient acuity. The study was comprised of two components. A prospective evaluation of inter-rater reliability was conducted by the blinded, paired simultaneous triage of 423 patients. A retrospective evaluation of validity consisted of the analysis of 33,850 patients triaged with the system over an 8-month period. The system’s validity was measured by in-hospital admission rate, Emergency Department (ED) length-of-stay, hospital charges and Current Procedural Terminology (CPT) Codes 99281–99285. Evaluation of inter-rater reliability demonstrated a weighted kappa of 0.87 (95% confidence interval 0.84, 0.91). The in-hospital admission rates for patients triaged as Level 1 Immediate–Level 5 Non-Urgent were 43%, 30%, 13%, 3% and 1%, respectively (p < 0.0001). Similarly, there were significant differences in the means for all hospital charges, ED lengths-of-stay, and CPT Codes. In conclusion, the Soterion Rapid Triage System possesses high inter-rater reliability and validity. The system’s reliability and validity, and the availability of the system’s electronically archived data are characteristics beneficial to the development of a national standardized five-level triage acuity system.

Introduction

Triage, or the “sorting” of patients, is a critical function for the timely delivery of emergency health care and the appropriate allocation of medical resources. Without well-defined guidelines, triage is dependent upon the skill, education and experience of the individual performing triage. Australia, Canada and England have each adopted national five-level triage acuity systems due to the recognized need for a more uniform triage tool and the demonstration of greater inter-rater reliability with a five-level system as compared to previous three- and four-level systems (1, 2, 3, 4, 5, 6, 7).

The need for a uniform triage tool has been underscored by the American College of Emergency Physicians and the Emergency Nurses Association’s recent approval of resolutions for the development of a national five-level triage acuity system for the United States (8, 9). A national triage acuity system may provide the data necessary for determining whether a system provides timely and safe access to emergency care, allows for greater comparison of case mix between hospitals, and promotes the development of health care policy pertaining to the allocation of medical resources stemming from the emergency department care of patients.

The Soterion Rapid Triage System (SRTS) is a new, computerized, algorithm-driven five-level triage acuity system. The use of algorithms has the potential benefit of minimizing the variability of the triage decision-making process. The computerized nature of the system allows for the electronic capture and retrieval of all data acquired during the triage process, facilitating future use of this information in a national triage database.

The purpose of this study was to determine whether the SRTS possesses a high degree of inter-rater reliability and whether the system successfully stratifies patients of different acuity based on several acuity outcome measures.

Section snippets

Methods

The SRTS is a stand-alone, web-based computer application developed using Microsoft Visual Studio.NET Professional Version software. The database for the SRTS was created using the relational database management and analysis system SQL Server 2000. Authorized access to the software application was accomplished through a local intra-net web server using Internet Explorer version 6.0.

The SRTS is based on a five-level triage acuity scale. Level 1 conditions are defined as life- or limb-threatening

Characteristics of the Study Subjects

During the time period between April 12, 2003 and December 12, 2003, there were a total of 38,763 ED patient visits. Of those, 2587 (7%) were non-ambulatory ambulance patients and 1989 (5%) were deemed ill enough to warrant being brought back immediately without undergoing triage by the SRTS. There were 337 patients (< 1%) who left the ED without being seen. The remaining 33,850 patients (87%) were triaged by the SRTS.

The characteristics of the study subjects triaged by the SRTS are shown in

Discussion

In 1999, the American College of Emergency Physician’s (ACEP’s) Subcommittee on the National Triage Scale published the informational paper, “A Uniform Triage Scale in Emergency Medicine” (13). This document provided background on the history of triage, a comparison of existing standardized triage scales, the characteristics of an ideal triage scale, and the benefits to having a national standardized triage scale. The authors recommended that the ideal triage scale should possess the

Limitations

Our study excluded non-ambulatory ambulance patients as well as those patients that the triage nurse felt appeared ill enough that they needed to be brought back immediately without undergoing triage by the SRTS. These two groups represented 12% of the total ED visits, of whom 47% (2134/4576) were admitted. If patients in these two groups underwent triage by the SRTS and were included in the study, the percentage of patients in the higher acuity levels would have been larger and the differences

Conclusions

We have demonstrated that the Soterion Rapid Triage System possesses high inter-rater reliability and validity. In addition, the system’s functionality, effectiveness in clinical practice, and the availability of the system’s electronically archived data are characteristics beneficial to the development of a national standardized five-level triage acuity system.

Acknowledgments—

Dr. Maningas, Mr. Hime, and Dr. McMurry are partners of Soterion L.L.C., which developed the software application. Dr. Maningas is the inventor to whom Patent No. 6,786,406 B1 has been granted.

References (27)

  • Emergency triage

    (1996)
  • ACEP Policy Statement. Triage scale standardization. 2003. Available at: http://acep.org/1,33178,0.html. Accessed April...
  • Emergency Nurses Association. 2002 General Assembly. Resolution 02-02. National five-level triage...
  • Cited by (28)

    • The Canadian Triage and Acuity Scale for children: A prospective multicenter evaluation

      2012, Annals of Emergency Medicine
      Citation Excerpt :

      Also, nurses are invited to receive formal training provided by a certified Canadian Triage and Acuity Scale instructor before performing triage with the scale. Despite recognition of the importance of appropriate validation and reliability testing of triage tools,15-17 there is paucity of prospective studies assessing validity and reliability of triage tools in children.18-32 Most studies were performed at a single center and using written scenarios, which may inadequately incorporate the clinical appearance of the patient.18-32

    • Managing patient flow with triage streaming to identify patients for Dutch emergency nurse practitioners

      2012, International Emergency Nursing
      Citation Excerpt :

      To evaluate the validity of triage systems, researchers have used different proxy reference standards for prognosis. Amongst others, these include: length of stay (LOS), admission rate, intensive care admission rate, hospital charges, mortality, resource use, and a combination of vital signs at presentation, potentially life threatening conditions, diagnostic resources, therapeutic interventions and follow up, (Cooke and Jinks, 1999; Wuerz et al., 2000; Maningas et al., 2006; Worster et al., 2007; van Veen et al., 2008; Travers et al., 2009). Since resource use and admission rates are associated with the complexity and severity of a patient’s status, these parameters have been used as validity criteria in many studies (Wuerz et al., 2000; Tanabe et al., 2004; Roukema et al., 2006; Taboulet et al., 2009).

    View all citing articles on Scopus

    Computers in Emergency Medicine is coordinated by James Killeen, md, of the University of California San Diego Medical Center, San Diego, California

    View full text