Review
A systematic review on the validity and reliability of an emergency department triage scale, the Manchester Triage System

https://doi.org/10.1016/j.ijnurstu.2014.01.013Get rights and content

Abstract

Objective

To conduct a systematic review to check the level of validity and reliability of the Manchester Triage System and the quality of reporting of literature on this topic.

Design

This is a systematic review based on the PRISMA guideline on reporting systematic reviews.

Data sources

The systematic search of the international literature published from 1997 through 30 November 2012 in the PubMed, Embase, Cochrane Library, Cinahl, Web of Knowledge, and Scopus databases.

Review methods

This review included quantitative and qualitative research investigating the reliability and validity of the Manchester Triage System for the broad population of adults and children visiting the emergency department.

After a systematic selection process, included studies were assessed on their quality by three researchers using the STARD guidelines.

Results

Twelve studies were included in the review. The studies investigated the inter- and intra-rater reliability using the “kappa” statistic; the validity was tested with many measures: validity in predicting mortality, hospital admission, under- and overtriage, used resources, and length of stay in the emergency department, as well as a reference standard rating.

Conclusions

In this review, the Manchester Triage System shows a wide inter-rater agreement range with a prevalence of good and very good agreement. Its safety was low because of the high rate of undertriage and the low sensitivity in predicting higher urgency levels. The high rate of overtriage could cause unnecessarily high use of resources in the emergency department. The quality of the reporting in studies of the reliability and validity of the Manchester Triage System is good.

Introduction

Triage is the first assessment and sorting process used to prioritize patients arriving in the emergency department (ED). The need to prioritize these patients is stressed by the considerable demand for emergency care, frequent ED overcrowding, and limited resources. For this reason many triage scales were developed in the last twenty years.

Most current triage tools actually in use are based on a categorical measurement acuity scale and have five levels. The Australasian Triage Scale, ATS (Standards Committee Council, 1994); the Canadian Triage and Acuity Scale, CTAS (Beveridge, 1998); the Manchester Triage System, MTS (Mackway-Jones, 1997); and the Emergency Severity Index, ESI (Eitel et al., 2003, Tanabe et al., 2004, Wuerz et al., 2000, Wuerz et al., 2001), are all five-level triage tools.

Many studies have evaluated the reliability and validity of acuity ratings by a triage nurse (Eitel et al., 2003, Parenti et al., 2009, Taboulet et al., 2009; Van der Wulp, 2008; Worster et al., 2007), probably because a triage scale should meet at least these two criteria to perform as intended (Twomey et al., 2007). In fact, according to most triage experts, (Fernandes et al., 2005) the “ideal” triage scale must demonstrate the characteristics of reliability and validity.

For this reason we decided to conduct a systematic review on the state of studies on the reliability and validity of the Manchester Triage Scale, as MTS is one of the most-used triage scales in Europe.

To our knowledge there are few reviews on all triage scales actually in use (Christ et al., 2010, Farrohknia et al., 2011, Van Veen and Moll, 2009), but there are no systematic reviews on the validity and reliability of the Manchester Triage System.

A previous review on triage methods suggested that the reliability, validity, and safety of triage scales needed to be investigated further (Farrohknia et al., 2011). Van Veen and Moll (2009) concluded that MTS and PaedCTAS seem valid to triage children in emergency care, as well as that reliability of MTS is good, moderate-to-good for ESI, moderate for PaedCTAS, and poor to moderate for ATS.

Section snippets

Background

The Manchester Triage System is an in-hospital triage scale usually used by nurses in many emergency departments in Great Britain and Europe since 1997 (Mackway-Jones, 1997, Martins et al., 2009, Olofsson et al., 2009, Van Baar et al., 2007). It is based on 52 flow-chart diagrams; each of the flow charts depicts six key discriminators (Mackway-Jones, 1997). MTS considers five triage priority levels: level 1, immediate assessment; level 2, very urgent; level 3, urgent; level 4, standard; level

Aim

The primary aim was to check the state of studies on the reliability and validity of the Manchester Triage Scale for the broad population of adults and children visiting the ED.

The questions for the review were:

  • (1)

    What is the level of reliability of MTS among the selected studies?

  • (2)

    How valid is MTS in predicting outcomes and workload among the selected studies?

  • (3)

    How is the quality of reporting among published studies on reliability and validity of MTS?

Review protocol

This is a systematic review with a narrative

Results

We collected 12 studies for final analysis (Table 1). Three studies met fewer than 50% of the STARD items and had a very low-quality methodology in the opinion of the authors (Table 2). None of the studies selected met all 25 items of the STARD guidelines (Table 2). Five of the nine studies met more than 80% of the STARD items. The study that met the greatest number of items on the STARD checklist was that of Van der Wulp et al. (2008) (Table 2). In the opinion of authors, the Van Veen et al.

Summary of evidence

The purpose of this review was to check the level of reliability and validity of MTS and to verify the quality of reporting among published studies on this topic. The results suggest that there is a wide inter-rater agreement range among all studies included, from fair to very good agreement. The wide inter-rater agreement range could likely be explained by the difference in populations studied, the difference among the raters who used MTS, and the difference in the way MTS was applied in each

Conclusion

In this review, MTS seems to have a wide range of inter-rater agreement. More studies should be conducted on its intra-rater reliability and on its validity in predicting workload (use of resources, and length of stay in the ED). Additionally, the use of MTS as a triage system for special patient categories (the elderly and young children) should be studied.

MTS should probably be changed, and educational programs should be designed to improve MTS safety. Particular care should be taken to

Conflict of interest

None declared.

References (33)

  • C. Fernandes et al.

    A report from the ACEP/ENA Five-level Triage Task Force

    J. Emerg. Nurs.

    (2005)
  • P. Olofsson et al.

    Manchester Triage in Sweden – interrater reliability and accuracy

    Int. Emerg. Nurs.

    (2009)
  • M.A. Rotondi et al.

    A confidence interval approach to sample size estimation for interobserver agreement studies with multiple raters and outcomes

    J. Clin. Epidemiol.

    (2012)
  • Australasian College for Emergency Medicine

    Guidelines on the Implementation of the Australasian Triage Scale in Emergency Departments

    (1999)
  • R. Beveridge

    CAEP issues The Canadian Triage and Acuity Scale: a new and critical element in health care reform. Canadian Association of Emergency Physician

    J. Emerg. Med.

    (1998)
  • P.M. Bossuyt et al.

    The STARD statement for reporting studies of diagnostic accuracy: explanation and elaboration

    Ann. Intern. Med.

    (2003)
  • M. Christ et al.

    Modern triage in the emergency department

    Dtsch. Arztebl. Int.

    (2010)
  • D.R. Eitel et al.

    The emergency severity index triage algorithm version 2 is reliable and valid

    Acad. Emerg. Med.

    (2003)
  • N. Farrohknia et al.

    Emergency department triage scale and their components: a systematic review of the scientific evidence

    Scand. J. Trauma Resusc. Emerg. Med.

    (2011)
  • S.W. Goodacre et al.

    Consistency of retrospective triage decisions as a standardised instrument for audit

    J. Accid. Emerg. Med.

    (1999)
  • A.I. Grouse et al.

    The Manchester Triage System provides good reliability in an Australian emergency department

    Emerg. Med. J.

    (2009)
  • J.R. Landis et al.

    The measurement of observer agreement for categorical data

    Biometrics

    (1977)
  • A. Liberati et al.

    The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration

    BMJ

    (2009)
  • K. Mackway-Jones

    Manchester Triage Group: Emergency Triage

    (1997)
  • H.M.G. Martins et al.

    Is Manchester (MTS) more than a triage system? A study of its association with mortality and admission to a large Portuguese hospital

    Emerg. Med. J.

    (2009)
  • N. Parenti et al.

    Reliability and validity of two four-level emergency triage systems

    Eur. J. Emerg. Med.

    (2009)
  • Cited by (0)

    View full text