Reliability and validity of the Toowoomba adult trauma triage tool: a Queensland, Australia study

https://doi.org/10.1016/j.aaen.2004.07.003Get rights and content

Summary

Introduction:

In Australia a nationally adopted five tiered triage scale called the Australasian triage scale (ATS) is used for the purpose of differentiating patient acuity levels for all patients that present to an emergency department (ED). The scale was formulated with the aim of promoting a standardized approach to triage. Numerous studies now suggest that the ATS has not been successful in achieving this intention. The Toowoomba adult trauma triage tool (TATTT) seeks to address this deficiency by providing an evidence based, reproducible, reliable and valid method of triage categorisation, albeit in a select group of patients.

Method:

Ten triage nurses from Toowoomba health service (THS) and five from Princess Alexandra Hospital in Queensland, Australia were enrolled in the study. Each participant rated five written simulations before receiving training in the TATTT. Each participant then rated nine written scenarios, five simulated scenarios on video and one computer simulated scenario with the TATTT. Parallel triaging of actual adult injury based presentations to THS ED was also undertaken.

Results:

The overall percentage agreement for all simulations for which data were available was 87%. The overall κ statistic was 0.82 with a 95% confidence interval from 0.76 to 0.88. The overall percentage agreement between the triage category assigned using existing triage practices and the triage category assigned by the TATTT was approximately 47% with an estimated κ of κˆ=0.19 (95% confidence interval [−0.02,0.40]).

Conclusion:

Evidence from this study suggests that the TATTT provides systematically different results compared to current triage practices utilising the ATS. This provided evidence that triage practices did not reflect current evidence or available descriptors. Additionally TATTT has greater reliability than current triage practices.

Introduction

Triage in an emergency department (ED) context constitutes the formal process of immediate assessment and categorisation of all patients who present seeking treatment (Commonwealth Department of Health and Family Services and the Australasian College of Emergency Medicine, 1997). In Australia a five tiered scale for the purpose of differentiating patient acuity levels, called the National triage scale (NTS), has been widely adopted to facilitate this process. The NTS was formulated in 1993 by the Australasian College for Emergency Medicine (ACEM) with the aim of promoting a standardized approach to triage in Australian EDs (Considine et al., 2001). In 2000, the NTS was revised and renamed the Australasian triage scale (ATS).

If the NTS was effective in providing a standardised approach to triage, a patient should be able to present to any triage nurse, in any ED, at any time of the day with a specific problem and be allocated the same triage category each time (Considine et al., 2000).

A number of studies have identified experimentally that this is unlikely to occur. For example, Doherty (1996) conducted a study into the uniformity of triage between medical and nursing staff and between four different hospitals utilising the NTS. The study comprised 12 written patient scenarios. He found that no one scenario was triaged the same by all participants and 10 of the 12 scenarios received triage scores encompassing three or more categories (Doherty, 1996). Similarly, Dilley and Standen (1998) assessed the level of uniformity among Victorian public hospital triage nurses in utilizing the NTS. The study comprised 20 written patient scenarios administered to 188 nurses from 14 different Victorian hospital EDs. The authors found that no one patient scenario was triaged to the same category by all 188 triage nurses and 75% of scenarios were triaged to four different triage categories (Dilley and Standen, 1998). Additionally, Considine et al. (2000) examined triage nurses’ level of agreement in the allocation of triage categories using the NTS. They surveyed 31 triage nurses from two institutions with 10 written scenarios. The authors found that no one scenario was triaged the same by all participants. Six of the scenarios were triaged to two triage categories and four of the scenarios were triaged to three triage categories (Considine et al., 2000).

The implications of the variability in triage categorisation identified in these studies are numerous. If triage nurses fail to categorise patients appropriately then medical attention may be unacceptably delayed or unnecessarily expedited (Goodacre et al., 1999). As a consequence there will be inequity in patient access to ED services, potential risks to patient safety, inefficiencies in the management of patient workload and difficulty in making meaningful comparisons between institutions based on triage categorisation data (Considine et al., 2000, Goodacre et al., 1999, Doherty, 1996, Monash Institute of Health Services Research , 2001).

Algorithms can be used to provide decision support in the assessment of clinical urgency. The benefit of such an approach lies in the direction and structure they provide in guiding the decision for the triage nurse (Gerdtz and Bucknall, 1999). This concurs with the thoughts of other authors who suggest nurses need to use standard guidelines when assessing patients so that all patients are assessed similarly and according to standards of practice (Stephens et al., 1997). The need for a systematic method of assessment which is easy to understand and quick to use has been widely recognised (Williams, 1992).

The TATTT seeks to provide a reproducible, reliable and valid method of triage assessment and categorisation, albeit in a select group of patients (adult trauma). This paper, outlines the preliminary investigation into the validity and reliability of the TATTT.

The primary aims of this project were to: operationalise the TATTT, determine the most appropriate strategy for simulating triage presentations to allow validity and reliability testing of the TATTT, and commence validity and reliability testing.

Section snippets

Method

A computer package for the TATTT was written in Visual Basic on a Pocket PC 2002 device running the Windows CE (Compact Edition) 3.0 operating system.

A random sample of participants for the initial investigation of the TATTT was not required. Volunteers were sought from among qualified triage nurses employed at the two hospitals involved in this study. Fifteen registered nurses were recruited into the study. Ten of these nurses were employed at the Toowoomba health service (THS) ED and five

Data analysis

A first impression of validity of the TATTT was obtained by examining the agreement between ATS and TATTT on the 58 genuine patients. Further information was gleaned by a detailed clinical review of every patient in which the TATTT and ATS codes differed.

Inter-rater reliability was described using descriptive statistics and measured against the ‘expected’ triage category for each simulation provided by the research team. Intra-rater reliability was measured by comparing initial codes with

Results

The agreement between the triage category assigned using existing triage practices and the triage category assigned by the TATTT during parallel coding are summarised in Table 1. The overall percentage agreement between the two systems was approximately 47% with an estimated κ of κˆ=0.19 (95% confidence interval [−0.02, 0.40]). In total 27 of the 58 patients (47%) received the same triage rating on both methods, 22 (38%) received a more urgent rating on the TATTT than by conventional methods

Discussion

This study encompassed the computerisation and initial field trial of the TATTT. The trial was not designed to be comprehensive. It involved 15 triage nurses from two hospitals in one Australian state. Neither the participants nor the hospitals were randomly selected.

The trial was ethically constrained to triaging simulated patients as the TATTT has not yet been proven valid or reliable for use on actual patients. The simulations provide a highly simplified view of the real world. The viewer is

Conclusion

Preliminary evidence thus far suggests that the TATTT provides systematically different results compared to current triage practices utilising the ATS; has greater reliability than current triage practices and will be safe for use in the clinical environment. As such, it can be viewed as a viable option, worthy of further investigation, for augmenting current triage practice with the ATS.

Further evaluation of the TATTT in a larger prospective trial is planned to further validate the

Acknowledgements

The study was funded by the Queensland Nursing Council. Ethics approval was obtained from the Toowoomba Health Service and the University of Southern Queensland’s Human Research and Ethics Committees. Consent was obtained from both the nurses and the patients who participated in the study. Thanks to all the staff of PAH and THS ED for their support during the project.

References (19)

There are more references available in the full text version of this article.

Cited by (11)

  • Undergraduate nursing students' performance in recognising and responding to sudden patient deterioration in high psychological fidelity simulated environments: An Australian multi-centre study

    2014, Nurse Education Today
    Citation Excerpt :

    TEAM instrument correlations confirmed moderate to strong relationships within the instrument, between the 11-item scale and global rating, and the reliability of the instrument was also confirmed. These claims to validity and reliability have recently been independently corroborated (McKay et al., 2012) and the TEAM instrument has been identified as being quick and easy to use to assess team working and non-technical skills in resuscitation contexts and simulated environments and for global assessment of team performance (McKay et al., 2012) (Wollaston et al., 2004). In terms of situation awareness our findings suggest that this may be a poorly developed skill in nursing students.

  • Calculating kappas from adjusted data improved the comparability of the reliability of triage systems: A comparative study

    2010, Journal of Clinical Epidemiology
    Citation Excerpt :

    Of the studies measuring the reliability with case scenarios, the distribution of the numbers of cases per triage category was in four studies based on the casemix of the EDs under study [12,28,35,36]. In eight studies [15,37–43], the investigator selected scenarios that were not derived from actual ED casemix. In nine studies, this was not mentioned [10,17,44–50].

  • Adjusting weighted kappa for severity of mistriage decreases reported reliability of emergency department triage systems: a comparative study

    2009, Journal of Clinical Epidemiology
    Citation Excerpt :

    This kappa is adjusted for prevalence and bias by substituting actual values in the cells of a contingency table with average values [31]. In 10 studies, information on the type of kappa used (weighted or unweighted) was missing [32–41]. We concluded from the presented contingency tables that three of these studies reported an unweighted kappa [32,33,40].

View all citing articles on Scopus
View full text