Elsevier

Resuscitation

Volume 81, Issue 4, April 2010, Pages 446-452
Resuscitation

Simulation and education
Rating medical emergency teamwork performance: Development of the Team Emergency Assessment Measure (TEAM)

https://doi.org/10.1016/j.resuscitation.2009.11.027Get rights and content

Abstract

Aim

To develop a valid, reliable and feasible teamwork assessment measure for emergency resuscitation team performance.

Background

Generic and profession specific team performance assessment measures are available (e.g. anaesthetics) but there are no specific measures for the assessment of emergency resuscitation team performance.

Methods

(1) An extensive review of the literature for teamwork instruments, and (2) development of a draft instrument with an expert clinical team. (3) Review by an international team of seven independent experts for face and content validity. (4) Instrument testing on 56 video-recorded hospital and simulated resuscitation events for construct, consistency, concurrent validity and reliability and (5) a final set of ratings for feasibility on fifteen simulated ‘real time’ events.

Results

Following expert review, selected items were found to have a high total content validity index of 0.96. A single ‘teamwork’ construct was identified with an internal consistency of 0.89. Correlation between the total item score and global rating (rho 0.95; p < 0.01) indicated concurrent validity. Inter-rater (k 0.55) and retest reliability (k 0.53) were ‘fair’, with positive feasibility ratings following ‘real time’ testing. The final 12 item (11 specific and 1 global rating) are rated using a five-point scale and cover three categories leadership, teamwork and task management.

Conclusion

In this primary study TEAM was found to be a valid and reliable instrument and should be a useful addition to clinicians’ tool set for the measurement of teamwork during medical emergencies. Further evaluation of the instrument is warranted to fully determine its psychometric properties.

Introduction

The quality of cardiopulmonary resuscitation (CPR) and medical emergency team performance has been questioned given that survival from in-hospital resuscitation is low.1, 2 The determinants of effective team performance include technical and nontechnical skills such as leadership and teamwork,3, 4, 5, 6 decision making and situation awareness.7 First developed for aircraft cockpit crews, situation awareness (SA) has been measured in anaesthesia and acute medicine8, 9, 10 to ascertain awareness and understanding of environmental elements.11 These skills are encapsulated in the study of human factors which are defined as “the environmental, organisational, and job factors, and human individual characteristics which influence people at work”.12

Clinical experience is not a guarantee of competence13; teamwork needs to be learnt and practiced in safe simulated settings to enhance resuscitation performance,14, 15 with the consequential development of medical emergency teamwork courses.16, 17 The use of patient simulators (manikins, standardised patients and computer simulations) and audiovisual capture have been useful in up-skilling staff18 with benefits of replay, revision and debriefing19 in both individual and team training.20 However, there is also a need to focus upon additional professional skills, such as situation awareness, to enhance leadership and team performance.21

Specialist team rating scales have been developed to measure teamwork for patient safety22, 23; global assessment of clinical and teamwork skills for crisis resource management17; combined technical and non-technical trauma team skills10; anaesthetists’ and surgeons’ non-technical skills8, 24 and resuscitation team leadership.25 Two tools specifically focus on inter-professional teamwork in an emergency environment: the Mayo High Performance Teamwork Scale (MayoHPTS)26 and the Emergency Team Dynamics (ETD) scale.27 The MayoHPTS rates leadership, teamwork, communication, quality and adaptability in a 16 item scale, whilst ETD measures leadership (1 item) and teamwork including communication, co-operation, and work effort. Both scales have demonstrated degrees of reliability and validity however ETD lacks in depth review of leadership and requires additional validity and reliability testing, whilst MayoHPTS has a broadly focused crisis resource management focus. Our intention therefore was to produce a measure with an applicable focus on leadership and teamwork that was resuscitation context specific.

In this project we aimed to develop a valid, reliable and feasible resuscitation teamwork assessment tool for trained observers to rate team performance and deliver a constructive debrief28 in simulated and clinical settings. The development and preliminary testing of the Team Emergency Assessment Measure (TEAM) involved three developmental stages: (i) selection of items, (ii) establishing the validity and reliability of the instrument, and (iii) conducting ‘real time’ testing on a cohort of second year medical and nursing students.

Section snippets

Methods

The instrument was developed in five stages: firstly an extensive review of the literature for teamwork instruments; secondly development of a draft instrument with an expert clinical team; thirdly a review by an international team of seven independent experts for face and content validity; fourthly instrument testing on 56 video-recorded hospital (n = 3) and simulated resuscitation events (n = 53) for construct, consistency, concurrent validity and reliability; and a final set of ratings for

Instrument development: results

During the production of the instrument judgments were made by the research team which included experienced clinicians and academics (19–41 years) who were Resuscitation Officers (n = 1), from emergency care (n = 2 RNs), general practice (n = 1), psychology and medical education (n = 3). Four members of the team had Resuscitation Council (UK and Australia) approved provider or instructor qualifications. Instrument testing was achieved by rating previously recorded and ethically approved

Stage 1: selection of items

A search of the literature was conducted to locate and review existing teamwork measurement instruments. Electronic databases used in the search included Medline, ProQUEST, PsycINFO and specialty websites (e.g. National Patient Safety Association UK and Resuscitation Council UK). Access strategies included keywords (e.g. teamwork; performance analysis; medical emergency team), author and journal searches. Seventeen teamwork instruments were located of which fourteen were considered relevant (

Content validity

The face and content validity of the TEAM were assessed by an international (United Kingdom, Australia and New Zealand) panel of six resuscitation experts with 15–29 years acute care experience. The panel comprised two doctors and four nurses/resuscitation officers. Each member of the team was asked to independently rate the relevance of the twelve TEAM items using a five-point scale (1 = not at all relevant, to 5 = most relevant). Level of agreement was determined by calculating a content validity

Stage 3: ‘Real time’ testing

The practicality and feasibility of the TEAM tool was pilot tested following a one day immediate life support course attended by second year medical and nursing students.

Discussion

Many adverse medical events are attributed to non-technical skill failures7, 24, 36 and may be exacerbated by the individual and transient nature of medical work. In response a variety of assessment tools and training schemes have been developed. These are often based on self- and subordinate ratings of performance; however observational team performance ratings in ‘real’ and simulated settings with applicable feedback are generally considered to be more rigorous.37

The Team Evaluation

Conclusion

Following rigorous development and initial testing in a simulated environment the TEAM has emerged as a valid, reliable and feasible nontechnical observational tool for the assessment of resuscitation team performance. The instrument will enable team performance rating and feedback which is likely to impact on patient safety. Further evaluation of the instrument is warranted in a variety of clinical settings to fully determine its psychometric properties.

Conflict of interest statement

None.

Acknowledgements

Gippsland Small Grant Research Support Scheme. Tracy McConnell- Henry for assistance with training and data collection.

Contributors: Dr S Cooper contributed to the study design, data collection, statistical analysis and wrote and edited the paper. Dr Robyn Cant and Dr Ken Sellick contributed to the study design, statistical analysis and wrote and edited the paper. Jo Porter and Dr George Somers contributed to the study design, data collection and edited the paper. Leigh Kinsman and Professor

References (48)

  • S. Cooper et al.

    Collaborative practices in unscheduled emergency care: role and impact of the emergency care practitioner qualitative and summative findings

    Emerg Med J

    (2007)
  • M.A. DeVita et al.

    Improving medical emergency team (MET) performance using a novel curriculum and a computerized human patient simulator

    Qual Saf Health Care

    (2005)
  • E.J. Thomas et al.

    Teaching teamwork during the Neonatal Resuscitation Program: a randomized trial

    J. Perinatol

    (2007)
  • R. Flin et al.

    Non-technical skills: identifying, training, and assessing safe behaviours

  • R. Flin et al.

    Leadership for safety: industrial experience

    Qual Saf Health Care

    (2004)
  • R. Flin et al.

    Identifying and training non-technical skills for teams in acute medicine

    Qual Saf Health Care

    (2004)
  • Endsley MR. Situation awareness global assessment technique (SAGAT). National Aerospace and Electronics Conference...
  • Health and Safety Executive. Health and safety Executive; 2009. http://www.hse.gov.uk/humanfactors/ [accessed...
  • D.B. Wayne et al.

    Simulation-based training of internal medicine residents in advanced cardiac life support protocols: a randomized trial

    Teach Learn Med

    (2005)
  • P. Clark

    Teamwork: building healthier workplaces and providing safer patient care

    Crit Care Nurs Quart

    (2009)
  • Resuscitation Council. Immediate Life Support Resuscitation Council;...
  • J. Kim et al.

    A pilot study using high-fidelity simulation to formally evaluate performance in the resuscitation of critically ill patients: the University of Ottawa critical care medicine, high-fidelity simulation, and crisis resource management I study

    Crit Care Med

    (2006)
  • D. Brown et al.

    The effect of simulation learning on critical thinking and self-confidence when incorporated into an electrocardiogram nursing course

    Clin Simul Nurs

    (2009)
  • K. Moorthy et al.

    A human factors analysis of technical and team skills among surgical trainees during procedural simulations in a simulated operating theatre

    Ann Surg

    (2005)
  • Cited by (249)

    View all citing articles on Scopus

    A Spanish translated version of the abstract of this article appears as Appendix in the final online version at doi:10.1016/j.resuscitation.2009.11.027.

    View full text