Simulation and educationRating medical emergency teamwork performance: Development of the Team Emergency Assessment Measure (TEAM)☆
Introduction
The quality of cardiopulmonary resuscitation (CPR) and medical emergency team performance has been questioned given that survival from in-hospital resuscitation is low.1, 2 The determinants of effective team performance include technical and nontechnical skills such as leadership and teamwork,3, 4, 5, 6 decision making and situation awareness.7 First developed for aircraft cockpit crews, situation awareness (SA) has been measured in anaesthesia and acute medicine8, 9, 10 to ascertain awareness and understanding of environmental elements.11 These skills are encapsulated in the study of human factors which are defined as “the environmental, organisational, and job factors, and human individual characteristics which influence people at work”.12
Clinical experience is not a guarantee of competence13; teamwork needs to be learnt and practiced in safe simulated settings to enhance resuscitation performance,14, 15 with the consequential development of medical emergency teamwork courses.16, 17 The use of patient simulators (manikins, standardised patients and computer simulations) and audiovisual capture have been useful in up-skilling staff18 with benefits of replay, revision and debriefing19 in both individual and team training.20 However, there is also a need to focus upon additional professional skills, such as situation awareness, to enhance leadership and team performance.21
Specialist team rating scales have been developed to measure teamwork for patient safety22, 23; global assessment of clinical and teamwork skills for crisis resource management17; combined technical and non-technical trauma team skills10; anaesthetists’ and surgeons’ non-technical skills8, 24 and resuscitation team leadership.25 Two tools specifically focus on inter-professional teamwork in an emergency environment: the Mayo High Performance Teamwork Scale (MayoHPTS)26 and the Emergency Team Dynamics (ETD) scale.27 The MayoHPTS rates leadership, teamwork, communication, quality and adaptability in a 16 item scale, whilst ETD measures leadership (1 item) and teamwork including communication, co-operation, and work effort. Both scales have demonstrated degrees of reliability and validity however ETD lacks in depth review of leadership and requires additional validity and reliability testing, whilst MayoHPTS has a broadly focused crisis resource management focus. Our intention therefore was to produce a measure with an applicable focus on leadership and teamwork that was resuscitation context specific.
In this project we aimed to develop a valid, reliable and feasible resuscitation teamwork assessment tool for trained observers to rate team performance and deliver a constructive debrief28 in simulated and clinical settings. The development and preliminary testing of the Team Emergency Assessment Measure (TEAM) involved three developmental stages: (i) selection of items, (ii) establishing the validity and reliability of the instrument, and (iii) conducting ‘real time’ testing on a cohort of second year medical and nursing students.
Section snippets
Methods
The instrument was developed in five stages: firstly an extensive review of the literature for teamwork instruments; secondly development of a draft instrument with an expert clinical team; thirdly a review by an international team of seven independent experts for face and content validity; fourthly instrument testing on 56 video-recorded hospital (n = 3) and simulated resuscitation events (n = 53) for construct, consistency, concurrent validity and reliability; and a final set of ratings for
Instrument development: results
During the production of the instrument judgments were made by the research team which included experienced clinicians and academics (19–41 years) who were Resuscitation Officers (n = 1), from emergency care (n = 2 RNs), general practice (n = 1), psychology and medical education (n = 3). Four members of the team had Resuscitation Council (UK and Australia) approved provider or instructor qualifications. Instrument testing was achieved by rating previously recorded and ethically approved
Stage 1: selection of items
A search of the literature was conducted to locate and review existing teamwork measurement instruments. Electronic databases used in the search included Medline, ProQUEST, PsycINFO and specialty websites (e.g. National Patient Safety Association UK and Resuscitation Council UK). Access strategies included keywords (e.g. teamwork; performance analysis; medical emergency team), author and journal searches. Seventeen teamwork instruments were located of which fourteen were considered relevant (
Content validity
The face and content validity of the TEAM were assessed by an international (United Kingdom, Australia and New Zealand) panel of six resuscitation experts with 15–29 years acute care experience. The panel comprised two doctors and four nurses/resuscitation officers. Each member of the team was asked to independently rate the relevance of the twelve TEAM items using a five-point scale (1 = not at all relevant, to 5 = most relevant). Level of agreement was determined by calculating a content validity
Stage 3: ‘Real time’ testing
The practicality and feasibility of the TEAM tool was pilot tested following a one day immediate life support course attended by second year medical and nursing students.
Discussion
Many adverse medical events are attributed to non-technical skill failures7, 24, 36 and may be exacerbated by the individual and transient nature of medical work. In response a variety of assessment tools and training schemes have been developed. These are often based on self- and subordinate ratings of performance; however observational team performance ratings in ‘real’ and simulated settings with applicable feedback are generally considered to be more rigorous.37
The Team Evaluation
Conclusion
Following rigorous development and initial testing in a simulated environment the TEAM has emerged as a valid, reliable and feasible nontechnical observational tool for the assessment of resuscitation team performance. The instrument will enable team performance rating and feedback which is likely to impact on patient safety. Further evaluation of the instrument is warranted in a variety of clinical settings to fully determine its psychometric properties.
Conflict of interest statement
None.
Acknowledgements
Gippsland Small Grant Research Support Scheme. Tracy McConnell- Henry for assistance with training and data collection.
Contributors: Dr S Cooper contributed to the study design, data collection, statistical analysis and wrote and edited the paper. Dr Robyn Cant and Dr Ken Sellick contributed to the study design, statistical analysis and wrote and edited the paper. Jo Porter and Dr George Somers contributed to the study design, data collection and edited the paper. Leigh Kinsman and Professor
References (48)
- et al.
A decade of in-hospital resuscitation: outcomes and prediction of survival
Resuscitation
(2006) - et al.
Team performance in the emergency room: assessment of inter-disciplinary attitudes
Resuscitation
(2001) - et al.
Anaesthetists’ non-technical skills (ANTS): evaluation of a behavioural marker system
Br J Anaesth
(2003) - et al.
Quality of CPR during advanced resuscitation training
Resuscitation
(2008) - et al.
Simulation applications for human factors and systems evaluation
Anesthesiol Clin
(2007) - et al.
Human factors affect the quality of cardiopulmonary resuscitation in simulated cardiac arrests
Resuscitation
(2004) - et al.
Non-technical skills for surgeons in the operating room: a review of the literature
Surgery
(2006) - et al.
Leadership of resuscitation teams: ‘Lighthouse Leadership’
Resuscitation
(1999) - et al.
Non-technical skills in the intensive care unit
Br J Anaesth
(2006) - et al.
Quality of cardiopulmonary resuscitation during in-hospital cardiac arrest
JAMA
(2005)
Collaborative practices in unscheduled emergency care: role and impact of the emergency care practitioner qualitative and summative findings
Emerg Med J
Improving medical emergency team (MET) performance using a novel curriculum and a computerized human patient simulator
Qual Saf Health Care
Teaching teamwork during the Neonatal Resuscitation Program: a randomized trial
J. Perinatol
Non-technical skills: identifying, training, and assessing safe behaviours
Leadership for safety: industrial experience
Qual Saf Health Care
Identifying and training non-technical skills for teams in acute medicine
Qual Saf Health Care
Simulation-based training of internal medicine residents in advanced cardiac life support protocols: a randomized trial
Teach Learn Med
Teamwork: building healthier workplaces and providing safer patient care
Crit Care Nurs Quart
A pilot study using high-fidelity simulation to formally evaluate performance in the resuscitation of critically ill patients: the University of Ottawa critical care medicine, high-fidelity simulation, and crisis resource management I study
Crit Care Med
The effect of simulation learning on critical thinking and self-confidence when incorporated into an electrocardiogram nursing course
Clin Simul Nurs
A human factors analysis of technical and team skills among surgical trainees during procedural simulations in a simulated operating theatre
Ann Surg
Cited by (249)
Use of pediatric trauma simulations to facilitate exposure to pediatric trauma resuscitations during training
2023, Surgery (United States)Reducing physical restraints in pediatrics: A quality improvement mixed-methods analysis of implementing a clinical debriefing process after behavioural health emergencies in a Children's Hospital
2023, Current Problems in Pediatric and Adolescent Health CareInfluence of additional personal protective equipment on team performance in simulation-based emergency scenarios: a randomised controlled trial
2023, British Journal of Anaesthesia
- ☆
A Spanish translated version of the abstract of this article appears as Appendix in the final online version at doi:10.1016/j.resuscitation.2009.11.027.