The accuracy of clinical assessments as a measure for teamwork effectiveness

Simul Healthc. 2011 Oct;6(5):260-8. doi: 10.1097/SIH.0b013e31821eaa38.

Abstract

Introduction: Team training in healthcare is usually evaluated by observers who either score trainees' behaviors, social skills, and cognitive skills during simulation or measure changes in the clinical state of a mannequin. Both methods have shortcomings that limit their usefulness. We propose Brunswik's probabilistic functionalism and the Accuracy Score (AS), a measure emerging from judgment analysis, as elements of a complementary approach that could increase the objectivity of team training evaluation. We report an initial investigation.

Method: Three groups of neonatal clinicians participated in a resuscitation experiment involving three different training interventions. During the experiment, at various phases, the participants were required to assign an Apgar score to a mannequin.

Analysis: The AS was used to test how accurately the clinicians assigned Apgar scores to the mannequin across different levels of task demand, training content, and training delivery method.

Results: The AS was lower when task demand increased (P < 0.01). The AS was higher after teamwork training than after clinical training (P < 0.05) and better after hands-on teamwork training than after lecture-based teamwork training (P < 0.05).

Conclusions: Because it is simple and objective, the AS may complement existing measures for team training evaluation. Future studies are required in which the AS is tested with a larger number of trainees, in longitudinal experiments, across different training areas, and is compared with previously validated team performance measures.

Publication types

  • Research Support, Non-U.S. Gov't

MeSH terms

  • Clinical Competence
  • Cooperative Behavior
  • Humans
  • Infant, Newborn
  • Inservice Training / organization & administration*
  • Patient Care Team*
  • Reproducibility of Results
  • Resuscitation / education*
  • Task Performance and Analysis*
  • Videotape Recording