The accuracy of clinical assessments as a measure for teamwork effectiveness

Abstract

Introduction: Team training in healthcare is usually evaluated by observers who either score trainees' behaviors, social skills, and cognitive skills during simulation or measure changes in the clinical state of a mannequin. Both methods have shortcomings that limit their usefulness. We propose Brunswik's probabilistic functionalism and the Accuracy Score (AS), a measure emerging from judgment analysis, as elements of a complementary approach that could increase the objectivity of team training evaluation. We report an initial investigation.

Method: Three groups of neonatal clinicians participated in a resuscitation experiment involving three different training interventions. During the experiment, at various phases, the participants were required to assign an Apgar score to a mannequin.

Analysis: The AS was used to test how accurately the clinicians assigned Apgar scores to the mannequin across different levels of task demand, training content, and training delivery method.

Results: The AS was lower when task demand increased (P < 0.01). The AS was higher after teamwork training than after clinical training (P < 0.05) and better after hands-on teamwork training than after lecture-based teamwork training (P < 0.05).

Conclusions: Because it is simple and objective, the AS may complement existing measures for team training evaluation. Future studies are required in which the AS is tested with a larger number of trainees, in longitudinal experiments, across different training areas, and is compared with previously validated team performance measures.