Checklist Design Reconsidered: Understanding Checklist Compliance and Timing of Interactions

Abstract

We examine the association between user interactions with a checklist and task performance in a time-critical medical setting. By comparing 98 logs from a digital checklist for trauma resuscitation with activity logs generated by video review, we identified three non-compliant checklist use behaviors: failure to check items for completed tasks, falsely checking items when tasks were not performed, and inaccurately checking items for incomplete tasks. Using video review, we found that user perceptions of task completion were often misaligned with clinical practices that guided activity coding, thereby contributing to non-compliant check-offs. Our analysis of associations between different contexts and the timing of check-offs showed longer delays when (1) checklist users were absent during patient arrival, (2) patients had penetrating injuries, and (3) resuscitations were assigned to the highest acuity. We discuss opportunities for reconsidering checklist designs to reduce non-compliant checklist use.

Keywords: Checklist design; dynamic checklist; medical checklist; mixed methods; trauma resuscitation; video review.

Figure 1:

Example screens from the digital checklist with user notes in the margin area, typed and stylus notes on the primary survey (a), and checks/notes on the secondary survey (b).

Figure 2:

Timing of item check-offs categorized as (a) checked before task start, (b) during task performance, and (c) after task completion. Items that were checked with longest delay after task performance are shown in (d).