Quantifying the impact of health IT implementations on clinical workflow: a new methodological perspective

Abstract

Health IT implementations often introduce radical changes to clinical work processes and workflow. Prior research investigating this effect has shown conflicting results. Recent time and motion studies have consistently found that this impact is negligible; whereas qualitative studies have repeatedly revealed negative end-user perceptions suggesting decreased efficiency and disrupted workflow. We speculate that this discrepancy may be due in part to the design of the time and motion studies, which is focused on measuring clinicians' 'time expenditures' among different clinical activities rather than inspecting clinical 'workflow' from the true 'flow of the work' perspective. In this paper, we present a set of new analytical methods consisting of workflow fragmentation assessments, pattern recognition, and data visualization, which are accordingly designed to uncover hidden regularities embedded in the flow of the work. Through an empirical study, we demonstrate the potential value of these new methods in enriching workflow analysis in clinical settings.

Figure 1

A ‘timeline belt’ visualization exhibiting workflow fragmentation before and after the computerized provider order entry (CPOE) implementation. Each row (belt) represents a time and motion (T&M) observation session. Colored stripes designate the execution of clinical activities belonging to different task categories. For example, the purple stripes represent ‘talking/rounding’ activities and the black stripes represent ‘computer—read.’ Hence, color transitions indicate cross-category task switches. Length of a colored stripe is proportional to how long the task lasted.

Figure 2

Pre-post comparison: multiple measures (*p<0.05, **p<0.01, ***p<0.001; based on Welch's t test).

Figure 3

Network plots exhibiting bidirectional task transition frequencies. Nodes=task categories; edges=transitions between pairs of task categories. Width of an edge is proportional to the transition frequency between the pair (number of bidirectional transitions observed per hour between the two task categories). The edges representing the transitions between ‘talking/rounding’ and ‘computing—read’ or ‘computing—writing’ are highlighted in red because they are most relevant in studying health IT's impact on clinical workflow. In both graphs, numeric labels are provided for the top five most frequent task transitions.

Figure 4

Heatmaps exhibiting task transition probabilities (*p<0.05, **p<0.01, ***p<0.001; based on Welch's t test). Formula for color determination = red: 255; green: 255−transition probability×500, rounded to integers; blue: 0.