Developing software to "track and catch" missed follow-up of abnormal test results in a complex sociotechnical environment

Appl Clin Inform. 2013 Jul 31;4(3):359-75. doi: 10.4338/ACI-2013-04-RA-0019. eCollection 2013.


Background: Abnormal test results do not always receive timely follow-up, even when providers are notified through electronic health record (EHR)-based alerts. High workload, alert fatigue, and other demands on attention disrupt a provider's prospective memory for tasks required to initiate follow-up. Thus, EHR-based tracking and reminding functionalities are needed to improve follow-up.

Objectives: The purpose of this study was to develop a decision-support software prototype enabling individual and system-wide tracking of abnormal test result alerts lacking follow-up, and to conduct formative evaluations, including usability testing.

Methods: We developed a working prototype software system, the Alert Watch And Response Engine (AWARE), to detect abnormal test result alerts lacking documented follow-up, and to present context-specific reminders to providers. Development and testing took place within the VA's EHR and focused on four cancer-related abnormal test results. Design concepts emphasized mitigating the effects of high workload and alert fatigue while being minimally intrusive. We conducted a multifaceted formative evaluation of the software, addressing fit within the larger socio-technical system. Evaluations included usability testing with the prototype and interview questions about organizational and workflow factors. Participants included 23 physicians, 9 clinical information technology specialists, and 8 quality/safety managers.

Results: Evaluation results indicated that our software prototype fit within the technical environment and clinical workflow, and physicians were able to use it successfully. Quality/safety managers reported that the tool would be useful in future quality assurance activities to detect patients who lack documented follow-up. Additionally, we successfully installed the software on the local facility's "test" EHR system, thus demonstrating technical compatibility.

Conclusion: To address the factors involved in missed test results, we developed a software prototype to account for technical, usability, organizational, and workflow needs. Our evaluation has shown the feasibility of the prototype as a means of facilitating better follow-up for cancer-related abnormal test results.

Keywords: Diagnostic errors; care delays; health information technology; patient safety; test results.

Publication types

  • Research Support, U.S. Gov't, Non-P.H.S.

MeSH terms

  • Decision Support Systems, Clinical* / organization & administration
  • Decision Support Systems, Clinical* / statistics & numerical data
  • Electronic Health Records
  • Follow-Up Studies
  • Humans
  • Neoplasms / diagnosis
  • Software*
  • Workflow