Using event logs to observe interactions with electronic health records: an updated scoping review shows increasing use of vendor-derived measures

Adam Rule; Edward R Melnick; Nate C Apathy

doi:10.1093/jamia/ocac177

Using event logs to observe interactions with electronic health records: an updated scoping review shows increasing use of vendor-derived measures

J Am Med Inform Assoc. 2022 Dec 13;30(1):144-154. doi: 10.1093/jamia/ocac177.

Authors

Adam Rule¹, Edward R Melnick^{2

3}, Nate C Apathy^{4

5}

Affiliations

¹ Information School, University of Wisconsin-Madison, Madison, Wisconsin, USA.
² Emergency Medicine, Yale School of Medicine, New Haven, Connecticut, USA.
³ Biostatistics (Health Informatics), Yale School of Public Health, New Haven, Connecticut, USA.
⁴ MedStar Health National Center for Human Factors in Healthcare, MedStar Health Research Institute, District of Columbia, Washington, USA.
⁵ Regenstrief Institute, Indianapolis, Indiana, USA.

Abstract

Objective: The aim of this article is to compare the aims, measures, methods, limitations, and scope of studies that employ vendor-derived and investigator-derived measures of electronic health record (EHR) use, and to assess measure consistency across studies.

Materials and methods: We searched PubMed for articles published between July 2019 and December 2021 that employed measures of EHR use derived from EHR event logs. We coded the aims, measures, methods, limitations, and scope of each article and compared articles employing vendor-derived and investigator-derived measures.

Results: One hundred and two articles met inclusion criteria; 40 employed vendor-derived measures, 61 employed investigator-derived measures, and 1 employed both. Studies employing vendor-derived measures were more likely than those employing investigator-derived measures to observe EHR use only in ambulatory settings (83% vs 48%, P = .002) and only by physicians or advanced practice providers (100% vs 54% of studies, P < .001). Studies employing vendor-derived measures were also more likely to measure durations of EHR use (P < .001 for 6 different activities), but definitions of measures such as time outside scheduled hours varied widely. Eight articles reported measure validation. The reported limitations of vendor-derived measures included measure transparency and availability for certain clinical settings and roles.

Discussion: Vendor-derived measures are increasingly used to study EHR use, but only by certain clinical roles. Although poorly validated and variously defined, both vendor- and investigator-derived measures of EHR time are widely reported.

Conclusion: The number of studies using event logs to observe EHR use continues to grow, but with inconsistent measure definitions and significant differences between studies that employ vendor-derived and investigator-derived measures.

Keywords: audit log; electronic health record; event log.

Publication types

Review
Research Support, N.I.H., Extramural
Research Support, U.S. Gov't, P.H.S.
Research Support, Non-U.S. Gov't

MeSH terms

Commerce
Electronic Health Records*
Humans
Physicians*

Abstract

Publication types

MeSH terms

Grants and funding