An assessment of Health Care Information and Management Systems Society and Leapfrog data on computerized provider order entry

Abstract

Objective: To assess the internal consistency and agreement between the Health Care Information and Management Systems Society (HIMSS) and the Leapfrog computerized provider order entry (CPOE) data.

Data sources: Secondary hospital data collected by HIMSS Analytics, the Leapfrog Group, and the American Hospital Association from 2005 to 2007.

Study design: Dichotomous measures of full CPOE status were created for the HIMSS and Leapfrog datasets in each year. We assessed internal consistency by calculating the percent of full adopters in a given year that report full CPOE status in subsequent years. We assessed the level of agreement between the two datasets by calculating the κ statistic and McNemar's test. We examined responsiveness by assessing the change in full CPOE status rates, over time, reported by HIMSS and Leapfrog data, respectively.

Principal findings: Findings indicate minimal agreement between the two datasets regarding positive hospital CPOE status, but adequate agreement within a given dataset from year to year. Relative to each other, the HIMSS data tend to overestimate increases in full CPOE status over time, while the Leapfrog data may underestimate year over year increases in national CPOE status.

Conclusions: Both Leapfrog and HIMSS data have strengths and weaknesses. Those interested in studying outcomes associated with CPOE use or adoption should be aware of the strengths and limitations of the Leapfrog and HIMSS datasets. Future development of a standard definition of CPOE status in hospitals will allow for a more comprehensive validation of these data.