Validity of Medical Record Abstraction and Electronic Health Record-Generated Reports to Assess Performance on Cardiovascular Quality Measures in Primary Care

Abstract

Importance: Cardiovascular disease is the leading cause of death in the United States. To improve cardiovascular outcomes, primary care must have valid methods of assessing performance on cardiovascular clinical quality measures, including aspirin use (aspirin measure), blood pressure control (BP measure), and smoking cessation counseling and intervention (smoking measure).

Objective: To compare observed performance scores measured using 2 imperfect reference standard data sources (medical record abstraction [MRA] and electronic health record [EHR]-generated reports) with misclassification-adjusted performance scores obtained using bayesian latent class analysis.

Design, setting, and participants: This cross-sectional study used a subset of the 2016 aspirin, BP, and smoking performance data from the Healthy Hearts for Oklahoma Project. Each clinical quality measure was calculated for a subset of a practice's patient population who can benefit from recommended care (ie, the eligible population). A random sample of 380 eligible patients were included for the aspirin measure; 126, for the BP measure; and 115, for the smoking measure. Data were collected from 21 primary care practices belonging to a single large health care system from January 1 to December 31, 2018, and analyzed from February 21 to April 17, 2019.

Main outcomes and measures: The main outcomes include performance scores for the aspirin, BP, and smoking measures using imperfect MRA and EHRs and estimated through bayesian latent class models.

Results: A total of 621 eligible patients were included in the analysis. Based on MRA and EHR data, observed aspirin performance scores were 76.0% (95% bayesian credible interval [BCI], 71.5%-80.1%) and 74.9% (95% BCI, 70.4%-79.1%), respectively; observed BP performance scores, 80.6% (95% BCI, 73.2%-86.9%) and 75.1% (95% BCI, 67.2%-82.1%), respectively; and observed smoking performance scores, 85.7% (95% BCI, 78.6%-91.2%) and 75.4% (95% BCI, 67.0%-82.6%), respectively. Misclassification-adjusted estimates were 74.9% (95% BCI, 70.5%-79.1%) for the aspirin performance score, 75.0% (95% BCI, 66.6%-82.5%) for the BP performance score, and 83.0% (95% BCI, 74.4%-89.8%) for the smoking performance score.

Conclusions and relevance: Ensuring valid performance measurement is critical for value-based payment models and quality improvement activities in primary care. This study found that extracting information for the same individuals using different data sources generated different performance score estimates. Further research is required to identify the sources of these differences.

Figure 1.. Schematic Illustration of a Bayesian Latent Class Model for the Aspirin Performance Measure Using Medical Record Abstraction (MRA) and Electronic Health Record (EHR)–Generated Data

Rectangles represent observed data; ovals represent unknown (ie, unobserved) parameters.

Figure 2.. Observed Medical Record Abstraction (MRA) and Electronic Health Record (EHR)–Generated Scores and Prior Set 1 Results

Results are based on expert opinion, misclassification-adjusted median scores for the aspirin, blood pressure (BP), and smoking performance measures. Percentages in parentheses represent the probability of meeting the 80% target. Error bars indicate 95% bayesian credible intervals.