Development and Validation of eRADAR: A Tool Using EHR Data to Detect Unrecognized Dementia

Abstract

Objectives: Early recognition of dementia would allow patients and their families to receive care earlier in the disease process, potentially improving care management and patient outcomes, yet nearly half of patients with dementia are undiagnosed. Our aim was to develop and validate an electronic health record (EHR)-based tool to help detect patients with unrecognized dementia (EHR Risk of Alzheimer's and Dementia Assessment Rule [eRADAR]).

Design: Retrospective cohort study.

Setting: Kaiser Permanente Washington (KPWA), an integrated healthcare delivery system.

Participants: A total of 16 665 visits among 4330 participants in the Adult Changes in Thought (ACT) study, who undergo a comprehensive process to detect and diagnose dementia every 2 years and have linked KPWA EHR data, divided into development (70%) and validation (30%) samples.

Measurements: EHR predictors included demographics, medical diagnoses, vital signs, healthcare utilization, and medications within the previous 2 years. Unrecognized dementia was defined as detection in ACT before documentation in the KPWA EHR (ie, lack of dementia or memory loss diagnosis codes or dementia medication fills).

Results: Overall, 1015 ACT visits resulted in a diagnosis of incident dementia, of which 498 (49%) were unrecognized in the KPWA EHR. The final 31-predictor model included markers of dementia-related symptoms (eg, psychosis diagnoses, antidepressant fills), healthcare utilization pattern (eg, emergency department visits), and dementia risk factors (eg, cerebrovascular disease, diabetes). Discrimination was good in the development (C statistic = .78; 95% confidence interval [CI] = .76-.81) and validation (C statistic = .81; 95% CI = .78-.84) samples, and calibration was good based on plots of predicted vs observed risk. If patients with scores in the top 5% were flagged for additional evaluation, we estimate that 1 in 6 would have dementia.

Conclusion: The eRADAR tool uses existing EHR data to detect patients with good accuracy who may have unrecognized dementia. J Am Geriatr Soc 68:103-111, 2019.

Keywords: decision support techniques; dementia; early diagnosis.

Figure 1.

Key predictors of undiagnosed dementia in the electronic health record Risk of Alzheimer’s and Dementia Assessment Rule (eRADAR) included dementia risk factors, dementia-related symptoms, and healthcare utilization patterns.

Figure 2.

The performance characteristics of the final restricted eRADAR model are shown for the training sample (left panels) and test sample (right panels). Figure 2a shows the receiver operating characteristic (ROC) curves with c-statistics. The ROC curve plots the sensitivity (true positive rate) against 1 - specificity (false positive rate) for consecutive cutoffs for the probability of unrecognized dementia. The c-statistic reflects the area under the ROC curve (AUC), with values of 0.5 reflecting prediction no better than chance and 1 reflecting perfect prediction. Figure 2b provides a graphical assessment of calibration (i.e., the extent to which predicted risk matches actual risk). The mean predicted probability of unrecognized dementia is plotted against the observed proportion of unrecognized dementia cases by quintiles of predicted risk. The figure shows close alignment between observed and predicted values (ideal calibration aligns with the 45-degree line). The histogram at the bottom shows the distribution of the predicted risks. Most visits had relatively low predicted risk of undiagnosed dementia (<5%). Supplementary Figure S1 provides similar figures for the model that considered the full set of predictors.