Background: Electronic health record (EHR) vendors now offer "off-the-shelf" artificial intelligence (AI) models to client organizations. Our health system faced difficulties in promoting end-user utilization of a new AI model for predicting readmissions embedded in the EHR.
Objectives: The aim is to conduct a case study centered on identifying barriers to uptake/utilization.
Methods: A qualitative study was conducted using interviews with stakeholders. The interviews were used to identify relevant stakeholders, understand current workflows, identify implementation barriers, and formulate future strategies.
Results: We discovered substantial variation in existing workflows around readmissions. Some stakeholders did not perform any formal readmissions risk assessment. Others accustomed to using existing risk scores such as LACE+ had concerns about transitioning to a new model. Some stakeholders had existing workflows in place that could accommodate the new model, but they were not previously aware that the new model was in production. Concerns expressed by end-users included: whether the model's predictors were relevant to their work, need for adoption of additional workflow processes, need for training and change management, and potential for unintended consequences (e.g., increased health care resource utilization due to potentially over-referring discharged patients to home health services).
Conclusion: AI models for risk stratification, even if "off-the-shelf" by design, are unlikely to be "plug-and-play" in health care settings. Seeking out key stakeholders and defining clear use cases early in the implementation process can better facilitate utilization of these models.
Keywords: artificial intelligence; case management; clinical informatics; electronic health records; health system; machine learning; predictive analytics; predictive models; readmissions.