There is little known about how academic medical centers (AMCs) in the US develop, implement, and maintain predictive modeling and machine learning (PM and ML) models. We conducted semi-structured interviews with leaders from AMCs to assess their use of PM and ML in clinical care, understand associated challenges, and determine recommended best practices. Each transcribed interview was iteratively coded and reconciled by a minimum of 2 investigators to identify key barriers to and facilitators of PM and ML adoption and implementation in clinical care. Interviews were conducted with 33 individuals from 19 AMCs nationally. AMCs varied greatly in the use of PM and ML within clinical care, from some just beginning to explore their utility to others with multiple models integrated into clinical care. Informants identified 5 key barriers to the adoption and implementation of PM and ML in clinical care: (1) culture and personnel, (2) clinical utility of the PM and ML tool, (3) financing, (4) technology, and (5) data. Recommendation to the informatics community to overcome these barriers included: (1) development of robust evaluation methodologies, (2) partnership with vendors, and (3) development and dissemination of best practices. For institutions developing clinical PM and ML applications, they are advised to: (1) develop appropriate governance, (2) strengthen data access, integrity, and provenance, and (3) adhere to the 5 rights of clinical decision support. This article highlights key challenges of implementing PM and ML in clinical care at AMCs and suggests best practices for development, implementation, and maintenance at these institutions.
Keywords: artificial intelligence; machine learning; predictive models; qualitative evaluation.
© The Author(s) 2020. Published by Oxford University Press on behalf of the American Medical Informatics Association.