Incorporating Generative AI Into a Health Informatics Curriculum to Build 21st Century Competencies: Multisite Pre-Post Study

Abstract

Background: We designed learning assignments for students to develop knowledge, skills, and professional attitudes about generative artificial intelligence (AI) in 2 different Master's level courses in health informatics. Our innovative approach assumed that the students had no technical background or experience in using generative AI tools.

Objective: This study aims to offer generalizable methods and experiences on integration and assessment of generative AI content into the higher education's health informatics curricula. The study's central driver is the preparation of graduate students with generative AI tools, skills, ethical discernment, and critical thinking capacities aligned with the rapidly shifting job-market requirements, independent of graduate students' backgrounds and technical expertise.

Methods: During the semester, students completed a pretest and posttest to assess knowledge about generative AI. Reflections explored their expectations and experiences using generative AI to complete their assignments and projects during the semester. Strong emphasis was placed on building skills and professional attitudes by using generative AI. Student engagement in behavioral, emotional, and cognitive domains was explored via detailed analysis of student reflections by faculty.

Results: Students at the University of Illinois Chicago increased their knowledge about generative AI from 81% to 93% through research of the basic generative AI concepts, as evidenced from outcomes of the open-book pre-and posttests given at the beginning and end of the capstone course. University of San Francisco students also improved from 77% to 80% by the end of the semester. Faculty analysis of student reflections upon completion of the course revealed primary interests in the essentials of generative AI, AI transformations to information and knowledge, and organizational changes influenced by AI adoption in the health care organizations, with ethics being a primary driver of students' interests and engagement.

Conclusions: Data from student reflections provided insight into generative AI skills that students developed and that health informatics programs can consider incorporating into their curricula. Building competencies in generative AI will prepare students for the 21st century workforce and enable them to build skills employers are seeking in the new digital health environment.

Keywords: AI; artificial intelligence; capstone experience; generative AI; generative AI ethics; generative artificial intelligence; pedagogical innovation; student engagement.