Online "spaced education progress-testing" of students to confront two upcoming challenges to medical schools

Abstract

Purpose: U.S. medical students will soon complete only one licensure examination sequence, given near the end of medical school. Thus, schools are challenged to identify poorly performing students before this high-stakes test and help them retain knowledge across the duration of medical school. The authors investigated whether online spaced education progress-testing (SEPT) could achieve both aims.

Method: Participants were 2,648 students from four U.S. medical schools; 120 multiple-choice questions and explanations in preclinical and clinical domains were developed and validated. For 34 weeks, students randomized to longitudinal progress-testing alone (LPTA) received four new questions (with answers/ explanations) each week. Students randomized to SEPT received the identical four questions each week, plus two-week and six-week cycled reviews of the questions/explanations. During weeks 31-34, the initial 40 questions were re-sent to students to assess longer-term retention.

Results: Of the 1,067 students enrolled, the 120-question progress-test was completed by 446 (84%) and 392 (74%) of the LPTA and SEPT students, respectively. Cronbach alpha reliability was 0.87. Scores were 39.9%, 51.9%, 58.7%, and 58.8% for students in years 1-4, respectively. Performance correlated with Step 1 and Step 2 Clinical Knowledge scores (r = 0.52 and 0.57, respectively; P < .001) and prospectively identified students scoring below the mean on Step 1 with 75% sensitivity, 77% specificity, and 41% positive predictive value. Cycled reviews generated a 170% increase in learning retention relative to baseline (P < .001, effect size 0.95).

Conclusions: SEPT can identify poorly performing students and improve their longer-term knowledge retention.