Background: The medical education research literature consistently recommends a structured format for the medical school preadmission interview. There is, however, little direct evidence to support this recommendation.
Purpose: To shed further light on this issue, the present study examines the respective reliability contributions from the structured and unstructured interview components at the University of Iowa.
Methods: We conducted three univariate G studies on ratings from 3,043 interviews and one multivariate G study using responses from 168 applicants who interviewed twice.
Results: Examining interrater reliability and test-retest types of reliability, the unstructured format proved more reliable in both instances. Yet, combining measures from the two interview formats yielded a more reliable score than using either alone.
Conclusions: At least from a reliability perspective, the popular advice regarding interview structure may need to be reconsidered. Issues related to validity, fairness, and reliability should be carefully weighed when designing the interview process.