On the Predictive Power of the In-Training Examination for the Certification of Anesthesiologist Assistants

Irina Grabovsky; Jerusha J Henderek; Ulana A Luciw-Dubas; Brent Pierce; Soren Campbell; Katherine S Monroe

doi:10.1177/23821205231179534

On the Predictive Power of the In-Training Examination for the Certification of Anesthesiologist Assistants

J Med Educ Curric Dev. 2023 Jun 26:10:23821205231179534. doi: 10.1177/23821205231179534. eCollection 2023 Jan-Dec.

Authors

Irina Grabovsky¹, Jerusha J Henderek¹, Ulana A Luciw-Dubas¹, Brent Pierce¹, Soren Campbell², Katherine S Monroe²

Affiliations

¹ NBME, Philadelphia, PA, USA.
² National Commission for Certification of Anesthesiologist Assistants, Florence, KY, USA.

Abstract

Objectives: In-training examinations (ITEs) are a popular teaching tool for certification programs. This study examines the relationship between examinees' performance on the National Commission for Certification of Anesthesiologist Assistants (NCCAA) ITE and the high-stakes NCCAA Certification Examination.

Methods: We utilized a mixed-methods approach in our study. Before estimating the models for the predictive validity study, a series of interviews with program directors were conducted to discuss the role of the ITE in students' education. Multiple linear regression analysis was then used to assess the strength of the relationship between the ITE and Certification Examination scores, while considering the percentage of program examinees completed in their anesthesiologist assistant program between their ITE and Certification Examination attempts. Logistic regression analysis was used to estimate the probability of passing the Certification Examination as a function of ITE score.

Results: Interviews with program directors confirmed that the ITE provided a valuable testing experience for students and highlighted the areas where students need to focus. Moreover, both the ITE score and the percentage of the program between exams were deemed statistically significant predictors for Certification Examination scores. The logistic regression model indicated that higher scores on the ITE implied a higher probability of passing the Certification Examination.

Conclusion: This research demonstrated the high predictive validity of the ITE examination scores in predicting the Certification Examination outcomes. Together with the proportion of the program covered between exams, the variables explain a significant amount of variability in Certification Examination scores. The ITE feedback helped students assess their preparedness and better focus their studies for the high-stakes certification examination for the profession.

Keywords: anesthesiology; certification examination; curriculum; mixed-methods; predictive validity; self-assessment.