Assessment of Clinical Reasoning in Undergraduate Medical Education: A Pragmatic Approach to Programmatic Assessment

Todd A Guth; Rachel M Wolfe; Ofelia Martinez; Raja G Subhiyah; Jerusha J Henderek; Caroline McAllister; Danielle Roussel

doi:10.1097/ACM.0000000000005665

Assessment of Clinical Reasoning in Undergraduate Medical Education: A Pragmatic Approach to Programmatic Assessment

Acad Med. 2024 Feb 27. doi: 10.1097/ACM.0000000000005665. Online ahead of print.

Authors

Todd A Guth¹, Rachel M Wolfe², Ofelia Martinez³, Raja G Subhiyah⁴, Jerusha J Henderek⁵, Caroline McAllister⁶, Danielle Roussel⁷

Affiliations

¹ T.A. Guth is associate professor, Department of Emergency Medicine, University of Colorado School of Medicine, Aurora, Colorado; ORCID: http://orcid.org/0000-0003-2734-8771.
² R.M. Wolfe is assistant professor, Department of Internal Medicine, Section of Rheumatology and Immunology, and director of medical decision-making thread, Wake Forest School of Medicine, Winston-Salem, North Carolina; ORCID: https://orcid.org/0000-0001-5061-2134.
³ O. Martinez is assistant dean of medical education and director of clinical skills, Hackensack Meridian School of Medicine at Seton Hall University, Hackensack, New Jersey.
⁴ R.G. Subhiyah is a principal measurement scholar and senior psychometrician, Psychometrics and Data Analysis, National Board of Medical Examiners, Philadelphia, Pennsylvania.
⁵ J.J. Henderek is manager of psychometrics, Psychometrics and Data Analysis, National Board of Medical Examiners, Philadelphia, Pennsylvania; ORCID: http://orcid.org/0009-0003-7451-0628.
⁶ C. McAllister is data analyst III, Psychometrics and Data Analysis, National Board of Medical Examiners, Philadelphia, Pennsylvania.
⁷ D. Roussel is assistant dean for medical student assessment and professor (clinical), Department of Anesthesiology, Spencer Fox Eccles School of Medicine, University of Utah, Salt Lake City, Utah; ORCID: http://orcid.org/0000-0002-3993-6987.

PMID: 38412485
DOI: 10.1097/ACM.0000000000005665

Abstract

Purpose: Clinical reasoning, a complex construct integral to the practice of medicine, has been challenging to define, teach, and assess. Programmatic assessment purports to overcome validity limitations of judgments made from individual assessments through proportionality and triangulation processes. This study explored a pragmatic approach to the programmatic assessment of clinical reasoning.

Method: The study analyzed data from 2 student cohorts from the University of Utah School of Medicine (UUSOM) (n = 113 in cohort 1 and 119 in cohort 2) and 1 cohort from the University of Colorado School of Medicine (CUSOM) using assessment data that spanned from 2017 to 2021 (n = 199). The study methods included the following: (1) asking faculty judges to categorize student clinical reasoning skills, (2) selecting institution-specific assessment data conceptually aligned with clinical reasoning, (3) calculating correlations between assessment data and faculty judgments, and (4) developing regression models between assessment data and faculty judgments.

Results: Faculty judgments of student clinical reasoning skills were converted to a continuous variable of clinical reasoning struggles, with mean (SD) ratings of 2.93 (0.27) for the 232 UUSOM students and 2.96 (0.17) for the 199 CUSOM students. A total of 67 and 32 discrete assessment variables were included from the UUSOM and CUSOM, respectively. Pearson r correlations were moderate to strong between many individual and composite assessment variables and faculty judgments. Regression models demonstrated an overall adjusted R2 (standard error of the estimate) of 0.50 (0.19) for UUSOM cohort 1, 0.28 (0.15) for UUSOM cohort 2, and 0.30 (0.14) for CUSOM.

Conclusions: This study represents an early pragmatic exploration of regression analysis as a potential tool for operationalizing the proportionality and triangulation principles of programmatic assessment. The study found that programmatic assessment may be a useful framework for longitudinal assessment of complicated constructs, such as clinical reasoning.