Validity Evidence for STATUS to Assess Resident Tolerance for and Competence in Communicating Medical Ambiguity

Ariel S Frey-Vogel; Kevin Ching; Michael G Healy; Dandan Chen; Yoon Soo Park; Emil Petrusa; Hadi B Anwar; Charles Bergman; Daniel A Hall; Logan Murray; Adin Nelson; Katherine A Sparger; Brooke I Spector; Brian P Youth; Leah Mallory

doi:10.1016/j.acap.2026.103235

Validity Evidence for STATUS to Assess Resident Tolerance for and Competence in Communicating Medical Ambiguity

Acad Pediatr. 2026 Feb 9;26(4):103235. doi: 10.1016/j.acap.2026.103235. Online ahead of print.

Authors

Affiliations

¹ Harvard Medical School and Mass General for Children (AS Frey-Vogel), Department of Pediatrics, Boston, Mass. Electronic address: afrey@mgh.harvard.edu.
² Weill Cornell Medicine and NewYork Presbyterian (K Ching), Departments of Emergency Medicine and Pediatrics, New York, NY.
³ Massachusetts General Hospital and Harvard Medical School (MG Healy, D Chen, and E Petrusa), Department of Surgery, Boston, Mass.
⁴ University of Illinois College of Medicine (YS Park), Department of Medical Education, Chicago, Ill.
⁵ Pediatic Critical Care of Virginia (HB Anwar), Richmond, Va.
⁶ Weill Cornell Medicine and NewYork Presbyterian (C Bergman), Division of Pediatric Critical Care Medicine, New York, NY.
⁷ Harvard Medical School And Mass General for Children (DA Hall), Department of Pediatrics, Boston, Mass.
⁸ Maine Medical Center (L Murray), Department of Pediatric Hospital Medicine, MaineHealth Hospital Medicine, Portland, Maine.
⁹ Weill Cornell Medicine and NewYork Presbyterian (A Nelson and BI Spector), Department of Pediatric Hospital Medicine, New York, NY.
¹⁰ Harvard Medical School And Mass General for Children (KA Sparger), Department of Pediatrics, Division of Newborn Medicine, Boston, Mass.
¹¹ Maine Medical Center (BP Youth), Department of Pediatrics, MaineHealth Hospital Medicine, Portland, Maine.
¹² Tufts University School of Medicine and MaineHealth (L Mallory), Department of Medical Education, Portland, Maine.

PMID: 41672143
DOI: 10.1016/j.acap.2026.103235

Abstract

Objective: No assessment instrument with validity evidence exists to assess resident competence in communicating medical ambiguity. Here, validity evidence was collected for STATUS (Scalable Tolerating Ambiguity/Uncertainty Tool Utilizing Simulation).

Methods: Using avatar patients and 2 simulated cases, investigators created a guidebook and trained 10 faculty in STATUS use. Pediatric residents completed 2 video-recorded simulated cases. Residents self-assessed tolerance for communicating medical ambiguity. Two faculty reviewed each video, assessing participants for communicating medical ambiguity. Validity evidence collected included the following: content, response process, internal structure, and relationship to other variables. Generalizability theory analysis was conducted to understand the assessment tools' reliability.

Results: Of 89 eligible residents, 43 (48.3%) had sessions recorded. Eighty-six videos were analyzed. Faculty rater training increased inter-rater reliability by 0.34 units. The Φ-coefficient was 0.72 for the resident self-assessment tool and 0.26 for the faculty rater assessment tool. The decision study found that with 11 faculty raters and 11 scenarios, the Φ-coefficient would be 0.70. Resident self-assessment was negatively associated with faculty rater assessment with a Spearman correlation of -0.21 overall, indicating a possible weak correlation.

Conclusion: Results provide sufficient reliability to measure resident self-assessment of tolerance for communicating medical ambiguity. More scenarios would likely result in higher reliability for faculty assessment.

Keywords: ambiguity; communication; competency-based assessment; simulation.