The Process of Developing an Assessment Checklist for Simulated Infant Respiratory Distress Using a Modified Delphi Method: A Mixed Methods Study

Justin M Jeffers; William Golden; Amit K Pahwa; Stacy Cooper; David Cooke; Rebekah Reisig; Christopher Grybauskas; Eric Balighian; Emily Frosch; John H Shatzer Jr

doi:10.7759/cureus.7866

The Process of Developing an Assessment Checklist for Simulated Infant Respiratory Distress Using a Modified Delphi Method: A Mixed Methods Study

Cureus. 2020 Apr 28;12(4):e7866. doi: 10.7759/cureus.7866.

Authors

Affiliations

¹ Pediatrics, The Johns Hopkins University School of Medicine, Baltimore, USA.
² Pediatrics, Johns Hopkins University, Baltimore, USA.
³ Medicine, Pediatrics, Johns Hopkins University, Baltimore, USA.
⁴ Psychiatry, Johns Hopkins University, Baltimore, USA.
⁵ School of Education, Johns Hopkins University, Baltimore, USA.
⁶ Medical Informatics, The Johns Hopkins University School of Medicine, Baltimore, USA.

Abstract

Introduction Assessing clinical performance, such as managing respiratory distress, in clinical trainees is challenging yet important. Our objective was to describe and evaluate an integrative and iterative approach to developing a checklist measuring simulated clinical performance for infant respiratory distress. Methods We implemented a five-step modified Delphi process with an embedded qualitative component. An implementation period occurred followed by a second qualitative data collection. Validity evidence was collected throughout the process. Results A 19-item assessment checklist was developed for managing infant respiratory distress by medical student learners in a simulation-based setting. The iterative process provided content validity while the qualitative data provided response process validity. Cohen kappa was 0.82 indicating strong rater agreement. The assessment checklist was found to be easy to use and measure what was intended. Conclusion We developed an accurate and reliable assessment checklist for medical student learners in a simulation-based learning setting with high interrater reliability and validity evidence. Given its ease of use, we encourage medical educators and researchers to utilize this method to develop and implement assessment checklists for their interventions.

Keywords: assessment; checklist development; delphi method; medical education; quantitative and mixed methods research; respiratory distress; simulation in medical education; simulation-based learning.