Development and validation of a computer-based situational judgement test to assess medical students' communication skills in the field of shared decision making

Abstract

Objective: To develop a computer-based test (CBT) measuring medical students' communication skills in the field of shared decision making (SDM) and to evaluate its construct validity.

Methods: The CBT was developed in the context of an experimental study comparing three different trainings for SDM (including e-learning and/or role-play) and a control group. Assessment included a CBT (Part A: seven context-poor questions, Part B: 15 context-rich questions) and interviews with two simulated patients (SP-assessment). Cronbach's α was used to test the internal consistency. Correlations between CBT and SP-assessment were used to further evaluate construct validity of the CBT.

Results: Seventy-two students took part in the study. Mean value for the CBT score was 72% of the total score. Cronbach's α was 0.582. After eliminating three items, Cronbach's α increased to 0.625. Correlations between the CBT and SP-assessment were low to moderate. The control group scored significantly lower than the training settings (p<0.001).

Conclusion: The CBT was reliable enough to test for group differences. For summative assessment purposes, considerably more questions would be needed.

Practice implications: We encourage teachers who particularly work with large student numbers to consider CBT as a feasible assessment method for cognitive aspects of communication skills.

Keywords: Communication skills; Computer-based assessment; Medical education; Shared decision making; Situational judgement test.