Background: The consistency of rating among different standardized patients (SPs) is very important in clinical performance examinations (CPX).
Aims: We evaluated the usefulness of simulated students to assess the reliability of a group of standardized patients in a CPX.
Methods: Five SPs in each case were trained to simulate the same patient in a CPX. Ten fourth-year medical student volunteers were selected to act as simulated students. After the simulated students encountered the first SP, they reviewed a video of their performance and sequentially repeated the performance with the other 4 SPs.
Results: The average maximum difference (Deltamax +/- SD) of multiple SPs' percent scores on a simulated student was 18 +/- 7. In 10 pairs of SPs who gave largest Deltamax on a given student, the median Deltamax was 30 (range, 26-38) while median difference of percent score in the faculty scoring on the same simulated student was 3 (range, 0-7). Five problematic SPs whose ratings were significantly different from 3 or more other SPs' ratings in each case were identified.
Conclusions: Simulated students may be a useful means to check the consistency of rating among different SPs in a CPX.