Background: In New Zealand, the family practice certification examination includes an assessment that uses simulated patients. In 1990, for the first time, consumers participated in evaluating the performance of examinees in the simulated-patient section of the examination.
Methods: The consumer examiners and medically trained examiners observed the candidates during interactions with simulated patients. The consumers, medical examiners, and simulated patients independently scored the candidates' performances. One hundred and nine examination candidates participated in two simulations each, and the scoring behavior of each rater was compared.
Results: Consumer and medical examiners showed significant differences in their scoring behaviors, disagreeing on whether the candidate should pass or fail in 7.8% of interviews. In comparing the scores given by consumers, medical examiners, and the simulated patients, low inter-observer correlations were noted.
Conclusion: Medical personnel, consumer observers, and simulated patients all provide a different perspective on the communication skills of physicians.