The objective of this study was to compare the assessment of medical students communication skills made by their academic teachers, with the assessment made by their role-playing 'patients'. It was a cross-sectional study, conducted at the Department of General Practice, University of Sydney, Australia, and consisted of 519 undergraduate medical students. Teachers rated students' communication skills using ten specific criteria, each marked on a five-point Likert scale. Teachers then rated students' overall performance using a 10-point scale. Patients rated students' overall performance on the same 10-point Likert scale. Only two of the 10 criteria, as rated by the academic teachers, correlated with the role-playing patients' overall score, and all 10 criteria accounted for only 10.1% of the variance in that score. The academic assessors' overall score accounted for only 9.7% of the variance of the patients' overall score. The communications skills emphasized by academic teachers do not reflect the skills considered to be important by role-playing patients.