Comparing student self-assessments of global communication with trained faculty and standardized patient assessments

Abstract

Background and purpose: Assess the reliability of first year pharmacy student assessments completed by faculty members in comparison with a standardized patient (SP), and student self-assessments during a structured educational module on communication.

Educational activity and setting: Pharmacy students completed four stations focused on communication with an SP. During each encounter, students completed a self-assessment and were evaluated by a faculty member and a trained SP. A five point Likert scale was used to evaluate student performance. Faculty assessments were compared against all others. A Pearson correlation coefficient for total scores was used and a Cohen's kappa was used to compare inter-rater reliability. Agreement and correlation was performed with student results categorized into poor, adequate, and exceptional performance based on faculty evaluation.

Findings: Twenty-four students participated. In all stations, student self-assessments were graded higher than corresponding faculty and SP assessments. Agreement between faculty, SP, and self-assessment was fair to slight (k < 0.4) for all comparisons but only significant (p < 0.05) between the faculty and self-assessment. After categorization, there was a small, non-significant correlation between faculty and self-assessment (r = 0.13, p = 0.21) and moderate and significant correlation between faculty and SP (r = 0.32, p = 0.001). Categorized inter-rater agreement was fair for all comparisons (k < 0.2) and only significant (p < 0.05) between faculty and SP assessment.

Discussion: Pharmacy students in their first professional year assess their communication skills more positively than other evaluators. Further instruction for students and reflection may be required to build understanding of global assessment in communication.

Summary: There is high incongruity between student self-assessment and faculty appraisal.

Keywords: Assessment; Evaluation; Grading rubric; Self-assessment; Standardized patient assessment.