Objective: To determine (a) the feasibility of implementing a system for the evaluation of teaching qualities (SETQ) of faculty in an academic medical centre, (b) the psychometric qualities of the questionnaires that are used for the generation of feedback and (c) how residents evaluate the teaching qualities of faculty members and how faculty rated themselves.
Design: Questionnaire study.
Methods: Residents evaluated the teaching qualities of faculty members and faculty also evaluated themselves. Specialty specific questionnaires were developed for both evaluations. The psychometric qualities of the questionnaires were determined by using exploratory factor analysis and by calculating the reliability coefficients of scale constructs and item-total correlation. Mean, median and range were calculated for all teaching aspects per training programme.
Results: In the course of one year, 16 residency programmes in our academic medical centre implemented the SETQ system for the evaluation of teaching faculty. 398 faculty members and 314 residents were invited to (self-) evaluate; the response rates were 80.9% and 73.6% respectively. Residents conducted 2,520 evaluations in total. Factor analysis resulted in the definition of 5 teaching domains: 'learning climate', 'professional attitude towards residents', 'communication of learning goals', 'evaluation of residents' and 'feedback to residents'. Item-total correlation and reliability were high for both the residents' questionnaire and the self-evaluation questionnaire (Cronbach's alpha: > 0.70), except for the educational aspect 'learning climate' on the self-evaluation questionnaire (alpha: 0.67) Faculty members were rated positively, but the self-evaluation of faculty members was slightly less positive. For both groups 'professional attitude towards residents' was the highest scoring teaching domain and 'communication of learning goals' the lowest one.
Conclusion: Implementing a system for the evaluation of teaching qualities (SETQ) of faculty in an academic hospital was proven feasible. The psychometric qualities of the underlying instruments was sufficient to good. Teaching faculty were evaluated positively, although interdepartmental variations existed.