There is an increasing demand for methods to objectively assess a surgeon's competence. The purpose of this study is to test two evaluation methods that assess a surgeon's skills in the removal of mandibular third molar teeth. One method uses an objective checklist scale by which twenty components of the operation are judged as correct or incorrect. The second method is a global rating scale, that scores different aspects of performance, such as respect for tissue, number of manoeuvres, knowledge of procedure and instrument handling. Using these two methods, assessors judged the removal of 22 mandibular third molar teeth by 17 different surgeons. Good inter-rater reliability was achieved using both scales. A greater preference was expressed for the global rating scale for assessment and for the objective checklist scale for guiding feedback in training. Scales such as these need to be developed for, and tested on, different surgical procedures in order that credible assessment of surgical skills can be obtained.