The clinical skills of medical interviewing, physical examination, and counseling remain vital to the effective care of patients, yet research continues to document serious deficiencies in clinical skills among students and residents. The most important method of evaluation is the direct observation of trainees performing these clinical skills. Standardized patients and other simulation technologies are important and reliable tools for teaching clinical skills and evaluating competence and will be incorporated in the near future as part of the United States Medical Licensing Examination. Standardized patients and simulation, however, cannot and should not replace the direct observation by faculty of trainees' clinical skills with actual patients. Faculty are in the best position to document improvement over time and to certify trainees have attained sophisticated levels of skill in medical interviewing, physical examination, and counseling. Unfortunately, current evidence suggests significant deficiencies in faculty direct observation evaluation skills. The author outlines the nature of the problems in clinical skills and their evaluation by faculty and ends with recommendations to improve the current state of faculty skills in evaluation.