Objective: To compare the interrater and intergroup agreement in judging physician maloccurrence and compliance with standards of care using the implicit case review process.
Design: Mail survey with questionnaire.
Participants: Case reviews and questionnaires were mailed to 140 board-certified ophthalmologists and 140 board-certified ophthalmologists with fellowship training.
Main outcome measure: Agreement judging maloccurrence and compliance with standard of care within each group and between general ophthalmologists and specialists.
Results: Ninety-seven (35%) questionnaires were returned. Overall, 35% of respondents believed that ophthalmologists in the case reviews committed an error of either commission or omission. Forty-five percent of reviewers believed that physicians did not meet the standard of care. There was good within-group agreement for finding clinical error in management and not meeting the standard of care for all groups (kappa coefficient range: 0.55-0.83; P = < 0.004) except retina specialists (kappa coefficient = 0.12; P = 0.2).
Conclusions: Unstructured implicit case review is not a reliable method for determining physician error or for measuring compliance with standards of care. The process is susceptible to bias, and results may vary with reviewer background or training. Unstructured implicit case review needs to be regarded as a rough screening tool and used accordingly.