The "Big Dog" effect: variability assessing the causes of error in diagnoses of patients with lung cancer

Abstract

Purpose: The frequency of diagnostic error in patients who have a lung mass and a pathology specimen is as high as 15%. This study examined the role of inter-pathologist agreement in identifying the cause of error in these patients.

Methods: Pathologists from six institutions reviewed the slides of 40 patients who had a pulmonary specimen false-negative diagnosis. The initial assessment of error cause arose from cytologic-histologic correlation slide review of discrepant diagnostic samples in patients who had both a bronchial brushing cytologic and surgical specimen. The cause of error was attributed either to clinical sampling (diagnostic material obtained in one but not the other sample) or interpretation (pathologist failed to identify the salient diagnostic features). The pairwise kappa (kappa) statistic was used to calculate interobserver agreement between the review and original diagnoses and between the separate review diagnoses.

Results: The pairwise kappa statistic ranged widely from -0.154 to 1.0, and the pairwise kappa statistic of the slides from one institution was undetermined because that institutional pathologist never made the assessment that error was secondary to interpretation. Agreement for observers within the same institution was better than agreement between observers from different institutions.

Conclusion: Pathologists exhibit poor agreement in determining the cause of error for pulmonary specimens sent for cancer diagnosis. We developed a psychosocial hypothesis (the "Big Dog" Effect) that partially explains biases in error assessment. This lack of agreement precludes confident targeting of these errors for quality improvement interventions with prospects of success across a variety of institutions.