Pulmonary embolism risk assessment screening tools: the interrater reliability of their criteria

Am J Emerg Med. 2007 Mar;25(3):285-90. doi: 10.1016/j.ajem.2006.08.016.


Background: Diagnostic evaluation for suspected pulmonary embolism (PE) is challenging. Dimerized plasmin fragment D (D-dimer) assays are increasingly used but have been validated only in "low-risk" patients. The accurate interpretation and application of risk assessment criteria are critical to the appropriate use of D-dimer. We sought to determine the interrater agreement of attending and third-year resident emergency medicine physicians in the specific elements of the Canadian and the Charlotte risk stratification tools and their clinical application.

Methods: We prospectively enrolled a convenience sample of patients presenting to an urban university emergency department with suspected PE. Standardized data collection sheets were used by an attending physician and a third-year resident physician to determine the presence or absence of risk factors included in published PE prediction instruments. Each physician was blinded to the other's results and the patients' D-dimer result. Interrater agreement was measured using kappa statistics (with 95% confidence intervals).

Results: Two hundred seventy-one patients were screened. The kappa scores for each risk criterion were as follows: previous deep vein thrombosis, 0.90 (95% confidence interval, 0.83-0.97); malignancy, 0.87 (0.76-0.97); deep vein thrombosis symptoms, 0.54 (0.39-0.70); immobilization, 0.41 (0.26-0.57); unexplained hypoxia, 0.58 (0.42-0.74); tachycardia, 0.94 (0.89-0.98); hemoptysis, 0.76 (0.51-1.0); and PE more likely than another diagnosis, 0.50 (0.36-0.64).

Conclusions: Interrater agreement was only fair for several important risk criteria. Small differences in determining pretest probability can lead to significant variability in risk assessment and how, or whether, the diagnosis of PE is evaluated. This study raises questions about the reliability and applicability of published PE screening criteria in clinical settings.

Publication types

  • Multicenter Study
  • Research Support, N.I.H., Extramural
  • Research Support, Non-U.S. Gov't

MeSH terms

  • Adult
  • Emergency Service, Hospital / statistics & numerical data*
  • Female
  • Fibrin Fibrinogen Degradation Products / isolation & purification*
  • Humans
  • Male
  • Mass Screening / methods*
  • Middle Aged
  • Nephelometry and Turbidimetry
  • Observer Variation
  • Prospective Studies
  • Pulmonary Embolism / diagnosis*
  • Risk Assessment


  • Fibrin Fibrinogen Degradation Products
  • fibrin fragment D