Interobserver agreement in emergency department triage

Ann Emerg Med. 2003 Feb;41(2):191-5. doi: 10.1067/mem.2003.47.


Study objectives: We measure the interobserver reliability of the triage process, examine the effect of vital signs on the triage process, and provide a context for the prior observation of poor interobserver agreement between in-person and telephonic interviews.

Methods: We performed a prospective observational study using a randomized crossover design at a university teaching hospital emergency department. Patients were eligible if they spoke English, were not presenting for a reevaluation, and were unlikely to be harmed by the delay created by a second triage interview. Every eligible patient underwent 2 independent, sequential, in-person ED intake interviews conducted by experienced ED triage nurses. After taking a history, each nurse chose 1 of 5 hypothetic triage designations (ED by 911, ED within 2 hours, see a physician within 8 hours, see a physician within 24 hours, or home care-see a physician in >24 hours) and, after being told the patient's vital signs, again selected a designation.

Results: Three hundred sixty-three patients presented during the study period: 113 were ineligible, 34 were missed by the investigators, and 15 refused to participate. Nineteen nurses participated in the triage of the 201 study patients. Agreement between 2 in-person designations made without knowledge of vital signs was poor (percent agreement 53%; kappa=0.30; tau(b)=0.50). Knowledge of vital signs did not improve agreement (percent agreement 49%; kappa=0.25; tau(b)=0.45).

Conclusion: There was poor interobserver agreement between certified triage nurses using a 5-item triage scale designed for telephonic triage. These findings suggest that only a small portion of the poor interobserver agreement observed in a prior study of telephonic versus in-person triage can be attributed to the use of the telephone.

Publication types

  • Clinical Trial
  • Randomized Controlled Trial
  • Research Support, Non-U.S. Gov't

MeSH terms

  • Cross-Over Studies
  • Emergency Service, Hospital / organization & administration*
  • Hospitals, University / organization & administration
  • Humans
  • Los Angeles
  • Nursing Staff, Hospital
  • Observer Variation
  • Prospective Studies
  • Telephone
  • Triage / organization & administration
  • Triage / standards*
  • Triage / statistics & numerical data