Skip to main page content
Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Comparative Study
, 33 (4), 319-23

Triage Tool Inter-Rater Reliability: A Comparison of Live Versus Paper Case Scenarios

Comparative Study

Triage Tool Inter-Rater Reliability: A Comparison of Live Versus Paper Case Scenarios

Andrew Worster et al. J Emerg Nurs.


Background: Published studies of triage scale inter-rater reliability assessment have been conducted mostly using paper case scenarios.

Objective: To determine if this method of inter-rater reliability assessment generated significantly different measures from those generated from live triage cases.

Methods: This was a multicenter, prospective, observational cohort study of a population-based random sample of patients triaged at 2 emergency departments during a period of 4 months. All patients presenting to the emergency department within the study periods were simultaneously and independently triaged using a 5-level triage acuity scale by 2 to 3 research triage nurses blinded to each other's assessment and to the study objective. After 6 months, the same nurses were asked to assign triage scores to paper case scenarios of the same patients that they had each previously triaged.

Results: Each of the 9 research nurses triaged approximately 90 cases. The inter-rater reliabilities as measured by an intraclass correlation coefficient were 0.9 (95% CI = 0.87 to 93) for the live triage assessments and 0.76 (95% CI = 0.73 to 0.79) for the paper case scenarios. The mean triage score assigned to the live cases was significantly less than that assigned to the paper-based cases (3.17; 95% CI = 3.08 to 3.26) (p < 0.001).

Conclusions: There is moderate to high agreement between live cases and paper case scenarios, and the inter-rater reliabilities, although significantly different, are acceptable in both cases. It is impossible to determine which triage setting provides a more accurate triage score but paper case scenarios generally receive lower triage scores than live cases.

Similar articles

See all similar articles

Cited by 14 articles

See all "Cited by" articles