Increasing the visibility of coding decisions in team-based qualitative research in nursing

Int J Nurs Stud. 2004 Jan;41(1):15-20. doi: 10.1016/j.ijnurstu.2003.03.001.


Objectives: To examine the use of the multi-rater Kappa measure of agreement (Nonparametric Statistics for the Behavioural Sciences, McGraw Hill, New York, 1988) in team based, mixed method, qualitative nursing research.

Design: The article presents an illustrative description of the application of the qualitative coding procedure and associated multi-rater Kappa measurement at four time points in 9 months amongst a five person health services research team.

Main outcome measures: The multi-rater Kappa statistic. This is a measure of the extent to which observers achieve possible agreement beyond any agreement expected to occur by chance alone.

Results: Closeness to primary qualitative research data, working relationships over time, and focused research team discussion can all lead to greater agreement and convergence at the level of descriptive coding. The method of measuring agreement between groups of coders was easily applied and appeared a feasible option for similar research projects wishing to demonstrate transparency in their coding procedures.

Conclusion: Measuring agreement beyond chance by the multi-rater Kappa statistic has some utility for research teams whose qualitative coding tasks are primarily descriptive. The method offers a standard and transparent approach for demonstrating agreement between coders and should be a feature of qualitative research reporting where appropriate.

Publication types

  • Research Support, Non-U.S. Gov't

MeSH terms

  • Abstracting and Indexing / methods
  • Bias
  • Cooperative Behavior
  • Data Interpretation, Statistical
  • Decision Making, Organizational*
  • Humans
  • Interprofessional Relations
  • Nursing Methodology Research / organization & administration*
  • Patient Care Team / organization & administration*
  • Qualitative Research*
  • Research Design
  • Research Personnel* / organization & administration*
  • Research Personnel* / psychology
  • Sensitivity and Specificity
  • Statistics, Nonparametric*