Inter- and intra-rater reliability of the OCRA checklist method in video-recorded manual work tasks

Ida-Märta Rhén; Mikael Forsman

doi:10.1016/j.apergo.2019.103025

Inter- and intra-rater reliability of the OCRA checklist method in video-recorded manual work tasks

Appl Ergon. 2020 Apr:84:103025. doi: 10.1016/j.apergo.2019.103025. Epub 2020 Jan 10.

Authors

Ida-Märta Rhén¹, Mikael Forsman²

Affiliations

¹ Department of Industrial and Materials Science, Chalmers University of Technology, SE-412 96, Gothenburg, Sweden; IMM Institute of Environmental Medicine, Karolinska Institutet, SE-171 77, Stockholm, Sweden; Centre for Occupational and Environmental Medicine, Stockholm County Council, SE-113 65, Stockholm, Sweden. Electronic address: ida-marta.rhen@ki.se.
² IMM Institute of Environmental Medicine, Karolinska Institutet, SE-171 77, Stockholm, Sweden; Centre for Occupational and Environmental Medicine, Stockholm County Council, SE-113 65, Stockholm, Sweden; School of Engineering Sciences in Chemistry, Biotechnology and Health, KTH Royal Institute of Technology, SE-141 57, Huddinge, Sweden.

PMID: 31987511
DOI: 10.1016/j.apergo.2019.103025

Abstract

Observation-based risk assessments are often used for evaluation of biomechanical workload and the condition of the workplace ergonomics. However, the inter- and intra-rater reliability of the methods used for the assessments are insufficiently studied. The OCRA checklist method, included as a reference method in the ISO and CEN standards regarding upper limb repetitive risk assessment, was in this study investigated regarding its reliability. Eleven ergonomists were educated within the method. Each ergonomist risk-assessed ten video-recorded manual work tasks, on two occasions. The average inter-rater agreement of the five overall risk levels was 39% and the linearly weighted kappa was 0.43, the intra-rater dittos were 45% and 0.52. Both indicating a moderate reliability, but lower than what could be expected. The levels might be sufficient for coarse surveys. However, for risk assessments before and after an intervention, complementing technical methods should be considered.

Keywords: Inter-rater reliability; Intra-rater reliability; Observational methods; Risk assessment.

MeSH terms

Biomechanical Phenomena
Checklist*
Cumulative Trauma Disorders / physiopathology*
Female
Humans
Male
Occupations*
Reproducibility of Results
Risk Assessment / methods*
Task Performance and Analysis*
Upper Extremity / physiopathology*
Video Recording*