Psychometrics of the observational scales of the Utrecht Scale for Evaluation of Rehabilitation (USER): Content and structural validity, internal consistency and reliability

Michael Jansen; Arno J Doornebosch; Margot Wm de Waal; Elizabeth M Wattel; Dennis Visser; Bea Spek; Ewout B Smit

doi:10.1016/j.archger.2021.104509

Psychometrics of the observational scales of the Utrecht Scale for Evaluation of Rehabilitation (USER): Content and structural validity, internal consistency and reliability

Arch Gerontol Geriatr. 2021 Nov-Dec:97:104509. doi: 10.1016/j.archger.2021.104509. Epub 2021 Aug 26.

Authors

Michael Jansen¹, Arno J Doornebosch², Margot Wm de Waal², Elizabeth M Wattel³, Dennis Visser³, Bea Spek⁴, Ewout B Smit⁵

Affiliations

¹ Faculty of health, Physiotherapy, University of applied sciences Leiden, Leiden, The Netherlands; Woon Zorgcentra Haaglanden (WZH), The Hague, The Netherlands. Electronic address: jansen.mic@hsleiden.nl.
² Department of Public Health and Primary Care, Leiden University Medical Centre, Leiden, The Netherland.
³ Department of Medicine for Older People Amsterdam University Medical Centres - VU Amsterdam, Amsterdam, The Netherlands.
⁴ Department of Epidemiology and Data Science, Amsterdam University Medical Centres, University of Amsterdam, Amsterdam, The Netherlands.
⁵ Department of Medicine for Older People Amsterdam University Medical Centres - VU Amsterdam, Amsterdam, The Netherlands; Vivium Zorggroep Naarden, The Netherlands. Electronic address: eb.smit@amsterdamumc.nl.

PMID: 34509903
DOI: 10.1016/j.archger.2021.104509

Abstract

Introduction: Establish content and structural validity, internal consistency, inter-rater reliability, and measurement error of the physical and cognitive scales of the Utrecht Scale for Evaluation clinical Rehabilitation (USER) in geriatric rehabilitation.

Material and methods: First, an expert consensus-meeting (N=7) was organised for content validity wherein scale content validity index (CVI) was measured. Second, in a sample of geriatric rehabilitation patient structural validity (N=616) was assessed by confirmatory factor analyses for exploring unidimensionality. Cut-off criteria were: Root Mean Square Error of Approximation (RMSEA) ≤0.08; Comparative Fit Index (CFI) and Tucker Lewis Index (TLI) ≥0.95. Local independence (residual correlation<0.20) and monotonicity (H_i-coefficient ≥0.30 and H_s-coefficient ≥0.50) were also calculated. Cronbach alphas were calculated for internal consistency. Alpha's > 0.7 was considered adequate. Third, two nurses independently administered the USER to 37 patients. Intraclass-correlation coefficients (ICC) were calculated for inter-rater reliability (IRR), standard error of measurement (SEM) and Smallest Detectable Change (SDC).

Results: The CVI for physical functioning was moderate (0.73) and excellent for cognitive functioning (0.97). Structural validity physical scale was acceptable (CFI;0.95, TLI;0.93, RMSEA;0.07, ECV;0.78, OmegaH;0.87; Monotonicity;(H_i;0.52-0.75 and H_s;0.63)). Cognitive scale was good (CFI;0.98, TLI;0.96, RMSEA;0.05, ECV;0.66 and OmegaH;0.90. Monotonicity;(H_i;0.30 -0.70 and H_s;0.61)). Cronbach's alpha were high: physical scale;0.92 and cognitive scale;0.94. Reliability physical scale ICC;0.94, SEM;5 and SDC;14 and cognitive scale ICC;0.88, SEM;5 and SDC;13.

Conclusion: The observational scales of the USER have shown sufficient content and structural validity, internal consistency, and interrater reliability for measuring physical and cognitive function in geriatric rehabilitation.

Trial registration: N/A.

Keywords: Clinimetrics; Cognitive functioning; Geriatric rehabilitation; Physical functioning; Post-acute care.

Publication types

Research Support, Non-U.S. Gov't

MeSH terms

Aged
Factor Analysis, Statistical
Humans
Psychometrics*
Reproducibility of Results
Surveys and Questionnaires