Evaluating CollaboRATE in a clinical setting: analysis of mode effects on scores, response rates and costs of data collection

Abstract

Background: Shared decision-making (SDM) has become a policy priority, yet its implementation is not routinely assessed. To address this gap we tested the delivery of CollaboRATE, a 3-item patient reported experience measure of SDM, via multiple survey modes.

Objective: To assess CollaboRATE response rates and respondent characteristics across different modes of administration, impact of mode and patient characteristics on SDM performance and cost of administration per response in a real-world primary care practice.

Design: Observational study design, with repeated assessment of SDM performance using CollaboRATE in a primary care clinic over 15 months of data collection. Different modes of administration were introduced sequentially including paper, patient portal, interactive voice response (IVR) call, text message and tablet computer.

Participants: Consecutive patients ≥18 years, or parents/guardians of patients <18 years, visiting participating primary care clinicians.

Main measures: CollaboRATE assesses three core SDM tasks: (1) explanation about health issues, (2) elicitation of patient preferences and (3) integration of patient preferences into decisions. Responses to each item range from 0 (no effort was made) to 9 (every effort was made). CollaboRATE scores are calculated as the proportion of participants who report a score of nine on each of the three CollaboRATE questions.

Key results: Scores were sensitive to mode effects: the paper mode had the highest average score (81%) and IVR had the lowest (61%). However, relative clinician performance rankings were stable across the different data collection modes used. Tablet computers administered by research staff had the highest response rate (41%), although this approach was costly. Clinic staff giving paper surveys to patients as they left the clinic had the lowest response rate (12%).

Conclusions: CollaboRATE can be introduced using multiple modes of survey delivery while producing consistent clinician rankings. This may allow routine assessment and benchmarking of clinician and clinic SDM performance.

Keywords: PRIMARY CARE; mode effects; patient-reported experience measure; patient-reported measurement; shared decision-making.