Skip to main page content
Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Observational Study
. 2017 Mar 24;7(3):e014681.
doi: 10.1136/bmjopen-2016-014681.

Evaluating CollaboRATE in a Clinical Setting: Analysis of Mode Effects on Scores, Response Rates and Costs of Data Collection

Free PMC article
Observational Study

Evaluating CollaboRATE in a Clinical Setting: Analysis of Mode Effects on Scores, Response Rates and Costs of Data Collection

Paul J Barr et al. BMJ Open. .
Free PMC article


Background: Shared decision-making (SDM) has become a policy priority, yet its implementation is not routinely assessed. To address this gap we tested the delivery of CollaboRATE, a 3-item patient reported experience measure of SDM, via multiple survey modes.

Objective: To assess CollaboRATE response rates and respondent characteristics across different modes of administration, impact of mode and patient characteristics on SDM performance and cost of administration per response in a real-world primary care practice.

Design: Observational study design, with repeated assessment of SDM performance using CollaboRATE in a primary care clinic over 15 months of data collection. Different modes of administration were introduced sequentially including paper, patient portal, interactive voice response (IVR) call, text message and tablet computer.

Participants: Consecutive patients ≥18 years, or parents/guardians of patients <18 years, visiting participating primary care clinicians.

Main measures: CollaboRATE assesses three core SDM tasks: (1) explanation about health issues, (2) elicitation of patient preferences and (3) integration of patient preferences into decisions. Responses to each item range from 0 (no effort was made) to 9 (every effort was made). CollaboRATE scores are calculated as the proportion of participants who report a score of nine on each of the three CollaboRATE questions.

Key results: Scores were sensitive to mode effects: the paper mode had the highest average score (81%) and IVR had the lowest (61%). However, relative clinician performance rankings were stable across the different data collection modes used. Tablet computers administered by research staff had the highest response rate (41%), although this approach was costly. Clinic staff giving paper surveys to patients as they left the clinic had the lowest response rate (12%).

Conclusions: CollaboRATE can be introduced using multiple modes of survey delivery while producing consistent clinician rankings. This may allow routine assessment and benchmarking of clinician and clinic SDM performance.

Keywords: PRIMARY CARE; mode effects; patient-reported experience measure; patient-reported measurement; shared decision-making.

Conflict of interest statement

Competing interests: GE reports personal fees from Emmi Solutions LLC, personal fees from National Quality Forum, personal fees from Washington State Health Department, personal fees from Shared Decision Making 3rd edition, personal fees from Groups (Radcliffe Press), outside the submitted work; and GE has initiated and led the Option Grid patient decision aids Collaborative, which produces and publishes patient knowledge tools in the form of comparison tables ( GE has been a member of teams that have developed measures of shared decision-making and care integration. These tools and measures are published and are available for use. For further information see


Figure 1
Figure 1
Clinician scores by mode, adjusted for patient characteristics^*. ^While 15 clinicians participated in all four data collection modes, only eight reached 25 patient responses in all modes; therefore, eight of 15 clinicians are shown here. *During the electronic tablet/postal mail phase, responses were not linked to the electronic medical record; as a result, patient demographic data were unavailable.

Similar articles

See all similar articles

Cited by 7 articles

See all "Cited by" articles


    1. Wagner E, Coleman K, Reid R et al. Guiding transformation: how medical practices can become patient-centered medical homes. Commonwealth Fund, 2012. (accessed 7 Dec 2016).
    1. Dyer N, Sorra JS, Smith SA et al. Psychometric properties of the Consumer Assessment of Healthcare Providers and Systems (CAHPS®) Clinician and Group Adult Visit Survey. Med Care 2012;50:S28–34. 10.1097/MLR.0b013e31826cbc0d - DOI - PMC - PubMed
    1. U.S. Department of Health & Human Services. The CAHPS Program. (accessed 24 Aug 2016).
    1. U.S. Department of Health & Human Services. Clinician & Group Survey and Instructions. (accessed 24 Aug 2016).
    1. Kasper J, Heesen C, Köpke S et al. Patients’ and observers’ perceptions of involvement differ. Validation study on inter-relating measures for shared decision making . PLoS ONE 2011;6:e26255 10.1371/journal.pone.0026255 - DOI - PMC - PubMed

Publication types

LinkOut - more resources