Using Standardized Videos to Examine the Validity of the Shared Decision Making Process Scale: Results of a Randomized Online Experiment

Abstract

Background: The Shared Decision Making (SDM) Process scale is a brief, patient-reported measure of SDM with demonstrated validity in surgical decision making studies. Herein we examine the validity of the scores in assessing SDM for cancer screening and medication decisions through standardized videos of good-quality and poor-quality SDM consultations.

Method: An online sample was randomized to a clinical decision-colon cancer screening or high cholesterol-and a viewing order-good-quality video first or poor-quality video first. Participants watched both videos, completing a survey after each video. Surveys included the SDM Process scale and the 9-item SDM Questionnaire (SDM-Q-9); higher scores indicated greater SDM. Multilevel linear regressions identified if video, order, or their interaction predicted SDM Process scores. To identify how the SDM Process score classified videos, area under the curve (AUC) was calculated. The correlation between SDM Process score and SDM-Q-9 assessed construct validity. Heterogeneity analyses were conducted.

Results: In the sample of 388 participants (68% white, 70% female, average age 45 years) good-quality videos received higher SDM Process scores than poor-quality videos (Ps < 0.001), and those who viewed the good-quality high cholesterol video first tended to rate the videos higher. SDM Process scores were related to SDM-Q-9 scores (rs > 0.58; Ps < 0.001). AUC was poor (0.69) for the high cholesterol model and fair (0.79) for the colorectal cancer model. Heterogeneity analyses suggested individual differences were predictive of SDM Process scores.

Conclusion: SDM Process scores showed good evidence of validity in a hypothetical scenario but were lacking in ability to classify good-quality or poor-quality videos accurately. Considerable heterogeneity of scoring existed, suggesting that individual differences played a role in evaluating good- or poor-quality SDM conversations.

Keywords: measurement; shared decision making; validity.