Rationale, aims and objectives: To assess the criterion validity of paper-and-pencil vignettes to assess guideline adherence by physiotherapists in the Netherlands. The evidence-based physiotherapy practice guideline for low back pain was used as an example.
Methods: Four vignettes were constructed and pre-tested. Three vignettes were found to represent an adequate case-mix. They described one patient with specific low back pain, one with non-specific low back pain and a normal recovery process and one with non-specific low back pain and a delay in the recovery process. Invited to participate were 113 primary care physiotherapists who had joined an randomized controlled trial study 8 months before, in which guideline adherence had been measured by means of semi-structured treatment recording forms. The criterion validity was determined with Spearman's r(s), using Cohen's classification for the behavioural sciences to categorize its effect size.
Results: Of the 72 physiotherapists who agreed to participate, 39 completed the questions on the vignettes. In the end, both adherence measures were available for 34 participants, providing 102 vignettes and 268 recording forms. Mean guideline adherence scores were 57% (SD = 17) when measured by vignettes and 74% (SD = 15) when measured by recording forms. Spearman's r(s) was 0.31 (P = 0.036), which, according to Cohen's classification, is a medium effect size.
Conclusion: Vignettes are of acceptable validity, and are an inexpensive and manageable instrument to measure guideline adherence among large groups of physiotherapists. Further validation studies could benefit from the use of standardized patients as a gold standard, a more diverse case mix to better reflect real physiotherapy practice, and the inclusion of longitudinal vignettes that cover the patients' course of treatment.