Skip to main page content
Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2006 Aug 26;333(7565):417.
doi: 10.1136/bmj.38926.629329.AE. Epub 2006 Aug 14.

Developing a Quality Criteria Framework for Patient Decision Aids: Online International Delphi Consensus Process

Free PMC article

Developing a Quality Criteria Framework for Patient Decision Aids: Online International Delphi Consensus Process

Glyn Elwyn et al. BMJ. .
Free PMC article


Objective: To develop a set of quality criteria for patient decision support technologies (decision aids).

Design and setting: Two stage web based Delphi process using online rating process to enable international collaboration.

Participants: Individuals from four stakeholder groups (researchers, practitioners, patients, policy makers) representing 14 countries reviewed evidence summaries and rated the importance of 80 criteria in 12 quality domains on a 1 to 9 scale. Second round participants received feedback from the first round and repeated their assessment of the 80 criteria plus three new ones.

Main outcome measure: Aggregate ratings for each criterion calculated using medians weighted to compensate for different numbers in stakeholder groups; criteria rated between 7 and 9 were retained.

Results: 212 nominated people were invited to participate. Of those invited, 122 participated in the first round (77 researchers, 21 patients, 10 practitioners, 14 policy makers); 104/122 (85%) participated in the second round. 74 of 83 criteria were retained in the following domains: systematic development process (9/9 criteria); providing information about options (13/13); presenting probabilities (11/13); clarifying and expressing values (3/3); using patient stories (2/5); guiding/coaching (3/5); disclosing conflicts of interest (5/5); providing internet access (6/6); balanced presentation of options (3/3); using plain language (4/6); basing information on up to date evidence (7/7); and establishing effectiveness (8/8).

Conclusions: Criteria were given the highest ratings where evidence existed, and these were retained. Gaps in research were highlighted. Developers, users, and purchasers of patient decision aids now have a checklist for appraising quality. An instrument for measuring quality of decision aids is being developed.


Fig 1
Fig 1
Example of a criterion (second round). Quality domain=presenting probabilities
Fig 2
Fig 2
Criteria for which stakeholder effects were present at second round (see table for data and P values)

Similar articles

See all similar articles

Cited by 447 articles

See all "Cited by" articles

Publication types