Communicating data about the benefits and harms of treatment: a randomized trial

Abstract

Background: Despite limited evidence, it is often asserted that natural frequencies (for example, 2 in 1000) are the best way to communicate absolute risks.

Objective: To compare comprehension of treatment benefit and harm when absolute risks are presented as natural frequencies, percents, or both.

Design: Parallel-group randomized trial with central allocation and masking of investigators to group assignment, conducted through an Internet survey in September 2009. (ClinicalTrials.gov registration number: NCT00950014)

Setting: National sample of U.S. adults randomly selected from a professional survey firm's research panel of about 30,000 households.

Participants: 2944 adults aged 18 years or older (all with complete follow-up).

Intervention: Tables presenting absolute risks in 1 of 5 numeric formats: natural frequency (x in 1000), variable frequency (x in 100, x in 1000, or x in 10,000, as needed to keep the numerator >1), percent, percent plus natural frequency, or percent plus variable frequency.

Measurements: Comprehension as assessed by 18 questions (primary outcome) and judgment of treatment benefit and harm.

Results: The average number of comprehension questions answered correctly was lowest in the variable frequency group and highest in the percent group (13.1 vs. 13.8; difference, 0.7 [95% CI, 0.3 to 1.1]). The proportion of participants who "passed" the comprehension test (≥13 correct answers) was lowest in the natural and variable frequency groups and highest in the percent group (68% vs. 73%; difference, 5 percentage points [CI, 0 to 10 percentage points]). The largest format effect was seen for the 2 questions about absolute differences: the proportion correct in the natural frequency versus percent groups was 43% versus 72% (P < 0.001) and 73% versus 87% (P < 0.001).

Limitation: Even when data were presented in the percent format, one third of participants failed the comprehension test.

Conclusion: Natural frequencies are not the best format for communicating the absolute benefits and harms of treatment. The more succinct percent format resulted in better comprehension: Comprehension was slightly better overall and notably better for absolute differences.

Primary funding source: Attorney General Consumer and Prescriber Education grant program, the Robert Wood Johnson Pioneer Program, and the National Cancer Institute.