Bias in observational studies of prevalent users: lessons for comparative effectiveness research from a meta-analysis of statins

Abstract

Randomized clinical trials (RCTs) are usually the preferred strategy with which to generate evidence of comparative effectiveness, but conducting an RCT is not always feasible. Though observational studies and RCTs often provide comparable estimates, the questioning of observational analyses has recently intensified because of randomized-observational discrepancies regarding the effect of postmenopausal hormone replacement therapy on coronary heart disease. Reanalyses of observational data that excluded prevalent users of hormone replacement therapy led to attenuated discrepancies, which begs the question of whether exclusion of prevalent users should be generally recommended. In the current study, the authors evaluated the effect of excluding prevalent users of statins in a meta-analysis of observational studies of persons with cardiovascular disease. The pooled, multivariate-adjusted mortality hazard ratio for statin use was 0.77 (95% confidence interval (CI): 0.65, 0.91) in 4 studies that compared incident users with nonusers, 0.70 (95% CI: 0.64, 0.78) in 13 studies that compared a combination of prevalent and incident users with nonusers, and 0.54 (95% CI: 0.45, 0.66) in 13 studies that compared prevalent users with nonusers. The corresponding hazard ratio from 18 RCTs was 0.84 (95% CI: 0.77, 0.91). It appears that the greater the proportion of prevalent statin users in observational studies, the larger the discrepancy between observational and randomized estimates.

Figure 1.

Processing of A) randomized trials and B) observational studies in a review and meta-analysis of observational studies of statin therapy. One randomized trial and 2 observational studies included comparisons of both primary and secondary prevention. (CHD, coronary heart disease).

Figure 2.

Hazard ratio (HR) for mortality (squares) according to initiation of statin use in secondary prevention randomized clinical trials and pooled HR for treatment versus control status (diamond). Bars, 95% confidence interval (CI). (GISSI, Gruppo Italiano per lo Studio della Sopravvivenza nell’Infarto Miocardico; HPS, Heart Protection Study; LIPID, Long-term Intervention with Pravastatin in Ischaemic Disease; MAAS, Multicentre Anti-Atheroma Study; 4S, Scandinavian Simvastatin Survival Study).

Figure 3.

Hazard ratio (HR) for mortality (squares) according to statin use in secondary-prevention observational studies comparing prevalent users with nonusers and pooled HR for users versus nonusers (diamond). A) unadjusted results; B) adjusted results. Bars, 95% confidence interval (CI).

Figure 4.

Hazard ratio (HR) for mortality (squares) according to statin use in secondary-prevention observational studies comparing a combination of prevalent and incident users with nonusers and pooled HR for users versus nonusers (diamond). A) unadjusted results; B) adjusted results. Bars, 95% confidence interval (CI). (ITA, internal thoracic artery graft).

Figure 5.

Multivariate-adjusted hazard ratio (HR) for mortality (squares) according to statin use in secondary-prevention observational studies comparing incident users with nonusers and pooled HR for users versus nonusers (diamond). Bars, 95% confidence interval (CI).

Figure 6.

Proportion of patients stopping statin treatment in A) primary prevention studies and B) secondary prevention studies, by type of study and duration of follow-up.