Treatments effects from randomized trials and propensity score analyses were similar in similar populations in an example from cardiac surgery

Abstract

Objective: Analyses comparing randomized to nonrandomized clinical trials suffer from the fact that the study populations are usually different. We aimed for a comparison of randomized clinical trials (RCTs) and propensity score (PS) analyses in similar populations.

Study design and setting: In a systematic review, we "meta-matched" RCTs and PS analyses that compared the off- and the on-pump technique in coronary artery bypass grafting. "Meta-confounders" were summarized in a "meta-propensity score" and were used for "meta-matching." We compared treatment effects between RCTs and PS analyses for 10 previously defined binary clinical outcomes in this "meta-matched" population as differences in "meta-odds ratios."

Results: For all clinical outcomes, the estimated differences in "meta-odds ratios" were below an absolute value of 0.15, all confidence intervals included the null.

Conclusions: In our example, treatment effects of off-pump versus on-pump surgery from RCTs and PS analyses were very similar in a "meta-matched" population of studies, indicating that only a small remaining bias is present in PS analyses.