Reacting to prognostic covariate imbalance in randomised controlled trials

Abstract

Clinical trialists may regard an observed imbalance on a prognostic covariate as sufficiently troubling to warrant action.

Objective: To elucidate the issues associated with selecting, and switching between, an unadjusted versus an adjusted analysis in response to an observed covariate imbalance.

Study design and setting: Simulation study performed under the null hypothesis of no treatment effect using data from a large secondary prevention trial of statin therapy. The operating characteristics of three reaction strategies to baseline imbalances observed post-hoc were assessed.

Results: Unadjusted analyses produced valid p-values irrespective of chance imbalance on a prognostic covariate. Switching to an adjusted analysis introduced no bias when the decision was made without knowledge of the direction of the imbalance. When the decision was based on the direction of the imbalance, the risk of incorrectly declaring the experimental treatment superior was inflated (by up to 48% in the scenarios investigated).

Conclusion: Overreaction to baseline imbalances observed post-hoc is unwarranted and we support adherence to the ICH guideline recommendations on the use of covariates. A legitimate case for switching to an adjusted analysis prior to finalisation of the statistical analysis plan (SAP) could nevertheless be potentially made provided that the direction of an observed covariate imbalance is unknown. Investigators should avoid reviewing the distribution of baseline characteristics across randomised groups in an unblinded fashion, for open-label and blinded studies alike, prior to finalisation of the SAP.

What is new: ICH guidelines on adjustment for covariates in RCT analyses appropriately advise against overreaction to baseline imbalances observed post-hoc. CONSORT reporting guidelines nevertheless place an emphasis on comparability of baseline characteristics across randomised groups. We demonstrate through a series of simulation studies why the ICH guidance is sound, but that a switch to an adjusted analysis in reaction to an observed prognostic covariate imbalance could legitimately be made provided that, when reaching the decision, treatment allocation is masked, and the direction of the imbalance is unknown. Trialists should therefore consider preserving the masking of actual treatment assignment when assessing the distribution of baseline characteristics across randomised groups.

Keywords: Baseline imbalance; Covariate adjustment; Randomised controlled trial.