Background: Makuch and Simon [Sample size considerations for non-randomised comparative studies. J Chronic Dis 1980; 33: 175-81.] developed a sample size formula for historical control trials. When assessing power, they assumed the true control treatment effect to be equal to the observed effect from the historical control group. Many researchers have pointed out that the Makuch-Simon approach does not preserve the nominal power and type I error when considering the uncertainty in the true historical control treatment effect.
Purpose: To develop a sample size formula that properly accounts for the underlying randomness in the observations from the historical control group.
Methods: We reveal the extremely skewed nature in the distributions of power and type I error, obtained over all the random realizations of the historical control data. The skewness motivates us to derive a sample size formula that controls the percentiles, instead of the means, of the power and type I error.
Results: A closed-form sample size formula is developed to control arbitrary percentiles of power and type I error for historical control trials. A simulation study further demonstrates that this approach preserves the operational characteristics in a more realistic scenario where the population variances are unknown and replaced by sample variances.
Limitations: The closed-form sample size formula is derived for continuous outcomes. The formula is more complicated for binary or survival time outcomes.
Conclusions: We have derived a closed-form sample size formula that controls the percentiles instead of means of power and type I error in historical control trials, which have extremely skewed distributions over all the possible realizations of historical control data.