Power Analysis for Null Hypothesis Significance Testing

Abstract

Before conducting a scientific study, a power analysis is performed to determine the sample size required to test an effect within allowable probabilities of Type I error (α) or Type II error (β). The power of a study is related to Type II error by 1-β. Most scientific studies set α=0.05 and power=0.80 as minimums. More conservative study designs will decrease α or increase power, which will require a larger sample size. The third and final parameter required for a power analysis is the effect size (ES). ES is a measure of the strength of the observation in the outcome of interest (ie, the dependent variable). ES must be estimated from pilot studies or published values. A small ES will require a larger sample size than a large ES. It is possible to detect statistically significant findings even for very small ES, if the sample size is sufficiently large. Therefore, it is also essential to evaluate whether ES is sufficiently large to be clinically meaningful.