Properties of internal pilots with the univariate approach to repeated measures

Abstract

Uncertainty surrounding the error covariance matrix often presents the biggest barrier to achieving accurate power analysis in the 'univariate' approach to repeated measures analysis of variance (UNIREP). A poor choice gives either an overpowered study which wastes resources, or an underpowered study with little chance of success. Internal pilot designs were introduced to resolve such uncertainty about error variance for t-tests. In earlier papers, we extended the use of internal pilots to any univariate linear model with fixed predictors and independent Gaussian errors. Here we further extend our exact and approximate results to UNIREP analysis. For a fixed treatment effect, the inaccuracy in a power calculation depends only on the ratio of the true variance to the value used for planning. The greater complexity of repeated measures requires generalizing misspecification of error variance to the misspecification of the eigenvalues of the error covariance. We recommend approximating the misspecification in terms of the first and second moments of the eigenvalues, for both fixed sample and internal pilot designs. We also describe an unadjusted approach for internal pilots with repeated measures. Simulations illustrate the fact that both positive and negative properties in the univariate setting extend to repeated measures analysis. In particular, internal pilots allow maintaining power or reducing expected sample size when the covariance matrix used for planning differs from the true value. However, an unadjusted approach can inflate test size, at least with small to moderate sample sizes. Hence new, adjusted methods must be developed for small samples. At this time, we caution against using an internal pilot design with repeated measures without first conducting simulations to document the amount of test size inflation possible for the conditions of interest.