Comparing two small samples with an unstable, treatment-independent baseline

J Neurosci Methods. 2009 May 15;179(2):173-8. doi: 10.1016/j.jneumeth.2009.01.017. Epub 2009 Jan 31.

Abstract

Due to time and resource constraints, small samples (N=3-7 cases per group) are often used in neurobiological studies that employ multiple techniques. In a simulation study, five statistical tests were used to compare two small samples (treated and control) with an unstable, additive baseline. These five tests differed in the way that they used the baseline variable (B) to adjust or normalize the variable affected by the treatment (Y). We conclude that, if N=3 or 4, the independent t-test on Y-B tends to have the highest power; if N> or =7, ANCOVA on Y with B as the covariate tends to have the highest power; and both tests have comparably high power if N=5 or 6. The Wilcoxon rank-sum test (or, equivalently, the Mann-Whitney test) has precisely zero power if one group has 3 cases and the other has 3 or 4 cases. Some other problems of small-sample analysis are considered.

Publication types

  • Comparative Study
  • Research Support, Non-U.S. Gov't

MeSH terms

  • Algorithms
  • Artifacts
  • Computer Simulation
  • Data Interpretation, Statistical*
  • Humans
  • Models, Statistical
  • Neurosciences / methods*
  • Reproducibility of Results
  • Research Design / statistics & numerical data
  • Sample Size
  • Software