Sample size considerations of prediction-validation methods in high-dimensional data for survival outcomes

Genet Epidemiol. 2013 Apr;37(3):276-82. doi: 10.1002/gepi.21721. Epub 2013 Mar 7.


A variety of prediction methods are used to relate high-dimensional genome data with a clinical outcome using a prediction model. Once a prediction model is developed from a data set, it should be validated using a resampling method or an independent data set. Although the existing prediction methods have been intensively evaluated by many investigators, there has not been a comprehensive study investigating the performance of the validation methods, especially with a survival clinical outcome. Understanding the properties of the various validation methods can allow researchers to perform more powerful validations while controlling for type I error. In addition, sample size calculation strategy based on these validation methods is lacking. We conduct extensive simulations to examine the statistical properties of these validation strategies. In both simulations and a real data example, we have found that 10-fold cross-validation with permutation gave the best power while controlling type I error close to the nominal level. Based on this, we have also developed a sample size calculation method that will be used to design a validation study with a user-chosen combination of prediction. Microarray and genome-wide association studies data are used as illustrations. The power calculation method in this presentation can be used for the design of any biomedical studies involving high-dimensional data and survival outcomes.

Publication types

  • Research Support, N.I.H., Extramural
  • Research Support, Non-U.S. Gov't

MeSH terms

  • Adenocarcinoma / genetics
  • Adenocarcinoma / mortality
  • Adenocarcinoma of Lung
  • Computer Simulation
  • Genome-Wide Association Study
  • Human Genome Project
  • Humans
  • Lung Neoplasms / genetics
  • Lung Neoplasms / mortality
  • Microarray Analysis / methods
  • Microarray Analysis / statistics & numerical data
  • Mortality*
  • Multiple Myeloma / genetics
  • Multiple Myeloma / mortality
  • Proportional Hazards Models
  • Research Design
  • Sample Size*
  • Validation Studies as Topic*