Cluster-based computational methods for mass univariate analyses of event-related brain potentials/fields: A simulation study

J Neurosci Methods. 2015 Jul 30;250:85-93. doi: 10.1016/j.jneumeth.2014.08.003. Epub 2014 Aug 13.


Background: In recent years, analyses of event related potentials/fields have moved from the selection of a few components and peaks to a mass-univariate approach in which the whole data space is analyzed. Such extensive testing increases the number of false positives and correction for multiple comparisons is needed.

Method: Here we review all cluster-based correction for multiple comparison methods (cluster-height, cluster-size, cluster-mass, and threshold free cluster enhancement - TFCE), in conjunction with two computational approaches (permutation and bootstrap).

Results: Data driven Monte-Carlo simulations comparing two conditions within subjects (two sample Student's t-test) showed that, on average, all cluster-based methods using permutation or bootstrap alike control well the family-wise error rate (FWER), with a few caveats.

Conclusions: (i) A minimum of 800 iterations are necessary to obtain stable results; (ii) below 50 trials, bootstrap methods are too conservative; (iii) for low critical family-wise error rates (e.g. p=1%), permutations can be too liberal; (iv) TFCE controls best the type 1 error rate with an attenuated extent parameter (i.e. power<1).

Keywords: Cluster-based statistics; ERP; Family-wise error rate; Monte-Carlo simulations; Multiple comparison correction; Threshold free cluster enhancement.

Publication types

  • Research Support, Non-U.S. Gov't

MeSH terms

  • Brain / physiology*
  • Cluster Analysis
  • Computer Simulation
  • Datasets as Topic
  • Electroencephalography / methods*
  • Evoked Potentials*
  • Humans
  • Monte Carlo Method
  • Signal Processing, Computer-Assisted*
  • Software