The principled control of false positives in neuroimaging

Abstract

An incredible amount of data is generated in the course of a functional neuroimaging experiment. The quantity of data gives us improved temporal and spatial resolution with which to evaluate our results. It also creates a staggering multiple testing problem. A number of methods have been created that address the multiple testing problem in neuroimaging in a principled fashion. These methods place limits on either the familywise error rate (FWER) or the false discovery rate (FDR) of the results. These principled approaches are well established in the literature and are known to properly limit the amount of false positives across the whole brain. However, a minority of papers are still published every month using methods that are improperly corrected for the number of tests conducted. These latter methods place limits on the voxelwise probability of a false positive and yield no information on the global rate of false positives in the results. In this commentary, we argue in favor of a principled approach to the multiple testing problem--one that places appropriate limits on the rate of false positives across the whole brain gives readers the information they need to properly evaluate the results.

Fig. 1

Example figure of a hybrid corrected/uncorrected data presentation. Areas that are significant under an uncorrected threshold of P < 0.001 with a 10-voxel extent criteria are shaded in blue. Areas that are significant under a corrected threshold of FDR = 0.05 are shaded in orange.

Fig. 2

Demonstration of correction methods for the multiple testing problem. (a) A raw image of the simulated data used in this example. A field of Gaussian random noise was added to a 100 × 100 image with a 50 × 50 square section of signal in the center. (b) Thresholded image of the simulated data using a pixelwise statistical test. The threshold for this test was P < 0.05. Power is high at 0.80, but a number of false positives can be observed. (c) Thresholded image of the simulated data using a Bonferroni FWER correction. The probability of a familywise error was set to 0.05. There are no false positives across the entire set of tests, but power is reduced to 0.16. (d) Thresholded image of the simulated data while controlling the false discovery rate. The FDR for this example was set to 0.05. Out of the results, 4.9% are known to be false positives but power is increased to 0.54.