Skip to main page content
Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Comparative Study
. 2003 Nov;13(4):675-89.
doi: 10.1081/BIP-120024202.

Comparison of Methods for Estimating the Number of True Null Hypotheses in Multiplicity Testing

Affiliations
Comparative Study

Comparison of Methods for Estimating the Number of True Null Hypotheses in Multiplicity Testing

Huey-miin Hsueh et al. J Biopharm Stat. .

Abstract

When a large number of statistical tests is performed, the chance of false positive findings could increase considerably. The traditional approach is to control the probability of rejecting at least one true null hypothesis, the familywise error rate (FWE). To improve the power of detecting treatment differences, an alternative approach is to control the expected proportion of errors among the rejected hypotheses, the false discovery rate (FDR). When some of the hypotheses are not true, the error rate from either the FWE- or the FDR-controlling procedure is usually lower than the designed level. This paper compares five methods used to estimate the number of true null hypotheses over a large number of hypotheses. The estimated number of true null hypotheses is then used to improve the power of FWE- or FDR-controlling methods. Monte Carlo simulations are conducted to evaluate the performance of these methods. The lowest slope method, developed by Benjamini and Hochberg (2000) on the adaptive control of the FDR in multiple testing with independent statistics, and the mean of differences method appear to perform the best. These two methods control the FWE properly when the number of nontrue null hypotheses is small. A data set from a toxicogenomic microarray experiment is used for illustration.

Similar articles

See all similar articles

Cited by 31 articles

See all "Cited by" articles

Publication types

LinkOut - more resources

Feedback