Simultaneous and selective inference: Current successes and future challenges

Abstract

The previous decade can be viewed as a second golden for era Multiple Comparisons research. I argue that much of the success stems from our being able to address real current needs. At the same time, this success generated a plethora of concepts for error rate and power, as well as multiplicity of methods for addressing them. These confuse the users of our methodology and pose a threat. To avoid the threat, it is our responsibility to match our theoretical goals to the goals of the users of statistics. Only then should we match the methods to the theoretical goals. Considerations related to such needs are discussed: simultaneous inference or selective inference, testing or estimation, decision making or scientific reporting. I then further argue that the vitality of our field in the future - as a research area - depends upon our ability to continue and address the real needs of statistical analyses in current problems. Two application areas offering new challenges have received less attention in our community to date are discussed. Safety analysis in clinical trials, where I offer an aggregated safety assessment methodology and functional Magnetic Resonance Imaging.