Signal detection is routinely applied to spontaneous report safety databases in the pharmaceutical industry and by regulators. As an example, methods that search for increases in the frequencies of known adverse drug reactions for a given drug are routinely applied, and the results are reported to the health authorities on a regular basis. Such methods need to be sensitive to detect true signals even when some of the adverse drug reactions are rare. The methods need to be specific and account for multiplicity to avoid false positive signals when the list of known adverse drug reactions is long. To apply them as part of a routine process, the methods also have to cope with very diverse drugs (increasing or decreasing number of cases over time, seasonal patterns, very safe drugs versus drugs for life-threatening diseases). In this paper, we develop new nonparametric signal detection methods, directed at detecting differences between a reporting and a reference period, or trends within a reporting period. These methods are based on bootstrap and permutation distributions, and they combine statistical significance with clinical relevance. We conducted a large simulation study to understand the operating characteristics of the methods. Our simulations show that the new methods have good power and control the family-wise error rate at the specified level. Overall, in all scenarios that we explored, the method performs much better than our current standard in terms of power, and it generates considerably less false positive signals as compared to the current standard.
Keywords: bootstrap; high-dimensional data; nonparametric tests; permutation tests.
Copyright © 2018 John Wiley & Sons, Ltd.