Confidence intervals for an effect size measure based on the Mann-Whitney statistic. Part 1: general issues and tail-area-based methods

Abstract

For two random variables X and Y, theta = Pr[Y > X] + (1/2)Pr[Y = X] is advocated as a general measure of effect size to characterize the degree of separation of their distributions. It is estimated by U/mn, a generalization of the Mann-Whitney U statistic, derived by dividing U by the product of the two sample sizes. It is equivalent to the area under the receiver operating characteristic curve. It is readily visualized in terms of two Gaussian distributions with appropriately separated peaks. The effect of discretization of a continuous variable is explored. Tail-area-based confidence interval methods are developed which can be applied to very small samples or extreme outcomes.