The general aims of functional brain magnetic resonance imaging (fMRI) studies are to ascertain which areas of the brain are activated during a specific task, the extent of this activation, whether different groups of subjects demonstrate different patterns of activation, and how these groups behave in different tasks. Many steps are involved in answering such questions and if each step is not carefully controlled the results may be influenced. This work has three objectives. Firstly, to present a technique for quantitatively evaluating methods used in functional imaging data analysis. While receiver-operator-characteristic (ROC) analysis has been used effectively to evaluate the ability of post-processing algorithms to detect true activations while rejecting false activations, it is difficult to adapt such a technique for comparisons of methods for quantitating activations. We present a technique based on the ANOVA, between two or more regions of interest (ROIs), subject groups, or activation tasks, over a range of statistical thresholds, which reveals the sensitivity of different activation quantification metrics to noise and other variables. Secondly, we use this technique to compare two methods of quantifying localized brain activation. There are numerous ways of quantifying the amount of activation present in a specific region of the brain in an individual subject. We compare the pixel count approach, which simply counts the number of pixels above an arbitrary statistical threshold, with an approach based on the sum of t-values above the same arbitrary t-value threshold. Finally, we examine the sensitivity of the results from an analysis of variance, to user defined parameters such as threshold and region of interest size. Both simulated and real functional magnetic resonance data are used to demonstrate these techniques.