Psychophysical measurements in children: challenges, pitfalls, and considerations

Abstract

Measuring sensory sensitivity is important in studying development and developmental disorders. However, with children, there is a need to balance reliable but lengthy sensory tasks with the child's ability to maintain motivation and vigilance. We used simulations to explore the problems associated with shortening adaptive psychophysical procedures, and suggest how these problems might be addressed. We quantify how adaptive procedures with too few reversals can over-estimate thresholds, introduce substantial measurement error, and make estimates of individual thresholds less reliable. The associated measurement error also obscures group differences. Adaptive procedures with children should therefore use as many reversals as possible, to reduce the effects of both Type 1 and Type 2 errors. Differences in response consistency, resulting from lapses in attention, further increase the over-estimation of threshold. Comparisons between data from individuals who may differ in lapse rate are therefore problematic, but measures to estimate and account for lapse rates in analyses may mitigate this problem.

Keywords: Adaptive procedures; Auditory; Children; Developmental disorders; Dyslexia; Psychophysics; Sensory systems; Staircase; Vision.

Figure 1. Hypothetical psychophysical data.

Data from a hypothetical psychometric function (A) and an adaptive procedure-track (B). In (A), the data showing percentage of correct responses for six stimulus values have been fit with a Weibull function; dashed lines show the intersection of threshold and the 75%-correct point on this function. In (B), the procedure terminates after 20 reversals, indicated by circles.

Figure 2. Effects of reversal-count, slope, step-size, and adjustment rule on a typical staircase procedure.

The effects of reversal count (2A), slope (2B) and step-size (2C) on the mean and variability of thresholds measured with a 2-down, 1-up procedure. In all plots, the model subject had a known and fixed threshold of 10, indicated by the dashed line; the dotted line indicates the mean of the estimated thresholds. In (A), data are shown for 10, 20 and 30 reversals when the model subject had a fixed slope (β) of 1, for a 2-down, 1-up (1 dB) adaptive procedure. In (B), data are for 20 reversals with the same 2-down, 1-up procedure but the value of β is either 0.5, 1, or 3. In (C), all parameters are the same as in (A) but the step-size of the adaptive procedure is 2 dB instead of 1 dB. (D) illustrates the different relationship with reversal-count when the adjustment rule is changed, in this case to a 3-down, 1-up (1 dB) procedure. (E) shows mean thresholds, estimated by the 2-down, 1-up (1 dB) adaptive procedure, for a set of model subjects with a range of thresholds between 1 and 20 (β = 1). Their real thresholds are plotted against mean estimated thresholds based on 10, 20 and 100 reversals. The error bars indicate ±1 standard deviation in the estimated threshold. Points are artificially offset from each other to facilitate interpretation of the error bars.

Figure 3. Group comparisons.

Effect-sizes for group comparisons for estimated thresholds in a group of model observers, plotted as a function of the effect size for the same comparisons using their real thresholds, for a 2-down, 1-up procedure 3A and a 3-down, 1-up procedure 3B. Error bars show standard deviation.

Figure 4. The effects of lapse rate on estimated threshold.

(A) shows histograms of estimated thresholds, taken from 20 reversals, for a single model observer with a real threshold of 10 (β = 1), with different lapse rates. The data for the lapse rate of 0% in (A) are the same data as in the 20 reversals section of Fig. 2A. (B) shows the effect of lapse rate on mean estimated threshold across the same groups of model observers as in the reversal-count analysis from Fig. 3. (C) illustrates the group-sizes that would generate an artificial group difference for groups with lapse rates of 5% or 10%, even when veridical thresholds in both groups were identical, using the data in (A).