A comparative analysis of response times shows that multisensory benefits and interactions are not equivalent

Abstract

Multisensory signals allow faster responses than the unisensory components. While this redundant signals effect (RSE) has been studied widely with diverse signals, no modelling approach explored the RSE systematically across studies. For a comparative analysis, here, we propose three steps: The first quantifies the RSE compared to a simple, parameter-free race model. The second quantifies processing interactions beyond the race mechanism: history effects and so-called violations of Miller's bound. The third models the RSE on the level of response time distributions using a context-variant race model with two free parameters that account for the interactions. Mimicking the diversity of studies, we tested different audio-visual signals that target the interactions using a 2 × 2 design. We show that the simple race model provides overall a strong prediction of the RSE. Regarding interactions, we found that history effects do not depend on low-level feature repetition. Furthermore, violations of Miller's bound seem linked to transient signal onsets. Critically, the latter dissociates from the RSE, demonstrating that multisensory interactions and multisensory benefits are not equivalent. Overall, we argue that our approach, as a blueprint, provides both a general framework and the precision needed to understand the RSE when studied across diverse signals and participant groups.

Figure 1

Experimental design. (a) Redundant signals paradigm. Participants respond to trials with auditory (A), visual (V), and combined signals (AV), but not on catch trials. Signals were presented in a random order within a block of trials. (b) Following a 2 × 2 design, we tested four signal sets in separate blocks. Stimulus construction was either simple (e.g. a pure tone, in audition) or complex (e.g. a noise tone). The sequence of signal features was either consistent (e.g. one frequency) or alternating (e.g. randomly one of two frequencies).

Figure 2

Analysing multisensory benefits. (a) Empirical benefits are measured by the area between the cumulative RT distributions in the multisensory (AV) condition and the faster of the unisensory conditions (A, V; Equation (1)). Data from an example participant in the complex-alternating condition. (b) Empirical benefits as a function of benefits predicted by Raab’s model (Equation (2)). Each point represents a participant in one of the four conditions. Large symbols represent the group mean.

Figure 3

Measuring empirical interactions. (a) Violations of Miller’s bound are measured by the area between the cumulative RT distributions in the multisensory (AV) and the sum of the unisensory conditions (Equations (4) and (5)). Example data as in Fig. 2a. (b) Violations of Miller’s bound across conditions. Mean and SEM of 20 participants.

Figure 4

Model fitting. (a) Best-fitting context variant race model (solid line). The model is constrained by the fits of the LATER model in the unisensory conditions (dashed lines) and has the correlation Rho and the additional noise Eta as free parameters. Example data as in Fig. 2a. (b) Empirical benefits as a function of benefits calculated from the best-fitting context variant race model. Each point represents a participant in one of the four conditions. Large symbols represent the group mean. (c,d) Best fitting values of Rho and Eta across conditions. Mean and SEM of 20 participants.