Dissociating the time courses of the cross-modal semantic priming effects elicited by naturalistic sounds and spoken words

Abstract

The present study compared the time courses of the cross-modal semantic priming effects elicited by naturalistic sounds and spoken words on visual picture processing. Following an auditory prime, a picture (or blank frame) was briefly presented and then immediately masked. The participants had to judge whether or not a picture had been presented. Naturalistic sounds consistently elicited a cross-modal semantic priming effect on visual sensitivity (d') for pictures (higher d' in the congruent than in the incongruent condition) at the 350-ms rather than at the 1,000-ms stimulus onset asynchrony (SOA). Spoken words mainly elicited a cross-modal semantic priming effect at the 1,000-ms rather than at the 350-ms SOA, but this effect was modulated by the order of testing these two SOAs. It would therefore appear that visual picture processing can be rapidly primed by naturalistic sounds via cross-modal associations, and this effect is short lived. In contrast, spoken words prime visual picture processing over a wider range of prime-target intervals, though this effect was conditioned by the prior context.

Keywords: Audiovisual; Multisensory; Priming; Semantic; Sensitivity.

Fig. 1

a Sequence of three frames presented in each trial: A blank, a target picture (e.g., a dog), and a pattern mask. The target picture and pattern mask were presented in black in the center of a white background. b Represents the current experimental design in terms of signal detection theory. The distributions of dashed, dotted, and solid lines represent target present/congruent, target present/incongruent, and target absent conditions, respectively. In this design, the congruent and incongruent conditions share the same FA rate. The sensitivity (d') was calculated using the equations: d' = z(hit rate) – z(FA rate) in the congruent and incongruent conditions, separately (Green & Swets, ; Macmillan & Creelman, 2005)

Fig. 2

Mean sensitivity (d') at the 350- and 1,000-ms SOAs in Experiment 1. Error bars indicate ±1 SEM. Sound = naturalistic sounds; Word = spoken words

Fig. 3

Mean sensitivity (d') at the 350- and 1,000-ms SOAs for Group 1 (tested in the order 350- and then 1,000-ms SOA) and Group 2 (with the order reversed) in Experiment 2. Error bars indicate ±1 SEM. Sound = naturalistic sounds; Word = spoken words