Tactile-auditory shape learning engages the lateral occipital complex

Abstract

Shape is an object property that inherently exists in vision and touch, and is processed in part by the lateral occipital complex (LOC). Recent studies have shown that shape can be artificially coded by sound using sensory substitution algorithms and learned with behavioral training. This finding offers a unique opportunity to test intermodal generalizability of the LOC beyond the sensory modalities in which shape is naturally perceived. Therefore, we investigated the role of the LOC in processing of shape by examining neural activity associated with learning tactile-shape-coded auditory information. Nine blindfolded sighted people learned the tactile-auditory relationship between raised abstract shapes and their corresponding shape-coded sounds over 5 d of training. Using functional magnetic resonance imaging, subjects were scanned before and after training during a task in which they first listened to a shape-coded sound transformation, then touched an embossed shape, and responded whether or not the tactile stimulus matched the auditory stimulus in terms of shape. We found that behavioral scores improved after training and that the LOC was commonly activated during the auditory and tactile conditions both before and after training. However, no significant training-related change was observed in magnitude or size of LOC activity; rather, the auditory cortex and LOC showed strengthened functional connectivity after training. These findings suggest that the LOC is available to different sensory systems for shape processing and that auditory-tactile sensory substitution training leads to neural changes allowing more direct or efficient access to this site by the auditory system.

Figure 1.

Behavioral paradigm. A, Examples of abstract shapes used in the study. B, Behavioral testing that took place in the laboratory was based on a forced-choice task where subjects heard five soundscapes and touched one shape, and chose the soundscape that matched the tactile shape. C, Training involved a learning phase where subjects explored five shapes and heard their corresponding soundscapes and an evaluation phase where they touched a shape and chose its corresponding soundscape from a set of five. Feedback was provided for each evaluation trial.

Figure 2.

Pretraining and posttraining fMRI sessions involved matching and control tasks that were presented in paired trials. In the first part of each matching trial, blindfolded subjects listened to a soundscape repeated four times, and in the second part of the trial, touched a tactile shape that either matched or did not match the shape corresponding to the soundscape heard in the first trial. Subjects responded “yes” or “no” with a key press during the scan acquisition following the second part of the trial. A control task involved the same paired-trial procedure during which subjects first listened to an auditory stimulus that was matched for acoustic characteristics to the soundscapes, and then touched a scrambled tactile shape; they made a key press at the end but no yes/no response was required. The posttraining session also contained an additional condition where subjects saw a visual shape in the second half of the matching trial and responded whether or not the soundscape matched the visual shape. The corresponding control condition included a visual scrambled shape.

Figure 3.

Behavioral results from the auditory–tactile shape-matching task (percentage correct shown). Performance in both the laboratory and the scanner was above chance before training (dark gray) and improved significantly after training (light gray). Performance on the auditory–visual task (black) was equivalent to that on the auditory–tactile task at posttraining. Dotted lines indicate chance level of performance. *p < 0.05; significance based on paired-samples t tests.

Figure 4.

The shape-listen minus control-listen contrast yielded activity in the LOC both before (left) and after (right) training in addition to expected activity in auditory cortical areas. Red arrows indicate LOC activity on the coronal and transverse slices.

Figure 5.

The shape minus control contrast involving each of the auditory and tactile modalities commonly yielded a subregion of the LOC as an area of activity both before and after training. Top, Left hemisphere sagittal sections; bottom, horizontal sections. The LOC was defined as the occipitotemporal area of visual activity resulting from the seeing-shape minus scrambled-shape (data acquired only posttraining). Regions of activity involving one, two, or all three modalities were coded differentially by different colors as indicated in a Venn diagram. A, Auditory; T, tactile; V, visual.

Figure 6.

A, Results of the functional connectivity analysis shown in a t map. The analysis showed that BOLD activity at an auditory cortex seed region, selected as the peak from the shape-listen minus control-listen contrast (x_pre, y_pre, z_pre = −62, −14, 4; x_post, y_post, z_post = −58, −16, 6), was correlated with that of the LOC (dotted circles) after but not before training (activity shown at z = 0). B, The same analysis also yielded positive correlation with the early visual cortex before training (dotted circles at x = 8, z = 10) but negative correlation in more ventral portions of early visual cortex after training (dotted circles at x = −26, z = −8).