Supervised Multisensory Calibration Signals Are Evident in VIP But Not MSTd

Abstract

Multisensory plasticity enables our senses to dynamically adapt to each other and the external environment, a fundamental operation that our brain performs continuously. We searched for neural correlates of adult multisensory plasticity in the dorsal medial superior temporal area (MSTd) and the ventral intraparietal area (VIP) in 2 male rhesus macaques using a paradigm of supervised calibration. We report little plasticity in neural responses in the relatively low-level multisensory cortical area MSTd. In contrast, neural correlates of plasticity are found in higher-level multisensory VIP, an area with strong decision-related activity. Accordingly, we observed systematic shifts of VIP tuning curves, which were reflected in the choice-related component of the population response. This is the first demonstration of neuronal calibration, together with behavioral calibration, in single sessions. These results lay the foundation for understanding multisensory neural plasticity, applicable broadly to maintaining accuracy for sensorimotor tasks.SIGNIFICANCE STATEMENT Multisensory plasticity is a fundamental and continual function of the brain that enables our senses to adapt dynamically to each other and to the external environment. Yet, very little is known about the neuronal mechanisms of multisensory plasticity. In this study, we searched for neural correlates of adult multisensory plasticity in the dorsal medial superior temporal area (MSTd) and the ventral intraparietal area (VIP) using a paradigm of supervised calibration. We found little plasticity in neural responses in the relatively low-level multisensory cortical area MSTd. By contrast, neural correlates of plasticity were found in VIP, a higher-level multisensory area with strong decision-related activity. This is the first demonstration of neuronal calibration, together with behavioral calibration, in single sessions.

Keywords: multisensory; perceptual decision making; plasticity; self-motion; vestibular; visual.

Figure 1.

Behavioral shifts during multisensory calibration. Behavioral responses from the recording sessions are presented for Monkey Y (A) and Monkey A (B), for the “vestibular accurate” and “visual accurate” conditions (left and right columns, respectively). All data are presented as though Δ = 10°, and data collected with Δ = −10° were flipped before pooling. Blue and red colors represent the vestibular and visual responses, respectively. Dark hues represent precalibration (baseline) behavior. Lighter hues represent postcalibration data. The psychometric plots (Gaussian cumulative distribution functions) represent the proportion of rightward choices as a function of heading, adjusted according to external feedback; that is, the visual “inaccurate” plots (left column) were shifted by 10°, and the vestibular “inaccurate” plots (right column) were shifted by −10°. Here, these represent the average behavior (per condition, cue, block, and monkey). Intersection of each psychometric function with the horizontal dashed line (y = 0.5) marks the PSE (perceptual estimate of straight ahead). Vertical dotted lines indicate the expected PSE for “accurate” perception, according to external feedback. PSE histograms present the data from all sessions. Horizontal bars above the PSE histograms represent the mean ± SEM intervals. Significant shifts (p < 0.05) are marked by asterisk symbols above the histograms, and by horizontal arrows on the psychometrics (with the shift size presented in degrees). X represents significant shifts away from feedback (becoming less accurate).

Figure 2.

Example VIP recording during multisensory calibration. Behavior (A) and neuronal responses (B) for an example neuron from a single session. Blue represents vestibular responses. Red represents visual responses. Darkest hues represent precalibration. Medium hues represent postcalibration. Lightest hues (cyan and magenta) represent the responses after reverse calibration. For behavior (A), circles represent the proportion of rightward choices (fit by cumulative Gaussian psychometric curves). For the neuronal responses (B), circles and error bars represent mean FR (baseline subtracted) ± SEM. Inset, One hundred (randomly selected) overlaid spikes from each block. The tuning curves are presented in relation to the original (veridical) headings (not adjusted according to external feedback as in Fig. 1). C, Reverse versus primary calibration shifts for vestibular (left) and visual (right) cues. Gray represents behavioral shifts. Blue and red represent neuronal shifts (vestibular and visual, respectively). Solid lines indicate Type II regressions (excluding the + outlier on the bottom left of the left plot).

Figure 3.

VIP neuronal shifts correlate with behavioral shifts. Correlations between neuronal shifts (calculated using linear fits in A,B; and neurometric fits in C,D) and behavioral shifts, are presented for VIP (A,C; both vestibular and visual accurate conditions) and MSTd (B,D; vestibular accurate condition; the visual accurate condition was not tested in MSTd). Blue represents vestibular data. Red represents visual data. Darker hues represent the primary calibration sequence (blocks 1-3). Lighter hues represent the reverse-calibration sequence (blocks 3-5). + and open circle represent Monkeys A and Y, respectively. Black dashed lines indicate the diagonal (y = x). Solid lines indicate regression lines of the data.

Figure 4.

Calibration of VIP responses as a function of time. A, Average visual and vestibular neuronal shifts (red and blue lines, respectively) as a function of time for the primary calibration (top row) and reverse calibration (second row). Red and blue bars represent the number of visual and vestibular neurons, respectively, with significant tuning (per time step) used to calculate the shifts. Stimulus onset was at x = 0 s, and stimulus offset at x = 1 s. Data are presented as though Δ = 10° (data collected with Δ = −10° were flipped for pooling). Positive values reflect rightward shifts. Circle markers represent mean ± SEM shifts for the regular time window (0.2-1 s; All). Asterisks indicate significant clusters. B, Correlation of VIP neuronal versus behavioral shifts over time (pooling the primary and reverse-calibration data). Asterisks indicate significant correlations (p < 0.05).

Figure 5.

Multisensory calibration of visual responses in state space. (A) VIP and (C) MSTd population responses for each heading direction are projected onto a two-dimensional state space that captures variance related to heading (stimulus) and choice. Time is embedded within the state space trajectories (these start around the origin; arrows indicate the end of each trajectory). B, D, State space projections for VIP and MSTd, respectively, as a function of time. The state space axes were constructed from the precalibration responses (using only neurons that were stable both precalibration and postcalibration). Responses of these same neurons were projected both precalibration and postcalibration onto the same axes (left and right columns, respectively, in each subplot). Data are presented as Δ = 10° (data collected with Δ = −10° were flipped). Green shaded regions represent 95% distribution intervals (from surrogate data) for the 0° heading condition. The choice axis was orthogonalized relative to the heading axis. Therefore, the percentage of explained variance (presented in parentheses in A and C) represents the unique variance explained by choice, whereas the heading value represents its unique component plus any overlap between the two. Multisensory calibration is seen for VIP primarily in the choice domain as the projections shift toward more rightward choices. This is best seen for the 0° heading condition (green) and the near central headings (−1.5° and 1.5°). N = 182 for VIP, and N = 127 for MSTd.

Figure 6.

Multisensory calibration of vestibular responses from VIP in state space. (A) Population responses projected onto state space. (B) State space projections as a function of time. Conventions are the same as in Figure 5.