Audio-visual experience strengthens multisensory assemblies in adult mouse visual cortex

Abstract

We experience the world through multiple senses simultaneously. To better understand mechanisms of multisensory processing we ask whether inputs from two senses (auditory and visual) can interact and drive plasticity in neural-circuits of the primary visual cortex (V1). Using genetically-encoded voltage and calcium indicators, we find coincident audio-visual experience modifies both the supra and subthreshold response properties of neurons in L2/3 of mouse V1. Specifically, we find that after audio-visual pairing, a subset of multimodal neurons develops enhanced auditory responses to the paired auditory stimulus. This cross-modal plasticity persists over days and is reflected in the strengthening of small functional networks of L2/3 neurons. We find V1 processes coincident auditory and visual events by strengthening functional associations between feature specific assemblies of multimodal neurons during bouts of sensory driven co-activity, leaving a trace of multisensory experience in the cortical network.

Fig. 1. Tone-specific response enhancement at a subset of multimodal neurons after audio-visual pairing.

a–b Expression of GCaMP6f in L2/3 excitatory neurons enabling 2-P imaging in awake or lightly anaesthetised animals in V1. Scale bar: 20 µm. c Raw (grey) and smoothed (red) calcium responses to tone presentation. Scale bars: 50%ΔF₁/F₀ and 4 s. d Average response of multimodal neuron to tones (top) and gratings (bottom). Scale bars: 50%ΔF₁/F₀ and 4 s. e Percentage of non-responsive, visually-responsive, auditory-responsive and multimodal neurons in anaesthetised conditions. f Timeline depicting baseline, audio-visual pairing or repeated unpaired presentation and testing. Icons depict stimuli with resting state activity depicted by the mouse in the dark. g Percentage of neurons showing greater (>20% of baseline) responses to the paired (red) or unpaired (black) tone or grating after audio-visual pairing. h Neurons with an increased response to the paired tone vs. baseline sensory response profiles. Icons depict response to stimuli. i Average calcium activity (ΔF₁/F₀/s) following Log₁₀ transformation for multimodal neurons to the paired (red) or unpaired (black) tone before (open) or after (filled) paired and unpaired trials. Inset: response to paired tone before (grey) and after (red) pairing. Scale bars: 50%ΔF₁/F₀ and 4 s. j Percentage of multimodal neurons showing an enhanced (>20%) tone response to either the paired (red) or unpaired (black) tone. Neurons grouped by baseline sensory response profiles to separate presentation of paired auditory and visual stimuli. Responsive to, (Left) paired visual and auditory stimuli, (Middle) paired visual stimuli only and (Right) paired auditory stimuli only. Icons depict visual/auditory baseline response. k V1 region and responses during baseline (left) and 24-h after pairing (right) in awake animals. Scale bars: 20 µm, 100%ΔF₁/F₀ and 30 s. l Average calcium (ΔF₁/F₀/s) response of multimodal neurons to pairing in awake animals. Open circles are tone presentation in the baseline (BL), re-testing and testing 24-h after pairing. Calcium response to pairing stimuli is shown as filled circles. m Average calcium activity (ΔF₁/F₀/s) for paired tone before (red, open) and after (red, filled) pairing and for unpaired tone (black bars). Figure 1b–j uses 332 neurons from six regions across six animals in light anaesthetic conditions and Fig. 1k–m uses 408 neurons from four regions across four animals in awake conditions. In all panels, *p < 0.05 (see Table 1 and associated Supplementary Table 1). Error bars: mean and ± S.E.M. Source data are provided as a Source Data file.

Fig. 2. Audio-visual pairing results in a tone-specific reduction in subthreshold hyperpolarisation.

a (Top, left) Schematic of intersectional genetic approach for the expression of Chi-VSFPBfly1.2 at excitatory (CaMK2A) cortical neurons. (Top, right) Cartoon depicting 2-P imaging of CaMK2A-Chi-VSFPBfly1.2 expressing mouse under anaesthetised conditions. (Bottom) Example region taken from L2/3 of V1 in a CaMK2A-Chi-VSFPBfly1.2 expressing adult mouse showing FRET channels for the donor (mCitrine–left) and acceptor (mKate2–right). Scale bar: 10 µm. b Example ratio traces (Ratio = Acceptor: mKate2/Donor: mCitrine) showing change in tone response following pairing. (Top) Response to tone presentation during the baseline mapping phase. (Middle) Response to the same tone following audio-visual pairing. (Bottom) Difference between the tone presentation before and after audio-visual pairing. Black line shows the average ratio response across nine cortical regions and grey lines show the S.E.M. of responses. Scale bars are 1 % and 1 s. c Population level cross-modal plasticity measured with CaMK2A-Chi-VSFPBfly1.2 following repeated audio-visual pairing (red) and repeated unpaired tone presentation (black). The data for Fig. 2a–c uses 356 regions taken from five animals under anaesthetised conditions. For all panels, **p < 0.01, ***p < 0.001 (see Table 2 and associated Supplementary Table 2). Error bars: mean and ± S.E.M. Source data are provided as a Source Data file.

Fig. 3. Multimodal subnetworks exhibit strengthening and weakening of functional associations.

a, d Correlations between multimodal neurons during spontaneous activity before (a) and after (d) pairing. Example shows multimodal neurons that either have an increased tone response after pairing (red, filled) or do not (black, open). Scale bar: 20 s and 100 %ΔF₁/F₀. b Average correlation strength (r) prior to audio-visual pairing between multimodal neurons that either: have an increased response to the paired tone after pairing (left) or do not (right). Average correlation value in each case is with other neurons that either: have increased tone responses (red, filled) or do not (black, open). c Schematic showing multimodal networks to which a neuron with increased (Left: red, filled) or not increased (Right: black, open) responses to the paired tone may belong. Red line represents associations between neurons with an increased tone response. Black line represents associations between neurons that do not have an increased tone response. Grey line indicates associations between neurons from different groups. e, f Change in the percentage of the correlation coefficient (out of summed total of all correlation coefficients) attributable to multimodal neurons with different responses to pairing. For neurons with increased (e) or non-increased (f) responses, the percentage change in total correlation coefficient attributable to associations with other increasing neurons (red) or other neurons that do not increase (black). The grey dashed line in e, f depicts 0% change. Inset: average percentage change for increasing (e) and non-increasing (f) cells. g, h Schematic of a multimodal subnetwork to which a neuron that has an enhanced tone response (red, filled) may belong to, before (g) and after (h) audio-visual pairing. Neurons which do not show an enhanced response are shown as black and open. Red lines represent strong functional associations, which increase in strength (as indicated by the thickness of the line and addition sign) whilst black line represents weaker functional associations, which weaken (as indicated by the dashed black line and subtraction sign). Figure 3a–f uses 103 neurons taken from five cortical regions across five animals in anaesthetised conditions. For all panels, *p < 0.05, **p < 0.01, ***p < 0.001 (see Table 3 and associated Supplementary Table 3). Error bars: mean and ± S.E.M. Source data are provided as a Source Data file.

Fig. 4. Periods of co-activity predict multimodal network plasticity.

a–c Schematic of simulation with key (b) and plasticity protocol (c). Width of grey lines indicate strength of connections following development (a). d Average change in all synaptic weights at multimodal neurons (preferring paired tone and grating–P_TP_G) icons as in b. e–h Main plasticity changes in the simulation (e, g). Thickness of arrow denotes average change in strengthened (green) and unchanged (black) synaptic weights. Change in synaptic weights from (e, f) or onto (g, h) multimodal neurons (preferring both the paired tone and grating - P_TP_G) with other cells. i) Change in responses at multimodal neurons after pairing to presentation of stimuli. j Change in response to paired tone for multimodal neurons tuned to either the paired or unpaired visual grating. k Co-activity between multimodal neurons during pairing and later synaptic strengthening. l Average calcium activity (ΔF₁/F₀/s following Log₁₀ transformation) for multimodal neurons that either exhibit an increased response (left, red) or do not (right, black) following audio-visual pairing. Bars show the average activity in response to either the paired visual stimuli alone (open) or the coincident presentation of the paired visual stimuli with the paired tone (filled). m, p Change in average activity when sound is presented with visual stimuli (m) or change in correlation coefficient between cell pairs after pairing (p) for cell pairs with either low (<25 %), medium (25–50 %) or high (>50 %) co-activity during audio-visual pairing trials. n, o Correlation plots for all multimodal neurons. The x-axis gives the change in activity between the response to auditory and visual stimuli when presented separately (summed) or simultaneously. The y-axis gives the change in calcium activity (ΔF₁/F₀/s following Log₁₀ transformation baseline vs re-testing phase) in response to n paired or o unpaired tone presentation. Figure 4a–k uses 50 networks of 200 neurons with results averaged across all network simulations. Experimental data in Fig. 4l–p uses 332 neurons taken from six cortical regions across six animals in anaesthetised conditions. For all panels, *p < 0.05, **p < 0.01, ***p < 0.001 (see Table 4 and associated Supplementary Table 4). Error bars, mean and ± S.E.M. Source data are provided as a Source Data file.