Computing by modulating spontaneous cortical activity patterns as a mechanism of active visual processing

Abstract

Cortical populations produce complex spatiotemporal activity spontaneously without sensory inputs. However, the fundamental computational roles of such spontaneous activity remain unclear. Here, we propose a new neural computation mechanism for understanding how spontaneous activity is actively involved in cortical processing: Computing by Modulating Spontaneous Activity (CMSA). Using biophysically plausible circuit models, we demonstrate that spontaneous activity patterns with dynamical properties, as found in empirical observations, are modulated or redistributed by external stimuli to give rise to neural responses. We find that this CMSA mechanism of generating neural responses provides profound computational advantages, such as actively speeding up cortical processing. We further reveal that the CMSA mechanism provides a unifying explanation for many experimental findings at both the single-neuron and circuit levels, and that CMSA in response to natural stimuli such as face images is the underlying neurophysiological mechanism of perceptual "bubbles" as found in psychophysical studies.

Fig. 1

The spatially extended, conductance-based spiking neural network generates co-activated patterns with critical dynamics. a The snapshot of spontaneous activity shows co-activated patterns in our circuit model ($250\times 250$ neurons). The magenta dots represent spikes at this moment, illustrating that the fraction of individual neurons participating in a wave is low ($2\pm 0.4$%, mean$\pm$s.d., s.d. represents standard deviation). As these localized activity patterns move in a complex way, all areas of the circuit would be visited by them. b An illustrative example of three cascades (dots represent spikes and the color encodes time). For each time step, connected components of spikes are clustered within a radius $r_{\mathrm{S}}$. A cascade is defined as a set of clustered spike objects whose center of mass changes by less than $r_{\mathrm{T}}$ between successive time ($T_2-T_1=1\mathrm{ms}$). The interaction between cascade 1 and 2 destroys cascade 1. c Distribution of cascade sizes follows a power-law function, $P\left(S\right)~S^{-\tau }$. Inset: the complementary cumulative distribution function $P\left(S\ge s\right)$ of the same data also follows a power-law function. d Same as in c but for cascade durations, $P\left(D\right)~D^{-\alpha }$. e Top: raster plot shows the spike times of a sub-population of randomly selected 70 excitatory neurons in 3 s. Bottom: single-neuron spike count of a randomly chosen neuron (250 ms bin, sliding over in 10-ms step). f Fano factor of spontaneous activity as a function of time window $\mathrm{\Delta }t$ during the spontaneous activity and after stimulation. g Spike-triggered averaged (STA) membrane potential ($V_{\mathrm{m}}$) during 2 s shows multiple correlated patterns. The black circle labels the seed neuron. Source data are provided as a Source Data file

Fig. 2

Spontaneous coherent activity with criticality reveals long-range correlation patterns with irregular structures. a Spontaneous activity correlation patterns based on two close seed points can be totally different (seed point 1 and 2) or similar (seed point 2 and 3). b Moving the seed-point reveals a punctuated rapid transition in global correlation structure expressed by a high rate of change in the correlation pattern between adjacent points (seed point 1 and 2), termed as fracture. Different seeds also can give rise to the similar correlation patterns (seed point 2 and 3). The symbols and colors are the same as in a. c The fracture generated by systematically moving the seed point when the neural circuit is in the critical regime. d The order parameter of collective motion (black) and the pattern shape-based order parameter (red) as the function of $\mathrm{\Delta }{W}_{\mathrm{E}}$ ($W_{\mathrm{E}}=W_{\mathrm{E}}^{\mathrm{original}}+\mathrm{\Delta }{W}_{\mathrm{E}}$). The cyan-white colormap encodes the heterogeneity of dynamical activity patterns. I, C, and II represent randomly wandering patchy pattern, critical, and regular wave states, respectively. The error bars represent s.d. e The fracture is consistent with the boundary of high and low firing patches. The semi-transparent fracture map (gray) covers on the firing rate map in 2 s ($\ln \left(\mathrm{firing}\mathrm{rate}\right)$ where $\ln$ is the natural logarithm.) encoded by the blue-yellow colormap. f Shifting the network to the state with regular patchy patterns gives rise to regular correlation patterns. The white circle with black edge labels the seed point. g Regular fractures of the state with regular patchy patterns. h, i Same as in f, g but for the state with regular crescent waves. Source data are provided as a Source Data file

Fig. 3

After the onset of stimuli, stimuli modulate spontaneous patterns and the modulation is maximized in the critical point. a–c Snapshots of membrane potentials after stimulus onset show that most spontaneous patterns do not disappear but their shape and positions are modulated. The dashed lines and circles in c are trajectories and start points of the patterns, respectively, since the stimulus onset. d Schematic diagram of the modulating process. The cross symbol represents RoIs. Circles with yellow dots signify the patchy patterns. Crescent with red dashed lines represents crescent-shaped waves. $T_1$ is the moment just before the onset of the stimulus, and $T_{2,3}$ are the following time moments. The left panel shows the scenarios of patchy patterns. If the patchy pattern is close to an RoI, the patchy pattern is gradually dragged to the RoI, otherwise not. The right panel shows the scenario of a crescent-shaped wave. If the wave moves toward an RoI, it can be modulated. e The modulation index based on the order parameter ($\eta$, Eq. 17) varies as the excitatory coupling strength changes, i.e., $W_{\mathrm{E}}=W_{\mathrm{E}}^{\mathrm{original}}+\mathrm{\Delta }{W}_{\mathrm{E}}$. It has a maximum value at the critical point. f The modulation index of firing rates ($\zeta$, Eq. 16) versus the excitatory coupling changes. $\zeta$ also has a maximum value at the critical point. The error bars in e and f represent s.e.m. (standard error of mean). Source data are provided as a Source Data file

Fig. 4

Stimuli slightly modulate the temporal autocorrelation and spatial correlation. a The averaged temporal autocorrelation of instantaneous firing rates (bin width: 20 ms, randomly selected neurons) of spontaneous and evoked activities. The temporal correlation of evoked activity decays more slowly than that of spontaneous activity. The black dashed lines in a, b are correlations for shuffled responses. The lighter shaded lines represent ten s.e.m. Lines are an average over an ensemble of ten trials with random initial conditions at each point. b Average correlation coefficients of the instantaneous firing rate (bin width: 50 ms), computed over all neuron pairs in RoIs, as a function of their distance. The lighter shaded lines represent s.e.m. Lines are an average over an ensemble of ten trials with random initial conditions at each point. c The 2D Pearsons correlation between the smoothed $V_{\mathrm{m}}$ frame just before the stimulus onset and other frames every 0.1 ms from −300 to 300 ms (0 ms is the moment just before stimulus onset). The lighter shaded lines represent 3 s.e.m. Lines are an average over an ensemble of 180 trials with random initial conditions at each point. Spon.: Spontaneous activity, Evoked: Evoked activity, Shuffled: shuffled activity, i.e., randomly shuffling every neuron’s activity; Autocorr., Autocorrelation. Source data are provided as a Source Data file

Fig. 5

Spontaneous activity patterns speed up the stimulus-related response. a In a typical example, decoding analyses show that the response of the CMSA case (M.) is faster to attain significant decoding accuracy than that of control case (C., the preceding spontaneous activities are randomly shuffled). Inset: the statistic of two conditions’ decoding latency. Red dots are data points. 0 s is just before the stimulus onset. The error bars represent 3 s.e.m. b The co-activation property of the activity patterns benefit the modulation processes. The decoding latency of modulation processes decreases with the increase of the number of patterns in the network. The error bars represent s.e.m.; data is averaged over an ensemble of ten trials with random initial conditions at each point. Source data are provided as a Source Data file

Fig. 6

CMSA explains the relations between spontaneous and evoked activities and response variability. a Scatter plot of the mean $V_{\mathrm{m}}$ of the evoked response over a period of $T$ (2–120 ms, mean value: 8 ms) after the stimulus onset versus its preceding spontaneous mean $V_{\mathrm{m}}$ over $T$ immediately before the stimulus onset. $T$ is the duration of the modulation process, that is, the time interval from the stimulus onset to the time when the spontaneous pattern is trapped. b Scatter plot of the number of spikes occurring in the first $2T$ of the response and the mean spontaneous $V_{\mathrm{m}}$ in the $T$ preceding the stimulus onset. c Scatter plot of latency to the first spike and the spontaneous $V_{\mathrm{m}}$. d Spontaneous $V_{\mathrm{m}}$ and latency of neurons depend on the distance between the ongoing pattern and the test neuron. These curves are smoothed by Gaussian-weighted moving average over each window. e Latency and number of spikes of the test neuron versus spontaneous $V_{\mathrm{m}}$. The curves are also smoothed as in d. f The distribution of response times in simulations. The tail of the distribution of response time can be fitted as a power-law function (dashed line, exponent: −2.5). g The pixel intensity of face stimuli (filtered by DoG) decays as the distance from the RoIs increases. The shaded area represent 3 s.d.; the line averages over an ensemble of 16 face images. h The distribution of response times in the stochastic model. Source data are provided as a Source Data file

Fig. 7

Similarity of the bubbles selected by the ideal observer and the modulated activity patterns. a The best bubble mask selected by the ideal observer. b The diagnostic face generated by the bubble mask in a exposes the eyes, mouth, and outline. c The 2D converted map of modulated patterns in a single trial is similar to the best bubble mask. d The averaged 2D converted map of modulated patterns over 100 trials is similar to the best bubble mask. e The histogram of trial-by-trial correlation between the bubbles selected by the ideal observer and 2D converted maps among 100 trials (mean value: 0.67). Source data are provided as a Source Data file