Asynchronous Rate Chaos in Spiking Neuronal Circuits

Abstract

The brain exhibits temporally complex patterns of activity with features similar to those of chaotic systems. Theoretical studies over the last twenty years have described various computational advantages for such regimes in neuronal systems. Nevertheless, it still remains unclear whether chaos requires specific cellular properties or network architectures, or whether it is a generic property of neuronal circuits. We investigate the dynamics of networks of excitatory-inhibitory (EI) spiking neurons with random sparse connectivity operating in the regime of balance of excitation and inhibition. Combining Dynamical Mean-Field Theory with numerical simulations, we show that chaotic, asynchronous firing rate fluctuations emerge generically for sufficiently strong synapses. Two different mechanisms can lead to these chaotic fluctuations. One mechanism relies on slow I-I inhibition which gives rise to slow subthreshold voltage and rate fluctuations. The decorrelation time of these fluctuations is proportional to the time constant of the inhibition. The second mechanism relies on the recurrent E-I-E feedback loop. It requires slow excitation but the inhibition can be fast. In the corresponding dynamical regime all neurons exhibit rate fluctuations on the time scale of the excitation. Another feature of this regime is that the population-averaged firing rate is substantially smaller in the excitatory population than in the inhibitory population. This is not necessarily the case in the I-I mechanism. Finally, we discuss the neurophysiological and computational significance of our results.

Fig 1. Dynamics in the inhibitory population rate model with g(x) = ϕ(x).

Activity of 3 neurons in simulations (N = 32,000, K = 800, τ _syn = 10 ms). A: J ₀ = 4. B: J ₀ = 6. C: J ₀ = 15.

Fig 2. Dynamics in the inhibitory population rate model with g(x) = ϕ(x).

A: Phase diagram. Solid line: DMFT; Dots indicate where the largest Lyapunov exponent, Λ, changes sign in simulations (N = 32,000, K = 800, τ _syn = 10 ms). Inset: Λ vs. J ₀. I ₀ = 2 (black), 4 (red), 6 (blue). Parameters used in Fig 1A, 1B abd 1C are marked by ×, + and ▫, respectively. B: σ(τ) for I ₀ = 1, J ₀ = 15. Black: DMFT. Red and blue dots: Simulations for N = 32,000, K = 800, and N = 256,000, K = 2000, respectively (results averaged over 8 network realizations).

Fig 3. Spectrum of the matrix M/N for inhibitory population rate model with g(x) = ϕ(x).

The matrix was diagonalized numerically for N = 10000, K = 400, I ₀ = 1 and different values of J ₀. A: The bulk of the spectrum for J ₀ = 6 (blue) and for J ₀ = 1.12 (red). Left: The imaginary parts of the eigenvalues are plotted vs. their real parts for one realization of M. This indicates that the support of the spectrum is a disk of radius λ _max. Right: Histograms of N _eig/R (one realization of M) where N _eig is the number of eigenvalues with a modulus between R and R+ΔR (ΔR = 0.0428 (top), 0.0093 (bottom)) for J ₀ = 6 (top) and J ₀ = 1.12 (bottom). The distribution of eigenvalues is uniform throughout the spectrum support. B: The largest real part of the eigenvalues (black dots), λ _max, is compared with the conjecture, Eq (7) (solid line). The fixed point loses stability when λ _max crosses 1.

Fig 4. DMFT for the inhibitory rate model with g(x) = ϕ(x), I ₀ = 1.

A: The PAC amplitude, σ ₀ − σ _∞, is plotted against J ₀. At fixed point σ ₀ − σ _∞ = 0 (blue). When J ₀ = J _c ≈ 4.995 (black dot, BP) the chaotic state appears. For J ₀ > J _c, the fixed point is unstable (dashed blue) and the network settles in the chaotic state (σ ₀ − σ _∞ > 0, black). Red: Perturbative solution in the limit J ₀ → J _c (see S2 Text). Inset: σ ₀ − σ _∞ vanishes linearly when δ = J ₀ − J _c → 0⁺. Black: Numerical solution of the DMFT equations. Red: Perturbative solution at the leading order, O(δ). B: (σ − σ _∞)/δ is plotted for different values of δ > 0 showing the convergence to the asymptotic form (Eq (11) in S2 Text) in the limit δ → 0. C: Blue dots: Decorrelation time, τ _dec vs. PAC amplitude. The PAC, σ(τ) − σ _∞, was obtained by solving numerically the DMFT equations and τ _dec was estimated by fitting the result to the function A/cosh ²(τ/τ _dec). Red: In the whole range, J ₀ ∈ [5, 7] considered, τ _dec can be well approximated by ${\tau }_{dec}=4.97/\sqrt{{\sigma }_0-{\sigma }_{\infty }}$. This relation becomes exact in the limit σ ₀ − σ _∞ → 0. Inset: Numerical solution of the DMFT equations for J ₀ = 6.65 (blue dots) and the fit to A/cosh ²(τ/τ _dec) (red). The fit is very good although this is far from bifurcation.

Fig 5. Phase diagrams of inhibitory rate models with g(x) = x ^γ H(x), K = 400.

A: γ = 3 (gold), 1 (black), 0.7 (red), 0.51 (purple). B: J _c vs. γ for I ₀ = 1. Black: DMFT. Blue and red: Simulations with N = 32000, K = 400. Blue: Zero-crossing of Λ. Red: The fraction of networks with stable fixed point is 50%, 5% and 95% on the solid, bottom-dashed and top-dashed lines respectively.

Fig 6. Spectrum of the matrix M/N for inhibitory rate models with g(x) = x ^γ H(x).

A-B: γ = 1. The matrix was diagonalized numerically for N = 10000, K = 400, I ₀ = 1 and different values of J ₀. A: The bulk of the spectrum (one realization). Left panel: Imaginary vs. real parts of the eigenvalues for one realization of M. Blue: J ₀ = 2.045. Red: J ₀ = 0.307. Right panel: Histograms (100 realizations) of N _eig/R where N _eig is the number of eigenvalues with modulus between R and R+ΔR (ΔR = 0.0479 (top), 0.0122 (bottom)) for J ₀ = 2.045 (top) and J ₀ = 0.307 (bottom). The eigenvalues are almost uniformly distributed throughout the disk of radius λ _max (except very close to the boundary). B: The largest real part of the eigenvalues, λ _max (one realization, black dots) is compared with the conjecture Eq (7) (solid line). C,D: Same as in A, B, for γ = 0.55. Blue: J ₀ = 3.01, ΔR = 0.0491; red: J ₀ = 0.75, ΔR = 0.0246 (red). The agreement with Eq (7) is good for J ₀ not too large but the eigenvalues distribution is non-uniform. Quantitatively similar results are found for N = 20000, K = 400 as well as N = 40000, K = 600 (not shown).

Fig 7. DMFT for the inhibitory rate model with threshold-linear transfer function.

A: The PAC amplitude, σ ₀ − σ _∞, is plotted against J ₀. At fixed point σ ₀ − σ _∞ = 0 (blue). When $J_0=J_c=\sqrt{2}$ (black dot, BP) a bifurcation occurs and the chaotic state appears. For J ₀ > J _c, the fixed point is unstable (dashed blue) and the network settles in the chaotic state (σ ₀ − σ _∞ > 0, black). Red: Perturbative solution in the limit J ₀ → J _c (see S4 Text). Inset: $\sqrt{{\sigma }_0-{\sigma }_{\infty }}$ plotted against δ = J ₀ − J _c showing that σ ₀ − σ _∞ vanishes quadratically when δ → 0⁺. Black: Full numerical solution of the DMFT equations. Red: Perturbative solution at the leading order, O(δ). B: (σ − σ _∞)/δ ² is plotted for different values of δ > 0 to show the convergence to the asymptotic function derived perturbatively in S4 Text. Inset: The function (σ(τ) − σ _∞)/δ ² (black) can be well fitted to A/cosh(x/x _dec) (red dots, A = 12.11, x _dec = 2.84). C: Decorrelation time, τ _dec vs. PAC amplitude (blue). The function σ(τ) − σ _∞ was obtained by integrating numerically Eq (29) and τ _dec was estimated by fitting this function to A/cosh(τ/τ _dec). Red: In the whole range of J ₀ considered (J ₀ ∈ [1.4, 1.9] the relation between τ _dec and σ ₀ − σ _∞ can be well approximated by $y=5.29/\sqrt[4]{x}$. Inset: The PAC computed by solving the DMFT equations for J ₀ = 1.81 (blue dots) and the fit to 0.93/cosh(τ/4.6). D: The PAC for J ₀ = 2 and K = 1200. Blue: Numerical integration of Eq (29). Red: Numerical simulations for N = 256,000.

Fig 8. Patterns of activity in simulations of the LIF inhibitory spiking network.

N = 10000, K = 800, J ₀ = 2, I ₀ = 0.3. Voltage traces of single neurons (top), spike trains of 12 neurons (middle) and population averaged firing rates (in 50 ms time bins, bottom) are plotted. A: τ _syn = 3 ms. Neurons fire irregular spikes asynchronously. B: τ _syn = 100 ms. Neurons fire bursts of action potentials in an irregular and asynchronous manner.

Fig 9. Dependence of the dynamics on synaptic strength in the LIF inhibitory spiking model.

Simulation results for N = 40,000, K = 800, I ₀ = 0.3, τ _syn = 100 ms. From left to right: J ₀ = 2 (blue), 1.5 (red) and 1 (black). A: Examples of single neuron membrane voltages (top) and net inputs, h, (bottom). For the three values of J ₀, the mean firing rate of the example neuron is 11 Hz. As J ₀ decreases, the temporal fluctuations in the net input become smaller whereas the temporal average increases such that the firing rate remains nearly unchanged. B. Top: Population average firing rate increases like 100I ₀/J ₀ as implied by the balance condition. Bottom: PAC (σ − σ _∞, bottom). The dots correspond to the fit of the PAC to (σ ₀−σ _∞)⋅[cosh(τ/τ _dec)]⁻¹ which yields τ _dec/τ _syn = 2.5 (blue), 3.0 (red), 3.8 (black) for the three values of J ₀. Inset in the right panel: σ ₀ − σ _∞ vs. J ₀.

Fig 10. Comparison of the inputs and firing rate statistics in the inhibitory LIF spiking and rate models (simulations and DMFT).

N = 40,000, K = 800. J ₀ = 2, I ₀ = 0.3, τ _syn = 100 msec. A: σ(τ/τ _syn). B: Distributions of neuronal mean firing rates, ⟨r _i⟩, and net inputs, ⟨h _i⟩, (inset) in the spiking network (black) and rate model (red; dots: simulations, solid line: DMFT).

Fig 11. PACs in inhibitory LIF spiking and rate models.

All the results are from numerical simulations with N = 40,000, K = 800. A: J ₀ = 2, I ₀ = 0.3. B: J ₀ = 3, I ₀ = 0.3. C: J ₀ = 4, I ₀ = 0.6. D: J ₀ = 1, I ₀ = 0.3. In all four panels the PACs are plotted for the spiking network with τ _syn = 10 (gray), 20 (red) and 40 (green) ms. The results for the rate model are also plotted (black). The firing rates are ∼ 15 Hz in A and C, ∼ 10 Hz in B and ∼ 30 Hz in D, in good agreement with the prediction from the balance condition ([⟨r⟩] = 100I ₀/J ₀ Hz). As the population firing rate increases, a larger τ _syn is needed for good agreement between the spiking and the rate model.

Fig 12. Phase diagram of the inhibitory LIF rate model.

All the results are from numerical simulations with N = 40,000, K = 800. Black: zero-crossing of the maximum Lyapunov exponent, Λ. The fraction of networks for which the dynamics converge to a fixed point is 50%. 5% and 95% on the solid, top-dashed and bottom-dashed red lines respectively. Insets: I ₀ = 0.3. Voltage traces of a neuron in the inhibitory LIF spiking model for J ₀ = 2 (top inset), 0.3 (bottom inset) and τ _syn = 100 ms.

Fig 13. DMFT vs. numerical simulations in the one-population LIF rate model.

All simulation results depicted here were obtained in networks with N = 40,000, K = 800, I ₀ = 0.3. A: The PAC amplitude, σ ₀ − σ _∞, vs. inhibitory coupling, J ₀. Black: DMFT. Blue dots: Simulations of the rate model. Red ×’s: Simulations of the spiking network with τ _syn = 25 ms. Green ×: Spiking network with τ _syn = 7.5 ms. Right inset: The difference between PAC amplitudes obtained in simulations (Δσ _sim) and DMFT (Δσ _th) plotted against K (in log scale) for J ₀ = 3 (blue) and J ₀ = 4(red). Left inset: Closeup (J ₀ ∈ [0.2 0.5]) of the numerical solution of the DMFT equations. B: PACs were fitted as explained in the text to estimate τ _dec. The estimated decorrelation time, τ _dec, is plotted vs. the amplitude of the PAC for the rate (blue), spiking (black) networks and DMFT (red). Inset: The PAC in the rate model for J ₀ = 2 (black dots: simulation; red line: fit).

Fig 14. The phase diagram of the two-population rate model with threshold-linear transfer function.

$J_0^{EE}=0$, $J_0^{EI}=0.8$. The bifurcation lines predicted by the DMFT are plotted in the $J_0^{IE}-J_0^{II}$ parameter space for K = 400 (red), 10³ (blue), 10⁴ (black), and K → ∞ (green). Red dots: Zero-crossing of the largest Lyapunov exponent (average over 5 network realizations) in numerical simulations for K = 400. Color code: Ratio of the population average firing rate of the two populations (I/E) in log scale (right). White region: The activity of the E population is very small for finite K and goes to zero in the limit K → ∞. The boundary to the right of that region is given by: $J_0^{II}=J_0^{EI}\frac{I_I}{I_E}=0.8$.

Fig 15. The two mechanisms for asynchronous chaos in the two-population rate model with threshold-linear transfer function.

Simulations were performed for N _E = N _I = 8000, K = 400, I _E = I _I = 1, $J_0^{EE}=0$, $J_0^{EI}=0.8$. A: II mechanism for $J_0^{II}=6$, $J_0^{IE}=10$. Left panels: Examples of traces of excitatory (h _IE) and inhibitory inputs (h _EI, h _II) into one neuron. Right: PAC of the net inputs to the E neurons. Gray: τ _IE = τ _EI = τ _II = 10 ms; Black: τ _IE = 100 ms, τ _EI = τ _II = 10 ms; Blue: τ _II = 100 ms, τ _IE = τ _EI = 10 ms; Purple: τ _II = 1 ms, τ _EI = τ _IE = 10 ms. Inset: All PACs plotted vs. τ/τ _II. B: EIE mechanism for $J_0^{II}=1$, $J_0^{IE}=15$. Other parameters are as in A. Inset: All PACs plotted vs. τ/τ _IE.

Fig 16. The two mechanisms for asynchronous chaos in two-population LIF spiking and rate networks.

Simulations were performed with N _E = N _I = 16000, K = 400, I _E = 0.2, I _I = 0.1, $J_0^{EE}=0$, $J_0^{EI}=0.8$, $J_0^{IE}=3$. A: II mechanism. PACs of the net inputs in E neurons are plotted for $J_0^{II}=4$, τ _IE = 100 ms, τ _EI = 3 ms and τ _II = 3, (red), 10 (black), 40 (blue) and 100 ms (purple). Solid line: Spiking model. Dots: Rate model. Inset: All PACs (spiking network) are plotted vs. τ/τ _II. B: Voltage of one E neuron for parameters as in A, purple. C: EIE mechanism. PACs of the net inputs in E neurons are plotted for $J_0^{II}=1$, τ _EI = τ _II = 3 ms and τ _IE = 100, (green), 200 (red) and 400 ms (black). Solid line: Spiking model. Dots: Rate model. Inset: All PACs (spiking network) are plotted vs. τ/τ _IE. D: Voltage of one E neuron in the spiking network with parameters as in C, green.

Fig 17. Dynamical mean-field theory for the one-population inhibitory rate model with g(x) = ϕ(x).

The potential, V(σ, σ ₀) is plotted for different values of σ ₀ as a function of σ. A_1–3: J ₀ = 4 < J _c (=4.995). B_1–5: J ₀ = 15 > J _c.