Synaptic scaling in combination with many generic plasticity mechanisms stabilizes circuit connectivity

Abstract

Synaptic scaling is a slow process that modifies synapses, keeping the firing rate of neural circuits in specific regimes. Together with other processes, such as conventional synaptic plasticity in the form of long term depression and potentiation, synaptic scaling changes the synaptic patterns in a network, ensuring diverse, functionally relevant, stable, and input-dependent connectivity. How synaptic patterns are generated and stabilized, however, is largely unknown. Here we formally describe and analyze synaptic scaling based on results from experimental studies and demonstrate that the combination of different conventional plasticity mechanisms and synaptic scaling provides a powerful general framework for regulating network connectivity. In addition, we design several simple models that reproduce experimentally observed synaptic distributions as well as the observed synaptic modifications during sustained activity changes. These models predict that the combination of plasticity with scaling generates globally stable, input-controlled synaptic patterns, also in recurrent networks. Thus, in combination with other forms of plasticity, synaptic scaling can robustly yield neuronal circuits with high synaptic diversity, which potentially enables robust dynamic storage of complex activation patterns. This mechanism is even more pronounced when considering networks with a realistic degree of inhibition. Synaptic scaling combined with plasticity could thus be the basis for learning structured behavior even in initially random networks.

Keywords: homeostasis; neural network; plasticity; synapse.

Figure 1

Combined conventional plasticity and weight-dependent scaling yields stable synapses. Phase space diagrams for synaptic scaling wⁿ with different weight-dependencies (n = 0: weight-independent scaling, n = 1: linear weight dependence, n = 2: non-linear weight dependence) and plasticity rule [Hebbian (A–D); BCM (E,F)]. Small insets (top-left of each panel) show the connectivity. White arrows indicate convergent weights, magenta arrows divergent weights. (A–C), (E,F) Weight changes in dependence of the input activity for a single input synapse. Colors indicate weight change dw/dt (blue: decrease, red: increase). White and magenta curves indicate stable and unstable fixed points of the weights, resp. (D) Simultaneous weight changes for two synapses showing one cross section through the (w₁,w₂,u₁,u₂)-phase space of a two-synapse system, fixing u₁ = u₂ = 1.0. Stable fixed points indicated by white disks, unstable fixed point at zero weights indicated by magenta disk. (A) one input, n = 0; (B,E) one input n = 1; (C,F) one input n = 0; (D) two inputs, n = 2, colors here indicate squared rate of change (dw/dt)² = (dw₁/dt)² + (dw₂/dt)². Parameters: (A–F) Relative time scales between conventional plasticity and scaling μ/γ = 10, target activity v_T = 0.3, (D) input activity u₁ = u₂ = 1.0, (E,F) Θ = 0.5.

Figure 2

Stabilization of synaptic weights follows the input pattern for different neuron models and are not affected by the neuron model. (A) System stabilization with one neuron that receives ten noisy inputs. Although the signal to noise distance of the input activities (shown for the two strongest inputs in the bottom inset) is very small and, therefore, the signals overlap to a large degree, the system maps them on distinguishable weights. (B) Neuron with 20 inputs, 10 following a plain Hebb rule, the other 10 the BCM rule using also different learning rates μ as indicated. (C–F) Weights stabilize also when using non-linear (spiking) activation functions F. Two neurons provide input to one target neuron. Inputs were Poisson spike trains with 2 (green) and 3 (blue) spikes per 100 simulations steps. The used neuron models are Integrate and Fire (firing threshold = 0.5) with (C) plain Hebb and (D) BCM and Izhikevich [“RS” neuron; Izhikevich (2003)] with (E) plain Hebb and (F) BCM. Parameters: Time axes in simulation steps. (A) relative time scales of plasticity and scaling μ/γ = 10, v_T = 0.5, (B) μ₁ = 0.1, μ₂ = 0.01, γ = 0.001 resulting in a ratio of μ/γ of 100 or 10, resp., v_T = 0.5, for BCM: Θ = 0.3. Each group of five inputs receives: u_{{1, 2, 3, 4, 5}} = {0.015,0.018, 0.020, 0.022, 0.025}, (C–F) μ/γ = 10, v_T = 0.1, (E,F) Θ = 0.5. For a detailed description of the used neuron models see Appendix.

Figure 3

Weight change depends on initial weight and on post-synaptic depolarization level. (A–D) Weight change dw/dt plotted against initial weight w₀ for different plasticity mechanisms and different degrees of scaling. Output activity was increased as indicted by the arrows on the right. In (A,B) the horizontal line at dw/dt = 0.01 reflects no scaling as v = v_T.(E,F) Replotting the data from (C,B) now showing weight change dw/dt plotted against output activity v. Initial weight was increased as indicted by the arrow on the right. Parameters: v_T = 0.5, μ = 0.1, u = 0.2, (A,B,F) μ/γ = 2, (C-E) μ/γ = 10.

Figure 4

Stabilization of bi-directional, recurrent connections. (A,B) Input synapses (dashed lines) are fixed. (A) displays synaptic dynamics for dissimilar inputs, (B) for similar inputs. (C,D) Input synapses are also allowed to change. (C) displays synaptic dynamics for dissimilar inputs, (D) for similar inputs. (E) as (D) but with one inhibitory neuron. Parameters: (A–D) relation between plasticity and scaling μ/γ = 10, v_T = 0.007. (A) u_{{1, 2}} = {0.01, 0.001} (B) u_{{1, 2}} = {0.002, 0.001} (C) u_{{1, 2}} = {0.01, 0.001} (D,E) u_{{1, 2}} = {0.002, 0.001}. (E) v_T = 0.05, excitatory-to-inhibitory connections are set to w_3,1 = w_3,2 = 0.15, and inhibitory-to-excitatory connections are w_1,3 = w_2,3 = 1.0, all unchanging.

Figure 5

Model predictions are consistent with experimental findings. (A) Decrease or increase of network activity proportionally changes synaptic weights. In experiment, Tetrodotoxin (TTX) inhibits network activity, whereas Bicuculline (BIC) disinhibits it. A recurrent network with random connectivity and plastic synapses behaves comparable to the experimental data. Here, each neuron receives an external input which is decreased (red) or increased (blue) compared to a control case (black line). [inset, modified from Turrigiano et al. (1998)]. (B) The same network can be used to predict synaptic weight distribution (main panel) qualitative consistent with weight distribution experimental found in a cortical network [inset, modified from Song et al. (2005)] Parameters: (A) v_T = 5 × 10⁻⁴, the input is Gaussian distributed around 3 × 10⁻⁴, relative time scales of plasticity and scaling μ/γ = 5, Red line is 1% of the control activity (black), and the blue line 200%. (B) Network size N = 100, relation between plasticity and scaling μ/γ = 5, v_T = 5 × 10⁻⁴, 10% random connectivity, u uniformly drawn from (0, 0.001).

Figure 6

Neuron-specific inputs robustly yield stable synaptic enhancement along several connection stages. (A) Schematic of post-synaptic connectivity of selected neurons up to stage four. Three neurons labeled a, b, c (pink) receive external inputs, neurons i, j, k serve as controls. Connectivity is only shown for neuron a. Red arrows depict the first connection stage, green second, blue third, and black the fourth. In this scheme, each neuron has three post-synaptic connections. As these are randomly drawn, neurons at all stages may connect to each other. (B) Neural activities (left column) and synaptic weights (right column) found for the first stages of post-synaptic connectivity. Three inputs, to a, b, c, are compared to three controls i, j, and k. Same color code as in (A). Values are rank ordered as this provides a clearer picture than a histogram, which is distorted by the binning. Weights descending from input neurons are significantly different from those descending from the control neurons up to the third-connection stage (Kolmogorov-Smirnov test, p < 0.01, asterisks), activities are different up to the second stage. Total number of connections when starting with three neurons are 9, 27, 81, and 243 along the first four stages (right). Parameters: Network size N = 100, connectivity 3%, initial activation of controls on average u = 0.005 and of inputs u = 0.05, v_T = 0.1, time scale of conventional plasticity relative to scaling μ/γ = 5, 20% global inhibition is performed by decreasing activity of each neuron according to 20% of the mean activity of the network.