Depression-biased reverse plasticity rule is required for stable learning at top-down connections

Abstract

Top-down synapses are ubiquitous throughout neocortex and play a central role in cognition, yet little is known about their development and specificity. During sensory experience, lower neocortical areas are activated before higher ones, causing top-down synapses to experience a preponderance of post-synaptic activity preceding pre-synaptic activity. This timing pattern is the opposite of that experienced by bottom-up synapses, which suggests that different versions of spike-timing dependent synaptic plasticity (STDP) rules may be required at top-down synapses. We consider a two-layer neural network model and investigate which STDP rules can lead to a distribution of top-down synaptic weights that is stable, diverse and avoids strong loops. We introduce a temporally reversed rule (rSTDP) where top-down synapses are potentiated if post-synaptic activity precedes pre-synaptic activity. Combining analytical work and integrate-and-fire simulations, we show that only depression-biased rSTDP (and not classical STDP) produces stable and diverse top-down weights. The conclusions did not change upon addition of homeostatic mechanisms, multiplicative STDP rules or weak external input to the top neurons. Our prediction for rSTDP at top-down synapses, which are distally located, is supported by recent neurophysiological evidence showing the existence of temporally reversed STDP in synapses that are distal to the post-synaptic cell body.

Figure 1. Schematic description of the model and learning rules.

a. Schematic description of the model used in the analytical and computational work. The model consists of two layers: a “lower” cortical area (units with activity L_i(t)) and a “higher” cortical area (units with activity H_j(t)). b. The strength of the all-to-all bottom-up connections from the lower area to the higher area is represented by the matrix Q (gray arrows). These synapses occur in proximal dendrites and their weights are fixed unless otherwise noted. The strength of the all-to-all top-down connections from the higher area to the lower area is represented by the matrix W (black arrows). These synapses occur in distal dendrites. The W weights evolve according to the plasticity rules described in c–d. There are no connections within each layer. c. Schematic description of “classical” spike-time dependent plasticity (cSTDP). For a given synapse, the y-axis indicates the change in the weight (Δw) and the x-axis represents the temporal difference between the post-synaptic action potential and the pre-synaptic action potential (). The green curve shows the learning rule used in the analytical section while the blue curve shows the learning rule used in the integrate-and-fire simulations. In cSTDP, a pre-synaptic action potential followed by a post-synaptic action potential (Δt>0) leads to potentiation (Δw>0). The learning rate at each synapse is controlled by the parameter μ and the ratio of depression to potentiation is controlled by α. In the computational simulations, the parameter τ_STDP controls the rate of weight change with Δt. d. Schematic description of “reverse” STDP (rSTDP).

Figure 2. Example numerical implementation of the analytical results for depression-biased rSTDP learning.

a. Development of top-down synaptic weights (W) over multiple stimulus presentations. N indicates the stimulus presentation number and here we show 4 snapshots of W (N). This model had 20 lower units and 20 higher units. The strength of each synaptic weight is represented by the color in the W matrix (see scale on the right). The algorithm converged after 2005 iterations and the final W is shown on the right (see Methods for convergence criteria). b–d. Measures of weight stability and diversity. b. Norm of the change in the top-down weight matrix () as a function of stimulus presentation number N (see text). As the algorithm converges, the change in the weights becomes smaller. The dotted lines mark the iterations corresponding to the snapshots shown in part a. c. Standard deviation of the distribution of top-down weights as a function of iteration presentation number (loosely represented in the y-axis as std(W)). The final value in this plot (N = 2005) corresponds to the standard deviation of the distribution shown in part e. d. Pearson correlation coefficient between the vectorized W (N) and W(N-100) (blue line, calculated only for N> = 100) and between W(N) and the predicted value of W at the fixed point (W* = Q⁻¹; green line, see text for details). As the algorithm converges, W (N)→. e. Measure of weight diversity: Distribution of the final synaptic weights after the algorithm converged. Bin size = 4. f. Measure of absence of strong loops: Mean (blue) and maximum (green) eigenvalue of the matrix WQ, as a function of stimulus presentation number. This matrix describes the activity changes produced in a full up-down loop through the network. Eigenvalues greater than one would correspond to the existence of strong loops. The maximum eigenvalue never surpasses 0.33, which is equal to 1/. The mean eigenvalue also eventually stabilizes at this value.

Figure 3. Example of the dynamics and evolution of top-down weights in the integrate-and-fire model.

a. Snapshots showing the evolution of W (N) in the integrate-and-fire network simulations over time defined by the number of stimulus presentations (N). The format is the same as in Figure 2a . This model had 100 lower units and 100 higher units. The parameters used in this simulation are shown in the last column of Table 1 , with rSTDP and  = 1.2. b–c. Measures of weight stability. b. Standard deviation of the distribution of top-down weights as a function of the stimulus presentation number. The convergence criterion for the standard deviation was that the slope of this plot (calculated as with ΔN = 6000) be less than 10⁻⁵. The convergence criterion was met at the point indicated by the red asterisk. The dotted vertical lines correspond to the times of the five snapshots shown in part a. c. Blue line: Pearson correlation coefficient between the vectorized W (N) and W (N-ΔN), for ΔN = 3000 iterations. For comparison with Figure 2 , we also show the correlation coefficient between W(N) and the inverse of Q (green line). We note that in the integrate and fire simulations we do not expect W(N) to converge to the described in the text and Figure 2 . A simulation run was classified as ‘convergent’ when the correlation coefficient was greater than 0.99 and when the std criterion in part b was met. In this example, the simulation achieved the correlation criterion at T = 75000 (red asterisk). d. Measure of weight diversity: Distribution of the synaptic weights for the final snapshot. Bin size = 0.1. e. Measure of absence of strong loops: Average firing rate for lower-level neurons as a function of stimulus presentation number. The average firing rate almost immediately stabilizes to a constant value, and does not increase to pathological levels as occurs in the presence of strong excitatory loops.

Figure 4. Representative results of integrate-and-fire simulations for different learning rules.

We consider here four possible learning rules: classical STDP (cSTDP, c,d), reverse STDP (rSTDP, a,b), depression-biased (a,c) and potentiation-based learning (b,d). For each learning rule, we show the results for a representative simulation (see summary results in Figure 5 .) The format and conventions for the subplots are the same as in Figure 3 . The subplots show the Pearson correlation coefficient between the vector containing all the entries of W (N) and that for W (N-ΔN), for ΔN = 3,000 iterations (first subplot), the standard deviation of the distribution of weights (second subplot), the distribution of weights (third subplot), the average firing rate of the lower level units (fourth subplot) and the final W. The simulation in part a converged; the convergence criteria were met at the value of N indicated by an asterisk. The simulations in b–d were classified as having “extreme weights” meaning that >50% of the weights were either at 0 or at the weight boundaries (±50). The arrows in the second subplot in b–d denote inflection points where the weights reached the boundaries and the standard deviation started to decrease. The parameters for each of these simulations are listed in the last column of Table 1 , with specifics as follows. a rSTDP,  = 1.2; b: rSTDP,  = 0.9; c: cSTDP,  = 1.2; d: cSTDP,  = 0.9. For the simulations in b–d, the weights varied most strongly across lower-level neurons, leading to the appearance of vertical bands in the final subplots (note the differences in the color scale and standard deviation values in 4b–d compared to 4a). Some lower-level neurons experienced greater joint activity than others due to the choice of Q (and hence greater plasticity); the instability of learning in these simulations then magnified these initial imbalances.

Figure 5. Summary of the results of the integrate-and-fire network simulations.

We consider the four possible learning rules illustrated in Figure 4 . Here we show the proportion of all the computational simulations in a parameter search ( Methods , Table 1 ) using integrate-and-fire units that converged (green), that reached extreme weights (red) or that did not converge (light blue). For comparison with Figure 6 , we included a category for simulations in which weights failed to achieve sufficient diversity (dark blue), although none of the current simulations fell into that category. The quantitative criteria for classifying the stimulations into these four categories as well as the network and parameters spanned are described in the text. The total number of simulations for each learning rule were 2298, 2304, 1148, and 1152. The only convergent simulations were seen for depression-biased rSTDP.

Figure 6. Summary of the results of the integrate-and-fire network simulations with additional stability mechanisms.

We show the results of simulations with homeostatic scaling, multiplicative plasticity, or concurrent bottom-up and top-down plasticity ( Methods , Table 1 ). The format is the same as in Figure 5 . The only convergent simulations were seen for depression-biased rSTDP, in the homeostatic scaling and concurrent plasticity cases. For all other learning conditions, homeostatic scaling simulations and concurrent plasticity reached extreme weights. Multiplicative plasticity always led to a lack of diversity.

Figure 7. Summary of the results of the integrate-and-fire simulations with external input to bottom-layer and higher-layer neurons.

a–c. In the simulations described here, external input was conveyed both to lower-level neurons and to higher-level neurons. The ratio of the external input strength to higher-level neurons to lower-level neurons was 0.1 in a, 1 in b and 10 in part c. The format and other parameters are the same as in Figure 5 . d. For those simulations that converged (green in parts a–c), the histogram shows the distribution of average activity levels. The gray bars denote simulations using rSTDP and the dark bars denote simulations using cSTDP. Results from all three strength ratios (a–c) are combined in this plot. Those few simulations which are convergent under cSTDP have very low average firing rates.

Figure 8. Integrate-and-fire network trained with rSTDP learns to reconstruct its input.

a. Example of the network's ability to reconstruct its inputs after training using depression-biased rSTDP. By construction, the strength of external input during a single stimulus presentation to each neuron in the lower layer (input strength, blue line) is similar to the average spike rate of each lower-level neuron during the initial period from 0–50 ms (initial activity, green line). The cyan and red lines show the average spike rate of each lower-level neuron during the later period (late activity, 80–160 ms), when activity is due to top-down stimulation, using the top-down weights given early in training (after 10 iterations, cyan line) or after 51,000 iterations (red line). b. Average correlation coefficient between early time and late-time neuronal activity rates as a function of the number of training iterations. The average is computed over n = 100 distinct external input stimuli, and the error bars represent the standard deviation of the correlation coefficients for the 100 stimuli. The arrows indicate the iteration numbers illustrated in part a.