Robustness of learning that is based on covariance-driven synaptic plasticity

Abstract

It is widely believed that learning is due, at least in part, to long-lasting modifications of the strengths of synapses in the brain. Theoretical studies have shown that a family of synaptic plasticity rules, in which synaptic changes are driven by covariance, is particularly useful for many forms of learning, including associative memory, gradient estimation, and operant conditioning. Covariance-based plasticity is inherently sensitive. Even a slight mistuning of the parameters of a covariance-based plasticity rule is likely to result in substantial changes in synaptic efficacies. Therefore, the biological relevance of covariance-based plasticity models is questionable. Here, we study the effects of mistuning parameters of the plasticity rule in a decision making model in which synaptic plasticity is driven by the covariance of reward and neural activity. An exact covariance plasticity rule yields Herrnstein's matching law. We show that although the effect of slight mistuning of the plasticity rule on the synaptic efficacies is large, the behavioral effect is small. Thus, matching behavior is robust to mistuning of the parameters of the covariance-based plasticity rule. Furthermore, the mistuned covariance rule results in undermatching, which is consistent with experimentally observed behavior. These results substantiate the hypothesis that approximate covariance-based synaptic plasticity underlies operant conditioning. However, we show that the mistuning of the mean subtraction makes behavior sensitive to the mistuning of the properties of the decision making network. Thus, there is a tradeoff between the robustness of matching behavior to changes in the plasticity rule and its robustness to changes in the properties of the decision making network.

Figure 1. The model.

(A) The decision making network consists of two populations of sensory neurons N_i, corresponding to the two targets, and two populations of premotor neurons M_i, corresponding to the two actions. Choice is determined by comparing the activities of the two populations of premotor neurons (see text). (B) The effect of the synaptic plasticity rule on synaptic efficacy. The decision making model was simulated in a concurrent VI reward schedule (see Materials and Methods) with equal baiting probabilities, and the efficacy of one of the synapses is plotted as a function of trial number. During the first 300 trials (blue), the synaptic efficacies evolved according to Eq. (2) with α = 0 and β = 1 (and thus γ = 0), resulting in small fluctuations of the efficacy around the initial conditions. A 10% mistuning of the mean subtraction after 300 trials (red arrow) to β = 0.9 (γ = 0.1) resulted in a linear divergence of the efficacy (red line). The addition of a linear decay term to the plasticity rule (Eq. (4) with ρ = 1) after 600 trials (black arrow) resulted in small fluctuations of the efficacy around 0.04 (black line).

Figure 2. Incomplete mean subtraction and deviations from matching behavior.

(A) The probability of choice as a function of fractional income. Each point corresponds to one simulation of the model, Eq. (4), in a concurrent VI reward schedule with fixed baiting probabilities. The level of deviation from matching behavior (black line) depends on the level of incomplete mean subtraction, γ and synaptic saturation stiffness, ρ. Red squares, γ = 0.05, ρ = 1; blue diamonds, γ = 0.5, ρ = 1; gray triangles γ = 0.5, ρ = 4; colored lines are the analytical approximations, Eq. (7). (B) Susceptibility of behavior as a function of γ. In order to quantify the effect of γ on deviation from matching behavior, we repeated the simulations of A for many values of γ and measured the susceptibility of behavior (the slope of the resultant curve, see text and Materials and Methods). Blue dots, ρ = 5; red dots, ρ = 1; black dots, ρ = 0.2. Lines correspond to the expected slope from the analytical approximation, Eq. (7).

Figure 3. Bias in the winner-take-all mechanism and deviations from matching behavior.

(A) The probability of choice as a function of fractional income. Each point corresponds to one simulation of the model (Eq. (4) with ρ = 1) in a concurrent VI reward schedule with fixed baiting probabilities. The level of deviation from matching behavior (black line) depends on the bias in the winner-take-all mechanism. Red squares, ε = −3σ; blue diamonds, ε = 0; gray triangle, ε = 3σ; γ = 0.05; colored lines are the analytical approximation, Eq. (8). (B) Choice bias. The simulation of A was repeated for different values of ε for two values of γ (blue dots, γ = 0.5; red dots, γ = 0.05), and the probability of choosing alternative 1 for a fractional income of r ₁ = 0.5 was measured. Lines correspond to the expected probability of choice from the analytical approximation, Eq. (8).