Skip to main page content
Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2019 Aug 1;14(8):e0220161.
doi: 10.1371/journal.pone.0220161. eCollection 2019.

Probabilistic Associative Learning Suffices for Learning the Temporal Structure of Multiple Sequences

Affiliations
Free PMC article

Probabilistic Associative Learning Suffices for Learning the Temporal Structure of Multiple Sequences

Ramon H Martinez et al. PLoS One. .
Free PMC article

Abstract

From memorizing a musical tune to navigating a well known route, many of our underlying behaviors have a strong temporal component. While the mechanisms behind the sequential nature of the underlying brain activity are likely multifarious and multi-scale, in this work we attempt to characterize to what degree some of this properties can be explained as a consequence of simple associative learning. To this end, we employ a parsimonious firing-rate attractor network equipped with the Hebbian-like Bayesian Confidence Propagating Neural Network (BCPNN) learning rule relying on synaptic traces with asymmetric temporal characteristics. The proposed network model is able to encode and reproduce temporal aspects of the input, and offers internal control of the recall dynamics by gain modulation. We provide an analytical characterisation of the relationship between the structure of the weight matrix, the dynamical network parameters and the temporal aspects of sequence recall. We also present a computational study of the performance of the system under the effects of noise for an extensive region of the parameter space. Finally, we show how the inclusion of modularity in our network structure facilitates the learning and recall of multiple overlapping sequences even in a noisy regime.

Conflict of interest statement

The authors have declared that no competing interests exist.

Figures

Fig 1
Fig 1. Network architecture and connectivity underlying sequential pattern activation.
Network architecture and connectivity underlying sequential pattern activation. (A) network topology. Units uij are organized into hypercolumns h1, …, hH. At each point in time only one unit per hypercolumn is active due to a WTA mechanism. Each memory pattern is formed by a set of H recurrently connected units distributed across hypercolumns. For simplicity and without compromising the generality we adopt the following notation for patterns P1=(u11,,u1H). We depict stereotypical network connectivity by showing all the units that emanate from unit u11. The unit has excitatory projections to the proximate units in the sequence (connections from u11 to u21 and u31 and the corresponding units in other hypercolumns) and inhibitory projections to both the units that are farther ahead in the sequence (u11 to u41) and the units that are not in the sequence at all (gray units). (B) abstract representation of the relevant connectivity for sequence dynamics. Please note that only connections from P2 are shown.
Fig 2
Fig 2. An instance of sequence recall in the model.
(A) Sequential activity of units initiated by the cue. (B) The time course of the adaptation current for each unit. (C) The total current s, note that this quantity crossing the value of wnexto (depicted here with a dotted line) marks the transition point from one pattern to the next. (D) The connectivity matrix where we have included pointers to the most important quantities wself for the self-excitatory weight, wnext for the inhibitory connection to the next element, wrest for the largest connection in the column after wnext and wprev for the connection to the last pattern that was active in the sequence.
Fig 3
Fig 3. Systematic study of persistence time Tper.
(A) Tper dependence of B. The blue solid line represents the theoretical prediction described in Eq 4 and the orange bullets are the result of simulations. Inset depicts what happens close to B = 0 where we can see that the lower limit is the time constant of the units τs. (B) An example of sequence recall where Tper = 100 ms. This example corresponds to configuration marked the black star in (A). (C) example of sequence recall with Tper = 500 ms. This example corresponds to the configuration marked with a black triangle in (A). (D) Recall of a sequence with variable temporal structure (varying Tper. The values of Tper are 500, 200, 1200, 100, and 400 ms respectively.
Fig 4
Fig 4. Sequence learning paradigm.
(A) Relationship between the connectivity matrix w and the z-traces. The weight wij from unit i to unit j is determined by the probability of co-activation of those units which in turn is proportional to the overlap between the z-traces (show in dark red). The symmetric connection wij is calculated through the same process but with the traces flipped (here shown in dark blue). Note that the asymmetry of the weights is a direct consequence of the asymmetry of the z-traces. (B) Schematic of the training protocol. In the top we show how the activation of the patterns (in gray) induces the z-traces. In the bottom we show the structure of the training protocol where the pulse time Tp and the inter-pulse interval (IPI) are shown for further reference. (C) We trained a network with only five units in a single hypercolumn for illustration. The first three epochs (50 in total) of the training protocol are shown for reference. The values of the parameters during training were set to Tp = 100 ms, IPI = 0 ms, τzpre=50ms and τzpost=5ms. (D) The matrix at the end of the training (after 50 epochs). (E) Evolution of the probability values during the first three epochs of training. The probability values of the pre (pi), post (pj) and joint probability (pij) evolve with every presentation. Note that the same color code is used in images C, E and F. (F) Long-term evolution of the probabilities with respect to the number of epochs. The values of the probability traces eventually reach a steady state. (G) Short-term evolution of the weight matrix at the points marked in the first epoch in C. Note that the colors are subjected to the same colorbar reference as in D.
Fig 5
Fig 5. Characterization of the effect of training in the connectivity weights and persistent times.
The equation on the inset in D relates Tper to Δwnext = wselfwnext which we show as dashed red lines in each of the top figures (note that here Δβ = 0 as we trained with an homogeneous protocol). When the parameters themselves are not subjected to variation their values are: Tp = 100 ms, IPI = 0 ms, τzpre=25ms, τzpost=20ms for all the units. (A-C) Show how the weights depend on the training parameters Tp, inter pulse interval and τzpre, respectively, whereas (D-E) illustrate the same effects on Tper. Here we are providing the steady state values of w obtained after 100 epochs of training.
Fig 6
Fig 6. Transition from the sequence regime to a random reactivation regime.
(A) An example of a sequential (ordered) activation of patterns. (B) Unordered reactivation of the learned attractors. (C) The two regimes (sequential in blue and random reactivation of attractors in red) in the relevant parameter space spammed by τzpre and inter pulse interval. The examples in (A) and (B) correspond to the black dot and the star, respectively.
Fig 7
Fig 7. Effects of noise reflected in current trajectories and persistence times.
(A) An example of current trajectories subjected to noise. The solid lines indicate the deterministic trajectories the system would follow in the zero noise case. In dotted, jagged and dashed lines we depict the currents induce wself, wnext and wrest for reference. (B) Change in the average of the actual value of Tper for different levels on noise. We Shaded the area between the 25th and the 75th percentile to convey and idea of the distribution for every value of σ (C) Success rate vs noise profile dependence on Tper. We ran 1000 simulations of recall and present the ratio of successful recalls as a function of σ. Confidence intervals from the binomial distribution are too small to be seen.
Fig 8
Fig 8. Sensitivity of network performance to noise for different parameters.
The base reference values of the parameters of interest are: Tp = 100 ms, IPI = 0 ms, τzpre=25ms, τzpost=15ms, sequence length = 5, #hypercolumns = 1. (A) Two examples of the success vs noise profiles (Tp = 50 ms, 200 ms). The value of σ50 is indicated in the abscissa for clarity, note that smaller σ50 implies a network that is more sensitive to noise (the success rate decays faster). (B) σ50 variation with respect to TP. We also indicate the σ50 for the values of Tp used in (A) with stars of corresponding colors.(C) σ50 variation with respect to the inter pulse intervals. (D) σ50 variation with respect to the value of τzpre. (E) σ50 variation with respect to sequence length. (F) σ50 variation with respect to the number of hypercolumns.
Fig 9
Fig 9. Overlapping representations and sequences.
(A1) Schematic of the parameterization framework. Black and gray stand for the representational overlap and the sequential overlap respectively (see text for details) (A2) Schematic of the sequence disambiguation problem. (B) An example of two sequences with overlap. Here each row is a hypercolumn and each column a pattern (patterns P1x, P2x, P3x, P4x, P5x, and P6x). The single entries represent the particular unit that was activated for that hypercolumn and pattern. (C) The superposition of the recall phase for the sequences in (B). Each sequence recall is highlighted by its corresponding color. We can appreciate inside the gray area that the second and third hypercolumns (sequential overlap of 2) have the same units activated (depicted in black). This reflects the fact those patterns have a representational overlap of 23 (two out of three hypercolumns).
Fig 10
Fig 10. Sequence recall performance for different overlap conditions.
The base line values of the parameters of interest are Tp = 100 ms, ΔTp = 0 ms, τzpre=25ms, τzpost=5ms, sequence length = 10, H = 10 and Tper = 50 ms. (A) Success rate for pairs of two sequences with different sequential and representation overlaps. We show here the performance over the parameter space. Success here is determined by correct recall of both sequences. Note that the white corner in the top-right is undefined as it corresponds to a degree of sequential overlap that would include either the first or the last pattern in the sequence (B) Success rate vs noise level for the sequences with configurations marked as 1, 2, 3, 4 in A. The values of σ50 are marked for illustration purposes. (C) σ50 as a function of the sequential overlap. The values of σ50 are calculated over the sequences with configurations given in the green horizontal line in A. (D) σ50 as a function of the representation overlap. The values of σ50 are calculated over the sequences with configurations given in the blue vertical line in A. (E) max disambiguation as a function of Tper. The network loses disambiguation power with long lasting attractors as the memory of the earlier pattern activation reflected in the currents fades. (F) Success rate vs noise profile in the disambiguation regime. The three curves correspond to overlapping sequence configurations marked with x, y, and z in A. Shaded areas correspond to 95% confidence intervals (1000 trials).
Fig 11
Fig 11. The BCPNN weights temporal co-activations against overall activations.
The significance of temporal associations. (A) Here we compare naive simple Hebbian learning with the BCPNN in terms of relative weighting of different temporal associations. In the presented example there are three associations EF, EG, and HG that have been observed 99, 1, 1 occasions respectively. Simple Hebbian learning weights just the frequency of the associations and, as a consequence, EG and HG end up with the same association weight. The BCPNN, on the other hand, differentiates the weights as it takes into account the total activation probability of each unit, rendering the temporal association HG more significant than EG.

Similar articles

See all similar articles

References

    1. Luczak A, Barthó P, Marguet SL, Buzsáki G, Harris KD. Sequential structure of neocortical spontaneous activity in vivo. Proceedings of the National Academy of Sciences. 2007;104(1):347–352. 10.1073/pnas.0605643104 - DOI - PMC - PubMed
    1. Jin DZ, Fujii N, Graybiel AM. Neural representation of time in cortico-basal ganglia circuits. Proceedings of the National Academy of Sciences. 2009; p. pnas–0909881106. 10.1073/pnas.0909881106 - DOI - PMC - PubMed
    1. Harvey CD, Coen P, Tank DW. Choice-specific sequences in parietal cortex during a virtual-navigation decision task. Nature. 2012;484(7392):62 10.1038/nature10918 - DOI - PMC - PubMed
    1. Tang A, Jackson D, Hobbs J, Chen W, Smith JL, Patel H, et al. A maximum entropy model applied to spatial and temporal correlations from cortical networks in vitro. Journal of Neuroscience. 2008;28(2):505–518. 10.1523/JNEUROSCI.3359-07.2008 - DOI - PMC - PubMed
    1. Barnes TD, Kubota Y, Hu D, Jin DZ, Graybiel AM. Activity of striatal neurons reflects dynamic encoding and recoding of procedural memories. Nature. 2005;437(7062):1158 10.1038/nature04053 - DOI - PubMed

Publication types

Grant support

This work was supported by grants from the Swedish Science Council (Vetenskapsrådet, VR2018-05360), Swedish e-Science Research Center (SeRC) and the EuroSPIN Erasmus Mundus doctoral programme. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Feedback