Continuous Attractor Neural Networks: Candidate of a Canonical Model for Neural Information Representation

Abstract

Owing to its many computationally desirable properties, the model of continuous attractor neural networks (CANNs) has been successfully applied to describe the encoding of simple continuous features in neural systems, such as orientation, moving direction, head direction, and spatial location of objects. Recent experimental and computational studies revealed that complex features of external inputs may also be encoded by low-dimensional CANNs embedded in the high-dimensional space of neural population activity. The new experimental data also confirmed the existence of the M-shaped correlation between neuronal responses, which is a correlation structure associated with the unique dynamics of CANNs. This body of evidence, which is reviewed in this report, suggests that CANNs may serve as a canonical model for neural information representation.

Keywords: Continuous Attractor Neural Network; Neural network; anticipative tracking; canonical model; multi-sensory information integration.

Figure 1.. A continuous attractor neural network (CANN) model.

( A) An illustration of a one-dimensional CANN, which encodes a continuous variable (e.g. orientation or direction) x in the region of (- π,π] with the periodic condition. Neurons are aligned in the network according to their preferred stimuli. The neuronal connection pattern J( x,x’) is translation-invariant in the space. The network can hold a continuous family of bump-shaped stationary states. ( B) The stationary states of the CANN form a subspace in which the network states are neutrally stable. The subspace is illustrated as a canyon in the state space of the network. The movement of the network state along the canyon corresponds to the position shift of a bump.

Figure 2.. The projection method.

The dynamics of a continuous attractor neural network are dominated by a few motion modes, corresponding to distortions of the bump shape in height, position, width, skewness, and so on. We can project the network dynamics on these dominating modes to simplify it significantly.

Figure 3.. Anticipative tracking with a continuous attractor neural network.

( A) In the presence of spike-frequency adaptation (SFA), the network bump (red solid curve) leads the external input (blue dotted curve) moving with velocity V _ext. Without SFA, the bump (green dashed curve) lags behind the external input. Inset: the positions of the bumps as a function of time when the external input starts to move at a constant velocity after t = 0. ( B) The anticipative time t _ant is approximately constant in a broad range of V _ext. Symbols represent anticipative time from Goodridge and Touretzky rescaled for comparison. ( C) Static and spontaneously moving phases in the space of inhibition strength $\tilde{k}\equiv k/k_c$ and SFA strength γ. The black curve indicates the phase boundary separating the static and moving phases. In the moving phase, the color code encodes the speed V _int of the spontaneously moving bump. ( D) Regions of delayed and anticipative tracking in the same space when there is a weak and slowly moving external input. The black curve indicates the boundary separating the delayed and anticipative tracking regions. The color code encodes the anticipative time (negative values indicate delayed time). Note the correspondence between the anticipative time and V _int in ( C).

Figure 4.. Optimal multi-sensory integration with coupled continuous attractor neural networks (CANNs).

( A) Multiple reciprocally coupled CANNs form a decentralized information integration system. ( B) An example of two-coupled CANNs for heading-direction inference with combined visual and vestibular cues. The mean ( C) and the variance ( D) of the network estimations agree with the Bayesian predictions. Adapted from .

Figure 5.. The special correlation structure associated with the unique dynamics of a continuous attractor neural network (CANN).

( A) The bump position-shift is the dominating motion mode of a CANN induced by input noises. Consider the true stimulus fixed at zero. Neurons at the same side of the stimulus are positively correlated (e.g. green ones), whereas neurons on different sides of the stimulus are negatively correlated (e.g. green versus blue ones). ( B) When the true stimulus is fixed at a constant value (e.g. zero), the correlations between all neuron pairs in a CANN display an anti-symmetric structure. ( C) When the stimulus value varies, the correlation between a fixed neuron pair with a typical separation of the bump width displays an M-shaped structure.