2012 Nov 21
Canonical Microcircuits for Predictive Coding
Item in Clipboard
Canonical Microcircuits for Predictive Coding
This Perspective considers the influential notion of a canonical (cortical) microcircuit in light of recent theories about neuronal processing. Specifically, we conciliate quantitative studies of microcircuitry and the functional logic of neuronal computations. We revisit the established idea that message passing among hierarchical cortical areas implements a form of Bayesian inference-paying careful attention to the implications for intrinsic connections among neuronal populations. By deriving canonical forms for these computations, one can associate specific neuronal populations with specific computational roles. This analysis discloses a remarkable correspondence between the microcircuitry of the cortical column and the connectivity implied by predictive coding. Furthermore, it provides some intuitive insights into the functional asymmetries between feedforward and feedback connections and the characteristic frequencies over which they operate.
Copyright © 2012 Elsevier Inc. All rights reserved.
This is a schematic of the classical microcircuit adapted from Douglas and Martin (1991). This minimal circuitry comprises superficial (layers 2 and 3) and deep (layers, 5 and 6) pyramidal cells and a population of smooth inhibitory cells. Feedforward inputs – from the thalamus – target all cell populations, but with an emphasis on inhibitory interneurons and superficial and granular layers. Note the symmetrical deployment of inhibitory and excitatory intrinsic connections that maintain a balance of excitation and inhibition.
This is a simplified schematic of the key intrinsic connections among excitatory (E) and inhibitory (I) populations in granular (L4), supragranular (L1/2/3) and infragranular (L5/6) layers. The excitatory interlaminar connections are based largely on Gilbert and Wiesel (1983). Forward connections denote feedforward extrinsic corticocortical or thalamocortical afferents that are reciprocated by backward or feedback connections. Anatomical and functional data suggest that afferent input enters primarily into L4 and is conveyed to superficial layers L2/3 that are rich in pyramidal cells, which project forward to the next cortical area, forming a disynaptic route between thalamus and secondary cortical areas (Callaway, 1998). Information from L2/3 is then sent to L5 and L6, which sends (intrinsic) feedback projections back to L4 (Usrey and Fitzpatrick, 1996). L5 cells originate feedback connections to earlier cortical areas as well as to the pulvinar, superior colliculus, and brain stem. In summary, forward input is segregated by intrinsic connections into a superficial forward stream and a deep backward stream. In this schematic, we have juxtaposed densely interconnected excitatory and inhibitory populations within each layer.
This schematic shows an example of a generative model. Generative models describe how (sensory) data are caused. In this figure, sensory states (blue circles on the periphery) are generated by hidden variables (in the centre). The left panel shows the model as a probabilistic graphical model, where unknown variables (hidden causes and states) are associated with the nodes of a dependency graph and conditional dependencies are indicated by arrows. Hidden states confer memory on the model by virtue of having dynamics, while hidden causes connect nodes. A graphical model describes the conditional dependencies among hidden variables generating data. These dependencies are typically modelled as (differential) equations with nonlinear mappings and random fluctuations
with precision (inverse variance) Π ω ~ ( i ) ( (see the equations in the insert on the left). This allows one to specify the precise form of the probabilistic generative model and leads to a simple and efficient inversion scheme (predictive coding; see next figure). Here i) denotes the set of hidden causes that constitute the parents of sensory v ~ p a ( i ) s̃ ( or hidden i) x̃ ( states. The ~ indicates states in generalised coordinates of motion: i) x̃ = ( x, x′, x″,...). An intuitive version of the model is shown on the right: here, we imagine that a singing bird is the cause of sensations, which – through a cascade of dynamical hidden states – produces modality-specific consequences (e.g., the auditory object of a bird song and the visual object of a song bird). These intermediate causes are themselves (hierarchically) unpacked to generate sensory signals. The generative model therefore maps from causes (e.g., concepts) to consequences (e.g., sensations), while its inversion corresponds to mapping from sensations to concepts or representations. This inversion corresponds to perceptual synthesis – in which the generative model is used to generate predictions. Note that this inversion implicitly resolves the binding problem - by explaining multisensory cues with a single cause.
This figure describes the predictive coding scheme associated with a simple hierarchical model shown on the left. In this model each node has a single parent. The ensuing inversion or generalised predictive coding scheme is shown on the right. The key quantities in this scheme are (conditional) expectations of the hidden states and causes and their associated prediction errors. The basic architecture – implied by the inversion of the graphical (hierarchical) model – suggests that prediction errors (caused by unpredicted fluctuations in hidden variables) are passed up the hierarchy to update conditional expectations. These conditional expectations now provide predictions that are passed down the hierarchy to form prediction errors. We presume that the forward and backward message passing between hierarchical levels is mediated by
extrinsic (feedforward and feedback) connections. Neuronal populations encoding conditional expectations and prediction errors now have to be deployed in a canonical microcircuit to understand the computational logic of intrinsic connections – within each level of the hierarchy – as shown in the next figure.
The left hand panel is the canonical microcircuit based on Haeusler and Maass (2007), where we have removed inhibitory cells from the deep layers – because they have very little interlaminar connectivity. The numbers denote connection strengths (mean amplitude of PSPs measured at soma in mV) and connection probabilities (in parentheses) according to Thomson et al. (2002). The right panel shows the proposed cortical microcircuit for predictive coding, where the quantities of the previous figure have been associated with various cell types. Here, prediction error populations are highlighted in pink. Inhibitory connections are shown in red, while excitatory connections are in black. The dotted lines refer to connections that are not present in the microcircuit on the left (but see Figure 2). In this scheme, expectations (about causes and states) are assigned to (excitatory and inhibitory) interneurons in the supragranular layers, which are passed to infragranular layers. The corresponding prediction errors occupy granular layers, while superficial pyramidal cells encode prediction errors that are sent forward to the next hierarchical level. Conditional expectations and prediction errors on hidden causes are associated with excitatory cell types, while the corresponding quantities for hidden states are assigned to inhibitory cells. Dark circles indicate pyramidal cells. Finally, we have placed the precision of the feedforward prediction errors against the superficial pyramidal cells. This quantity controls the postsynaptic sensitivity or gain to (intrinsic and top-down) pre-synaptic inputs. We have previously discussed this in terms of attentional modulation, which may be intimately linked to the synchronisation of pre-synaptic inputs and ensuing postsynaptic responses (Fries et al 2001; Feldman and Friston, 2010).
This schematic illustrates the functional asymmetry between the spectral activity of superficial and deep cells predicted theoretically. In this illustrative example, we have ignored the effects of influences on the expectations of hidden causes (encoded by deep pyramidal cells), other than the prediction error on causes (encoded by superficial pyramidal cells). The lower panel shows the spectral density of deep pyramidal cell activity, given the spectral density of superficial pyramidal cell activity in the upper panel. The equation expresses the spectral density of the deep cells as a function of the spectral density of the superficial cells; using Equation (2). This schematic is meant to illustrate how the relative amounts of low (beta) and high (gamma) frequency activity in superficial and deep cells can be explained by the evidence accumulation implicit in predictive coding.
Toward an Integrative Theory of Thalamic Function
RV Rikhye et al.
Annu Rev Neurosci 41, 163-183.
The thalamus has long been suspected to have an important role in cognition, yet recent theories have favored a more corticocentric view. According to this view, the thal …
Engagement of Pulvino-cortical Feedforward and Feedback Pathways in Cognitive Computations
J Jaramillo et al.
Neuron 101 (2), 321-336.e9.
Computational modeling of brain mechanisms of cognition has largely focused on the cortex, but recent experiments have shown that higher-order nuclei of the thalamus part …
Emergence of Cognitive Priming and Structure Building From the Hierarchical Interaction of Canonical Microcircuit Models
T Kunze et al.
Biol Cybern 113 (3), 273-291.
The concept of connectionism states that higher cognitive functions emerge from the interaction of many simple elements. Accordingly, research on canonical microcircuits …
Forward and Backward Connections in the Brain: A DCM Study of Functional Asymmetries
CC Chen et al.
Neuroimage 45 (2), 453-62.
In this paper, we provide evidence for functional asymmetries in forward and backward connections that define hierarchical architectures in the brain. We exploit the fact …
State-dependent Computations: Spatiotemporal Processing in Cortical Networks
DV Buonomano et al.
Nat Rev Neurosci 10 (2), 113-25.
A conspicuous ability of the brain is to seamlessly assimilate and process spatial and temporal features of sensory stimuli. This ability is indispensable for the recogni …
PubMed Central articles
Simulating Emotions: An Active Inference Model of Emotional State Inference and Emotion Concept Learning
R Smith et al.
Front Psychol 10, 2844.
The ability to conceptualize and understand one's own affective states and responses - or "Emotional awareness" (EA) - is reduced in multiple psychiatric populations; it …
Passing the Message: Representation Transfer in Modular Balanced Networks
B Zajzon et al.
Front Comput Neurosci 13, 79.
Neurobiological systems rely on hierarchical and modular architectures to carry out intricate computations using minimal resources. A prerequisite for such systems to ope …
Switching Operation Modes in the Neocortex via Cholinergic Neuromodulation : A Computational Model of Uncertainty, Learning, and Inhibition
JY Puigbò et al.
Mol Neurobiol 57 (1), 139-149.
In order to deal with the uncertainty in the world, our brains need to be able to flexibly switch between the exploration of new sensory representations and exploitation …
Two Distinct Neural Timescales for Predictive Speech Processing
PW Donhauser et al.
Neuron 105 (2), 385-393.e9.
During speech listening, the brain could use contextual predictions to optimize sensory sampling and processing. We asked if such predictive processing is organized dynam …
The Role of Top-Down Modulation in Shaping Sensory Processing Across Brain States: Implications for Consciousness
T Sikkens et al.
Front Syst Neurosci 13, 31.
Top-down, feedback projections account for a large portion of all connections between neurons in the thalamocortical system, yet their precise role remains the subject of …
Research Support, N.I.H., Extramural
Research Support, Non-U.S. Gov't
Research Support, U.S. Gov't, Non-P.H.S.
Cerebral Cortex / anatomy & histology
Nerve Net / anatomy & histology
Neural Pathways / physiology