Linking pattern completion in the hippocampus to predictive coding in visual cortex

Abstract

Models of predictive coding frame perception as a generative process in which expectations constrain sensory representations. These models account for expectations about how a stimulus will move or change from moment to moment, but do not address expectations about what other, distinct stimuli are likely to appear based on prior experience. We show that such memory-based expectations in human visual cortex are related to the hippocampal mechanism of pattern completion.

Figure 1

Analysis approach. (a) There were three types of trials during fMRI: full-sequence trials (purple lettering), in which cue A was replaced by outcome B if a button was pressed with the left hand and by outcome C if a button was pressed with the right hand; cue+action trials (gray), in which A was replaced by a blank screen upon either button press; and outcome-only trials (green), in which B or C appeared in isolation without a button press. (b) Pattern completion was operationalized as the amount of neural evidence elicited by a cue and action about the corresponding full sequence. This evidence was measured with a multivariate classifier trained on full-sequence trials to distinguish the two sequences for each cue and tested on cue+action trials (sequence decoding). (c) Predictive coding was operationalized as the amount of neural evidence elicited by a cue and action about the expected outcome. This evidence was measured with a multivariate classifier trained on outcome-only trials to distinguish the two outcomes for each cue and tested on cue+action trials (outcome decoding). For both sequence decoding and outcome decoding, separate classifiers were trained and tested on patterns of BOLD activity in ROIs from the hippocampus and visual cortex.

Figure 2

Decoding performance. (a) A priori ROIs included CA2–CA3–DG, CA1, and subiculum in the hippocampus, and V1 and V2 in early visual cortex. (b) Sequence decoding was reliable in CA2–CA3–DG (t₂₃=2.53, P = .02) and CA1 (t₂₃=2.72, P = .01), but not in subiculum, V1, or V2 (Ps>.81). (c) Outcome decoding was reliable in V1 (t₂₃=3.17, P = .004) and V2 (t₂₃=2.51, P = .02), but not in CA2–CA3–DG, CA1, or subiculum (Ps>.57). Error bars depict ±1 s.e.m. *P<.05, **P<.01

Figure 3

Hippocampal-visual relationship. (a) Outcome decoding in V1–V2 was more reliable (t₂₃=2.32, P = .03) for trials on which sequence decoding in CA–DG was correct (vs. 50% chance: t₂₃=3.45, P = .002) vs. incorrect (t₂₃=1.05, P = .30). Error bars depict ±1 s.e.m. (b) Individual differences in V1–V2 outcome decoding could be predicted from CA–DG sequence decoding (robust r₂₂=.60, P = .002). *P<.05, **P<.01