Evidence for model-based encoding of Pavlovian contingencies in the human brain

Abstract

Prominent accounts of Pavlovian conditioning successfully approximate the frequency and intensity of conditioned responses under the assumption that learning is exclusively model-free; that animals do not develop a cognitive map of events. However, these model-free approximations fall short of comprehensively capturing learning and behavior in Pavlovian conditioning. We therefore performed multivoxel pattern analysis of high-resolution functional MRI data in human participants to test for the encoding of stimulus-stimulus associations that could support model-based computations during Pavlovian conditioning. We found that dissociable sub-regions of the striatum encode predictions of stimulus-stimulus associations and predictive value, in a manner that is directly related to learning performance. Activity patterns in the orbitofrontal cortex were also found to be related to stimulus-stimulus as well as value encoding. These results suggest that the brain encodes model-based representations during Pavlovian conditioning, and that these representations are utilized in the service of behavior.

Fig. 1

Sequential Pavlovian conditioning paradigm. a Each trial was initiated by the onset of a central fixation cross. The distal and CSp fractals were presented sequentially in two random locations on the screen. The presentation of the CSp co-terminated with US delivery. b Main trial types and stimulus categories in the 4 learning sessions. Identical CSp fractals (CS-X and CS-Y) were used throughout sessions, with valence reversals between sessions. Two unique CSd fractals were introduced in each session. ITI inter-trial interval

Fig. 2

Behavioral measures of learning. a Consistent with model-free Pavlovian learning, participants’ subjective value ratings of CS fractals changed in concordance with Pavlovian contingencies, relative to their baseline ratings before Pavlovian conditioning. Plotted are the total number of CS fractal rating changes (after—before) contingent with Pavlovian associations. b Consistent with model-based Pavlovian learning mechanisms cognitive maps of Pavlovian contingencies, participants performed above chance on a test for explicit knowledge of Pavlovian associations. See Supplementary Fig. 1 for descriptive plots of ratings and test-score, grouped by stimulus type. Violin plots show mirrored density plot of behavioral results, boxplots show; Tukey-style box and whisker plots show the median, two hinges and two whiskers of data; Dots show individual participant results

Fig. 3

Results of ROI analyses of striatal representations. a Definition of functional striatal zones according to a previous meta-analysis. The nucleus accumbens is highlighted in red, the body of the caudate is highlighted in blue. b Stimulus value classifier accuracy in nucleus accumbens correlates with changes in participants’ ratings of CS fractals (r = 0.58, p = 0.003, Pearson). c Stimulus identity accuracy in the body of the caudate nucleus correlated with participants’ explicit knowledge of CS-US associations (r = 0.64, p = 6e−4, Pearson). Violin plots show mirrored density plot of behavioral results, boxplots show; Tukey-style box and whisker plots show the median, two hinges and two whiskers of data; Dots show individual participant results

Fig. 4

Searchlight results for orbitofrontal cortex. a When trained on CSp fractals and tested on CSd fractals, the stimulus identity (id) classifier accuracy was above chance in right central frontal orbital cortex (xyz = [34.2,37.8, −16.2]). b When trained and tested on CSd fractals, the stimulus value (rew) classifier accuracy was above chance in the lateral orbitofrontal cortex (xyz = [−34.2,30.6, −23.4]). For display purposes, we applied a threshold of p < 0.005. c Classifier accuracies for reward and identity classifiers in individual participants