Taking stock of value in the orbitofrontal cortex

Abstract

People with damage to the orbitofrontal cortex (OFC) have specific problems making decisions, whereas their other cognitive functions are spared. Neurophysiological studies have shown that OFC neurons fire in proportion to the value of anticipated outcomes. Thus, a central role of the OFC is to guide optimal decision-making by signalling values associated with different choices. Until recently, this view of OFC function dominated the field. New data, however, suggest that the OFC may have a much broader role in cognition by representing cognitive maps that can be used to guide behaviour and that value is just one of many variables that are important for behavioural control. In this Review, we critically evaluate these two alternative accounts of OFC function and examine how they might be reconciled.

Fig. 1 ∣. During decision-making, the OFC flip-flops between representing the value of either option.

a ∣ Monkeys learn the value of different pictures and then perform a choice task. After fixating on a red dot, one (forced choice) or two (free choice) pictures are presented and the animal must make its selection. b ∣ Individual orbitofrontal cortex (OFC) neurons typically encode the value of the chosen picture. About half of the value-encoding neurons have a positive relationship between firing rate and value (top), whereas the other half have a negative relationship (bottom). c ∣ A classifier was trained to recognize patterns of neural activity that are elicited by specific picture values on forced-choice trials. In the example, the neural ensemble consists of n neurons (r₁ … r_n) and the firing rate at time t is plotted. Each data point indicates a pattern of activity that was elicited on different trials, colour-coded according to the value of the outcome of the choice. A hyperplane can successfully separate the two groups of trials. d ∣ The trained classifier is then used to decode activity on the free-choice trial. Each successive dot corresponds to the activity of each neuron at successive time steps, t, within a trial. When ensemble activity lies above the hyperplane, the picture’s value is decoded as value A (blue dots), whereas below, it is decoded as value B (red dots). e ∣ Decisions are characterized by OFC neural ensembles ‘flip-flopping’ between representing the value of either option in turn. Panels a and b are adapted with permission from REF., Springer Nature Ltd.

Fig. 2 ∣. The relationship between the state-transition graph and value.

a ∣ The state-transition graph (left) specifies the relationship between states and the transition probability; for example, state A can directly lead to B or D, but not to the other three states of C, E or F. The state-transition graph can be integrated with the reward location to calculate the value of individual states (right). b ∣ A potential state-transition graph for having a meal in a restaurant. Different states are linked to other states with a specific probability of occurrence. The same graph can be used for many different restaurants, and the value of states can be rapidly updated in response to changes in goals.

Fig. 3 ∣. An illustrative example of the flexibility of cognitive maps.

a ∣ Imagine you need to get across town (start: black pin, goal: red pin). This can be done simply by following a rote set of directions (for example, take the second left followed by the first right). However, this method cannot cope with unexpected events such as construction works or an accident. By contrast, a map-like representation allows on-the-fly adjustments, providing a much more flexible way to navigate. b ∣ Cognitive maps can also be applied to behavioural tasks and provide the same kinds of advantages for high-level behaviour that maps provide for spatial navigation. Consider the classic reversal task with two reward-predictive cues, A and B. This task can be mapped as two distinct states (S₁ and S₂) that describe the likelihood of each cue predicting reward, with some probability of transitioning between the two states, T. Without such a map, the task could still be completed through trial-and-error learning (bottom, blue trace). However, the map enables more-flexible switching between states (bottom, red trace) because the participant can inferthe outcomes associated with each cue from a single trial.

Fig. 4 ∣. Responses in human vmPFC reflect encoding of state information.

a ∣ Map of the task. Participants saw houses and faces and had to judge the age of one of the categories. When the age changed, they had to switch to judging the other category. Each combination of task contingencies is defined as a state (circles), with transitions between each state governed by the structure of the task. b ∣ The researchers trained an algorithm to decode which of the 16 states was currently in effect. The only region in which this information could be decoded above chance was the ventromedial prefrontal cortex (vmPFC). c ∣ The degree to which state information could be extracted from vmPFC blood oxygen level-dependent signals (x-axis) predicted the number of incorrect decisions participants made (y-axis). Figure adapted with permission from REF., Elsevier.

Fig. 5 ∣. Value place neurons in the primate hippocampus.

Monkeys were trained to choose between pairs of presented pictures. There were three pictures in total and each was associated with a probability of receiving juice that gradually changed overtime, requiring the animal to track the changing contingencies. a ∣ A value space can be constructed in which each axis is the value of one of the pictures, such that a point in this space defines the value of the pictures relative to one another. The changing reward contingencies result in a circular trajectory through this value space. On the right, the same picture values are plotted independently. b ∣ Four examples of firing patterns of hippocampal (HC) neurons, showing how the firing rate varied across two successive completions of the circular trajectory. Peak firing rate values are shown for each neuron. Neurons encoded specific locations in value space. c ∣ Top: Activity of four example HC value place neurons plotted following conventions in part b. Bottom: Four example orbitofrontal cortex (OFC) neurons, plotted in the same manner. d ∣ Average peak spatial information encoded by OFC value neurons (left, purple) and HC value place neurons (right) on the circular trajectory. Figure adapted with permission from REF., Elsevier.