Goal-direction and top-down control

Abstract

We review the neural mechanisms that support top-down control of behaviour and suggest that goal-directed behaviour uses two systems that work in concert. A basal ganglia-centred system quickly learns simple, fixed goal-directed behaviours while a prefrontal cortex-centred system gradually learns more complex (abstract or long-term) goal-directed behaviours. Interactions between these two systems allow top-down control mechanisms to learn how to direct behaviour towards a goal but also how to guide behaviour when faced with a novel situation.

Keywords: basal ganglia; cognition; frontal lobe; goal direction; learning.

Figure 1.

Anatomy of neocortex and BG. Cortical projections enter the BG through the striatum and are thought to maintain separation throughout the BG (shown as different shades). ‘Direct’ pathway releases inhibition on the thalamus (labelled as D1+); ‘indirect’ pathway increases inhibition (D2+). Dopaminergic reward signals (in light orange) influence synapses in PFC but are much stronger in striatum. GPe, globus pallidus external parts; GPi, globus pallidus internal parts; SNpr, substantia nigra, pars reticulata; STN, subthalamic nucleus.

Figure 2.

Tree representation of complex tasks. A complex task can be modelled as a set of states (S, circles) with several possible behavioural responses (R, arrows). A response can either lead to a new state or an outcome (squares, either positive, green or negative, red). The fast learning in BG is thought to be ideal for capturing single nodes in the tree (blue cloud) while the slower learning in PFC can capture the entire task (yellow cloud). This more complete view of the task allows for immediately more rewarding responses (e.g. R′₁) to be avoided for longer term goals (e.g. R₁ followed by R₂).

Figure 3.

Cortico-ganglia loops support sequencing of behaviours. An initial state in a sequence is cued; activation of this state suppresses alternative states via lateral inhibition. In addition to local processing, the current state is also initially supported via recurrent loops with thalamus (left; dashed lines show ascending projections). The BG gates activity between PFC and the thalamus, acting to inhibit the current state and releasing inhibition on the next state in the sequence (right).

Figure 4.

Recurrent learning allows development of complex cognitive representations. Associative learning aids categorization. Initially, stimuli belonging to a common category (e.g. matches and a lighter can both ‘start fires’) are likely to have disparate representations (top row). However, BG can learn associations between stimuli and behaviours quickly. As this feeds back into PFC, it changes the representation of the stimulus itself, bringing the overall representations closer together (bottom row).

Figure 5.

Bootstrapping to developing complex cognitive representations. Recurrence allows for iteratively more complex functions to be learned. Initial learning by BG is fast but concrete (left). Learning is slower in PFC but this allows for more generalized functions to be learned over many different experiences (middle). In this example, a response is learned to a category (set) of stimuli. Once learned the more complex functions/representations are available for further learning by BG, allowing for evermore complex functions to be learned (right).