Perceptual learning rules based on reinforcers and attention

Abstract

How does the brain learn those visual features that are relevant for behavior? In this article, we focus on two factors that guide plasticity of visual representations. First, reinforcers cause the global release of diffusive neuromodulatory signals that gate plasticity. Second, attentional feedback signals highlight the chain of neurons between sensory and motor cortex responsible for the selected action. We here propose that the attentional feedback signals guide learning by suppressing plasticity of irrelevant features while permitting the learning of relevant ones. By hypothesizing that sensory signals that are too weak to be perceived can escape from this inhibitory feedback, we bring attentional learning theories and theories that emphasized the importance of neuromodulatory signals into a single, unified framework.

Figure 1. Factors that modulate visual cortical plasticity

(a) Left, a visual stimulus is initially registered by a fast feedforward wave of activity that activates feature selective neurons in the many areas of the visual cortex. Right, neurons in the frontal cortex engage in a competition to select a behavioral response, and the neurons that win the competition feed attentional signals back to the visual cortex. Neuromodulatory systems, including acetylcholine (green) and dopamine (purple), modulate the activity as well as the plasticity of sensory representations. (b) Contour grouping task where monkeys have to trace a target curve (T) that is connected to a fixation point (FP). A red circle at the end of this curve is the target for an eye movement (Figure 1b). The animals have to ignore a distractor curve (D). The two stimuli differ so that the RFs of neurons in areas V1 and FEF are either on the target curve (upper panel) or on the distractor curve (lower panel). (c) Neurons in the frontal cortex (area FEF) and the visual cortex (area V1) enhance their response when the target curve falls in their receptive field. Note that this attentional modulation comes at a delay (striped bar), while the initial neuronal responses do not discriminate between the relevant and irrelevant curve (black bar). Modified from Khayat et al. [70].

Figure 2. Task-irrelevant perceptual learning

(a) Subjects monitor a central RSPV stream to detect target digits that are presented among letter distractors. In the background, motion signals are presented that are irrelevant for the subject's task. (b) A separate test determines the motion sensitivity of the subjects. This test reveals that the sensitivity for the motion direction paired with the target digits (red arrows in a) increases. Learning occurs for paired motion stimuli that are at or below the visibility threshold, but not for very weak or strong motion stimuli. (c) Blue circles denote neurons involved in the RSVP task. Red circles denote the representation of the irrelevant motion signals. Left, weak motion stimuli are not registered well by the visual cortex and are not learned. Middle, threshold motion stimuli are represented by motion sensitive neurons and are learned if paired with the neuromodulatory signal. Right, strong motion signals might cause interference and therefore receive top-down inhibition from the frontal cortex. These suppressive signals block perceptual learning. Adapted from Tsushima et al. [90].

Figure I. Attention-gated reinforcement learning

(a) Left, activity is propagated from lower to higher layers through feedforward connections. The output units engage in a competition to determine the selected action. Right, if an action has been selected, the winning output units provide an attentional feedback signal that highlights the lower level units responsible for the selected action (thick connections) enabling their plasticity. The plasticity of other connections is blocked by inhibition (dashed connections). Different actions thereby enable plasticity for different sensory neurons. Neuromodulators indicate whether the rewarded outcome was better (green) or worse (red) than expected. (b) Inhibition (red connections) blocks the plasticity of connections that are not involved in the selected action. At the output level, the actions compete through inhibitory interactions (connection type 1). The winning output unit could directly inhibit neurons at lower levels (connection type 2) or indirectly through inhibitory lateral interactions between the lower level units (connection type 3). These inhibitory effects do not occur for stimuli too weak to be consciously perceived.