Distinct population codes for attention in the absence and presence of visual stimulation

Abstract

Visual neurons respond more vigorously to an attended stimulus than an unattended one. How the brain prepares for response gain in anticipation of that stimulus is not well understood. One prominent proposal is that anticipation is characterized by gain-like modulations of spontaneous activity similar to gains in stimulus responses. Here we test an alternative idea: anticipation is characterized by a mixture of both increases and decreases of spontaneous firing rates. Such a strategy would be adaptive as it supports a simple linear scheme for disentangling internal, modulatory signals from external, sensory inputs. We recorded populations of V4 neurons in monkeys performing an attention task, and found that attention states are signaled by different mixtures of neurons across the population in the presence or absence of a stimulus. Our findings support a move from a stimulation-invariant account of anticipation towards a richer view of attentional modulation in a diverse neuronal population.

Fig. 1

Task schematic and behavioral results. a Task design. Subjects initiated each trial by fixating their gaze on a central yellow dot. After a 300–500 ms fixation interval, Gabor stimuli were repeatedly flashed for 400 ms, separated by 300–500 ms inter-stimulus intervals. The subjects’ task was to detect a change in orientation of one of the two stimuli (the target) and make a saccade to the stimulus that changed. Each stimulus flash had a fixed chance of containing a target (30% for monkey P, 40% for monkey W). One location had a 90% chance of containing the eventual target (the valid location), and we alternated the side of hightarget probability after 80 correct detections (hits). For the initial trials in each block only one stimulus was presented at the valid location (cue trials) until the subject made 5 correct detections, after which bilateral stimuli were presented for the remainder of the block. The dashed circle represents the receptive field (RF) and was not actually present in the display. b Behavioral results. For each magnitude of orientation change, we calculated the discriminability index (d’). Discriminability was better when targets occurred at the valid location (red circles) compared to when they occurred at the invalid location (blue crosses). Moreover, average response times (insets) were faster for targets at the valid location compared to targets at the invalid location. This pattern of results indicates that the subjects selectively attended to the valid target location at the expense of the invalid location. Error bars indicate ±1 SEM (N = 24 sessions for monkey P and 23 sessions for monkey W)

Fig. 2

Grand-averaged peristimulus spike time histograms (PSTHs). We normalized the responses of each V4 neuron and then averaged over the visually responsive neurons for each session. Responses to sample (non-target) stimuli were stronger when the attention cue was in the RF (red) compared to when the cue was in the opposite hemifield (blue). Shading represents ±1 SEM (N = 47 sessions). Black underlining represents a significant difference between attention conditions (repeated-measures t-test with α = 0.05, uncorrected for multiple comparisons). The earliest detectable attention effect was not until 233 ms, which was later than the median reaction times for the task (inset pictogram; each symbol represents the median RT for one session; circles for monkey P, squares for monkey W. Note that the time axes for the behavioral and neural data are aligned for comparison)

Fig. 3

Example single-unit PSTHs with dynamic attention effects. All of these single units have significantly greater firing rates during the sustained response to the stimulus (200–400 ms; brown arrows) when the attention cue was in the RF (red) compared to when the cue was in the opposite hemifield (blue), and yet they also have significantly lesser firing rates prior to stimulus onset (−200 to 0 ms; orange arrows) when the cue was in the RF compared to when the cue was out of the RF (two-sample t-tests, α = 0.05). This pattern of results defies models of anticipatory attention based on a stimulation-invariant gain modulation, but could account for why population average pre-stimulus attentional modulations are typically much weaker than those observed during stimulus responses. For each neuron, the signal-to-noise ratio (SNR) of the action potential waveform is noted. Shading represents ±1 SEM

Fig. 4

Distributions of observed attention effects during both time periods of interest. The relative attention effect is quantified as: ([cue in RF]–[cue away])/[cue away]. Statistical significance was separately assessed during each time interval of interest using Mann–Whitney U tests with α = 0.05, uncorrected for multiple comparisons. a Distribution of pre-stimulus attention effects. The distribution had near-zero mean, yet many individual neurons were individually significant (orange). Arrowhead indicates mean attention effect value of 0.019 from the full distribution. b Joint distribution of attention effects. Note that many neurons had significant attention effects during both time periods of interest (pink), including neurons that were suppressed by attention pre-stimulus yet facilitated by attention post-stimulus (upper-left quadrant). The number of units with significant effects during both time periods in each quadrant is indicated. c Distribution of post-stimulus effects. Post stimulus, the distribution of attention effects was clearly shifted towards positive values (brown bars show individually significant units), indicating attention-related facilitation, although a minority of cells did show significant attention-related suppression. Arrowhead indicates mean attention effect value of 0.092 from the full distribution

Fig. 5

Illustration of attention axis identification procedure. In this example only two neurons from our sample are used for illustrative simplicity. In actuality, we used 76.6 ± 11.4 (mean ± SD) neurons per session for monkey P and 32.7 ± 17.8 (mean ± SD) neurons per session for monkey W. a First, we identified the post-stimulus attention axis (brown arrow). Each downward-pointing triangle represents the average joint firing rate for the two neurons in response to stimuli that immediately preceded a correctly detected target at the valid location when the RF was the valid target location (red) or when the RF was the invalid target location (blue). The post-stimulus attention axis is the vector through these two points (brown arrow). Second, we found the vector through the average firing rates during the pre-stimulus interval immediately preceding correctly detected targets at the valid location for the two attention conditions (gray arrow). We defined the pre-stimulus attention axis (orange arrow) as the component of the best pre-stimulus vector (gray arrow) that was orthogonal to the post-stimulus attention axis (brown arrow). Note that in two dimensions, as shown, there is trivially only one pre-stimulus attention axis that is orthogonal to the post-stimulus attention axis, but in higher dimensions this is not trivial. Constraining the pre-stimulus attention axis to be orthogonal to the post-stimulus attention axis explicitly eschews a possible interpretation of anticipatory attention based on time-invariant gain (Supplementary Figure 3). That is, we were interested in the degree to which attentional preparation could be captured by a pattern of activity that was linearly independent from that observed in response to a stimulus. Error bars represent ±1 SEM. b Peristimulus spike time histograms (PSTHs) for the neurons depicted in (a). We averaged the responses to all non-target stimuli on trials that resulted with a correct detection of a target at the valid location for each attention condition. Triangles indicate the time periods of interest as in (a). Shading represents ±1 SEM

Fig. 6

Attention axis projections. We separated trials based on the behavioral outcome (hit or miss), and compared distributions of attention axis projections that immediately preceded each target. a–c Schematic of analysis strategy with data from example session. a We separated trials depending on whether the subject correctly detected the target (green) or missed the target (purple). For illustration only, we combined neurons to show multiunit activity (MUA). b We projected the high-dimensional population activity from 200 to 400 ms following the onset of the sample stimulus that preceded each target onto the post-stimulus attention axis (brown) and compared the mean projection when the eventual target was detected (green) to the mean projection when the target was missed (purple; for this example: two-sample one-tailed t-test, p = 0.011). c We performed the same analysis as in (b), projecting the spontaneous population activity during the 200 ms immediately preceding the onset of each target onto the pre-stimulus attention axis (this example: two-sample one-tailed t-test, p = 0.019). d, e Average attention axis projections preceding hit and missed targets for all sessions. Line segments connect observations within each session. d When RF targets were detected (green), the average projection that preceded the target was shifted in the positive direction on both attention axes (consistent with attention towards the RF) relative to when RF target was missed (purple). We tested this difference across sessions with one-tailed repeated-measures t-tests (pre-stimulus, monkey P: p = 0.002, monkey W: p = 0.002; post-stimulus, monkey P: p < 0.001, monkey W: p < 0.001). e When targets out of the RF were detected (green), the average projection that preceded the target was shifted in the negative direction on both attention axes (consistent with attention away from the RF) relative to when the non-RF target was missed (purple; one-tailed repeated-measures t-tests: pre-stimulus, monkey P: p = 0.083 n.s., monkey W: p = 0.039; post-stimulus, monkey P: p = 0.007, monkey W: p = 0.014). However, compared to targets in the RF (d), the average attention axis projections preceding missed targets out of the RF were less consistent

Fig. 7

Hit rate performance for trials partitioned by attention axis positions. We partitioned the full set of trials from each session based on attention axis positions that preceded targets, and then calculated the hit rate for each subset of trials. a Partitioned hit rates for targets in the RF. For each session, we partitioned trials into 25 subsets based on the attention axis projections immediately preceding the target and calculated hit rate within each subset. We normalized the partitioned hit rates within each session by dividing by the overall hit rate for the session, and then averaged over sessions. The best performance (green) was seen when projections on both axes were consistent with attention toward the RF, while the worst performance (purple) was seen when projections on both axes were consistent with attention away from the RF. Within any given bin for one attention axis, variation along the other attention axis was related to performance. b Partitioned hit rates for targets out of the RF. The pattern of results was conversely similar to that seen for RF targets (a), although the relationship between attention axis positions and performance was weaker. c Marginal relationship between pre-stimulus (orange) and post-stimulus (brown) attention axis projections and performance for targets in the RF. Within each session we averaged over the partitions along one attention axis to isolate the unique contribution of the other attention axis. Error bars represent ±1 SEM (N = 47 sessions). d As in (c), but for targets out of the RF. The relationships were inverted and weaker, relative to those seen for RF targets (c)

Fig. 8

The strongest relationship between anticipatory attention states and stimulus responses occurs during the initial transient response. First, we performed a linear regression of the population visual response at each time-point onto the post-stimulus attention axis projection measured during the previous stimulus (brown); the proportion of total variance explained is shown. Second, we performed a linear regression of the residual visual response left unexplained onto the pre-stimulus attention axis projection measured immediately preceding the stimulus (orange); the proportion of residual variance explained is shown. We performed the regression steps in this order to isolate the unique contribution of the pre-stimulus attention state to predictive performance. Both attention axes explain variability in visual responses at earlier time points than was evident when comparing population averages between task block conditions (Fig. 2), although the predictive power of the post-stimulus attention axis peaked around 300 ms (brown arrow). In contrast, the unique contribution of the pre-stimulus attention axis peaked early during the visual response, at 71 ms (orange arrow). The dashed line at VAF = 0.001 represents the average chance value determined by a trial-shuffled control analysis. Shading represents ±1 SEM (N = 47 sessions). Underlining represents a significant difference from chance (one-sample t-test with α = 0.05, corrected for multiple comparisons by requiring 20 contiguous individually significant time points)