Context familiarity enhances target processing by inferior temporal cortex neurons

Abstract

Experience-dependent changes in the response properties of ventral visual stream neurons are thought to underlie our ability to rapidly and efficiently recognize visual objects. How these neural changes are related to efficient visual processing during natural vision remains unclear. Here, we demonstrate a neurophysiological correlate of efficient visual search through highly familiar object arrays. Humans and monkeys are faster at locating the same target when it is surrounded by familiar compared with unfamiliar distractors. We show that this behavioral enhancement is driven by an increased sensitivity of target-selective neurons in inferior temporal cortex. This results from an increased "signal" for target representations and decreased "noise" from neighboring familiar distractors. These data highlight the dynamic properties of the inferior temporal cortex neurons and add to a growing body of evidence demonstrating how experience shapes neural processing in the ventral visual stream.

Figure 1.

Visual search task. After fixating a central square, the stimulus array appeared and the monkey was allowed to freely view the display. The monkey's task was to press a button (left or right) that had been previously associated with the target. In the example displays, the target is the image of the silverware. The target appeared at 0° eccentricity in the left panel and at 4° eccentricity in the right panel. Dashed gray lines denote possible target eccentricities (0, 2, 4, 6, or 8°) and were not present in the actual displays. The left and right panels depict trials for monkey S with familiar and unfamiliar distractors, respectively. Note that distractor class is not distinguishable without previous knowledge of the monkey's experience with these specific images. The actual display contained full-color objects.

Figure 2.

Stimulus-selectivity for an example neuron and the neuron population. Neurons were chosen based on their selectivity for one target, the effective target, during a passive viewing task. Other targets and distractors were selected such that they did not activate the neuron to the same extent as the effective target. A, Response of an example neuron (same cell in Figs. 4A, 7A) to all 24 stimuli used in the visual search task. The response to the effective target was strongest. B, The same was true across the population of neurons. The normalized response to the effective target, the ineffective targets, and the best distractors (i.e., distractors that caused largest response) are shown. Lightly shaded regions denote 95% confidence intervals. C, Median responses of the neuron population to the familiar and unfamiliar distractors did not differ during the early (50–175 ms) or late (175–300 ms) response period across all distractors tested, or when limiting the comparison to the best distractors or worst distractor (p > 0.15 in all cases). Box denotes the 25th to 75th percentile and line denotes the 10th to 90th percentile.

Figure 3.

Behavioral results from the visual search task. A, Visual search for targets embedded among familiar distractors was ∼100 ms faster than search among unfamiliar distractors (p < 0.000001). B, Monkeys also required fewer fixations to locate targets among familiar distractors (p < 0.000001). C, There was no consistent difference in fixation durations across distractor type (p = 0.70). Center line denotes the mean, the box denotes the 25th to 75th percentile, and the line denotes the 10th to 90th percentile. D, Proportion of saccades to the target as a function of distractor class and the distance between fixation and the target position. Monkeys were more likely to saccade to a target embedded among familiar distractors and this difference was limited to nearby targets (p < 0.001). Error bars denote SEM.

Figure 4.

Neural response during active exploration of the search array. A, Example trial with the effective target for one neuron (same cell in Fig. 2A). Horizontal (H) and vertical (V) eye position (top, solid lines) and the target position (dashed lines) as well as the distance from the gaze position to the target (middle) are shown. Shaded regions denote fixations during search. Spike times, backward-shifted by the onset latency of the neuron, are depicted by the black vertical marks. Firing rates during each fixation were binned by the distance between the fixation position and the target position (bottom). Error bars denote SE. B, Normalized population response (±SE) during active exploration. For the effective target, the response magnitude decreased with increasing distance between the fixation position and the target (p < 0.00001) and fell off quicker when targets were embedded among unfamiliar distractors. Across all eccentricities, there was a 25% decrease in neural activity when targets were among unfamiliar distractors (p < 0.001). There was no significant difference across distractor condition when the target of the search was an ineffective target (p = 0.42).

Figure 5.

Control analysis for the unbalanced number of fixations across distractor type contributing to the active exploration analysis. A, Histogram of response times for trials with familiar (upward bars) and unfamiliar distractors (downward bars) for an example neuron. For this analysis, the fastest familiar distractor trials and slowest unfamiliar distractor trials were removed (light gray bars). For the remaining subset of trials, search was, on average, faster for unfamiliar distractor trials. B, Mean response times and number of fixations across the population of control trial subsets. In contrast to the results for the full trial set (see Fig. 3), response times were longer (p < 0.001) and more fixations were required (p = 0.001) for search among familiar distractors. The box denotes the 25th to 75th percentile and the line denotes the 10th to 90th percentile. C, Normalized population response during active exploration of the search array for the subset of control trials. Results were comparable with the same analysis on the full data set (see Fig. 7). For the effective target, the response magnitude decreased with increasing distance between the fixation position and the target (p < 0.00001) and fell off quicker when targets were embedded among unfamiliar distractors. Across all eccentricities, there was a 24% decrease in neural activity when targets were among unfamiliar distractors (p = 0.005). There was a significant interaction between target eccentricity and distractor type (p = 0.0002). There was no significant difference across distractor condition when the target of the search was an ineffective target (p = 0.17).

Figure 6.

Control analysis for possible differences in eye movement patterns across distractor type. The normalized population response during active exploration of the search array is plotted as a function of distractor type and distance between fixation and target position. For this subset of trials, all fixations that were immediately followed by a saccade to the target were removed. The results were comparable with the same analysis on the full data set (see Fig. 7). For the effective target, the response magnitude decreased with increasing distance between the fixation position and the target (p < 0.00001) and fell off quicker when targets were embedded among unfamiliar distractors. Across all eccentricities, there was a 25% decrease in target-related activity for searches through unfamiliar distractors (p = 0.007). There was a significant interaction between target eccentricity and distractor type (p = 0.001) for this control analysis. There was no significant difference in neural modulation across distractor condition when the search target was an ineffective stimulus (p = 0.38). D, Distractor; T, target.

Figure 7.

Neural response to the onset of the search array. A, Raster and spike density functions for an example neuron (same cell as in Fig. 2A) aligned to stimulus onset as a function of target eccentricity and distractor type. For effective target trials, the transient response magnitude decreased with increasing target eccentricity and was generally stronger when distractors were familiar. Rasters are only shown for effective target trials. B, Normalized population response histograms aligned to the onset latency of each neuron. C, Normalized population response (±SE) to the array onset in a 155 ms time window (shaded region in A and B) starting at the response latency of each neuron, as measured in a separate passive-viewing task. For the effective target, the response magnitude decreased with increasing target eccentricity (p < 0.00001) and fell off quicker when targets were embedded among unfamiliar distractors. Across all eccentricities, there was a 28% decrease in response magnitude when targets were among unfamiliar distractors (p < 0.001). There was no significant difference across distractor condition when the target of the search was an ineffective target (p = 0.37). D, Similar effects of distractor (D) familiarity were apparent when limiting this analysis to only those trials in which the monkey's first saccade was not to the target (T).

Figure 8.

Comparison of behavioral and neural effects across experimental session. Histograms of DFIs are shown for response time (far right) and firing rate during the array onset (top left) and active exploration (top right) analysis epochs. Positive values for DFI_FR indicate higher firing rates in response to the effective target when distractors were familiar. Negative values for DFI_RT indicate lower response times for familiar distractor searches. Arrowheads denote distribution means, which were all significantly different from zero (p < 0.00001 in all cases). The magnitudes of the behavioral and neural effects were significantly correlated (Pearson coefficient, r_p) across experimental session for both the array onset analysis (p = 0.02) and the active exploration analysis (p = 0.02). The open circle denotes the example neuron used in Figures 2A, 4A, and 7A.

Figure 9.

Multiunit activity during passive-viewing and visual search. A, Population MUA during passive viewing. Shaded region denotes 95% confidence intervals. B, Unfamiliar distractors induced stronger MUA than familiar distractors during both the early (50–175 ms) and late (175–300 ms) response periods. This difference was significant across all distractors tested, as well as when limiting the comparison to the “best” distractors or “worst” distractor of each class (p < 0.002 in all cases). The center line denotes median, the box denotes the 25th to 75th percentile, and the line denotes the 10th to 90th percentile. C, Selectivity broadness (proportion of stimuli causing a significant MUA response) was higher across unfamiliar distractors compared with familiar distractors during passive viewing for both the early (p = 0.004) and late (p = 0.000006) response phases. D, Histograms of distractor familiarity effect indices for MUA (DFI_MUA) are shown for the array onset (top) and active exploration (bottom) analysis epochs. These data are limited to ineffective target trials. Arrowheads denote distribution means, which were both significantly less than zero (p < 0.05 in both cases).

Figure 10.

Basic diagram of the experience-dependent changes in the response properties of IT neurons accounting for the distractor familiarity effect. A, B, The activity in a hypothetical population of IT neurons is shown during visual search through unfamiliar (A) and familiar distractors (B). Active neurons are denoted in black. Local inhibitory neurons are drawn as circles; output neurons are drawn as hexagons. T denotes a target-selective output neuron and D denotes distractor-selective output neurons. The wider the line connecting a neuron and its target, the stronger the connection. Long-term experience with a distractor set leads to (1) sparser activity in IT for both local inhibitory neurons and distractor-selective output neurons, (2) less inhibitory influence on the output neurons of IT, and (3) a stronger output for target-selective neurons. The end result is a higher signal-to-noise ratio for target representations compared with familiar distractor representations. Note that changes in other areas, such as the input to IT from “lower” visual areas, are also likely to occur, but are not depicted here.