Spectral receptive fields do not explain tuning for boundary curvature in V4

Abstract

The midlevel visual cortical area V4 in the primate is thought to be critical for the neural representation of visual shape. Several studies agree that V4 neurons respond to contour features, e.g., convexities and concavities along a shape boundary, that are more complex than the oriented segments encoded by neurons in the primary visual cortex. Here we compare two distinct approaches to modeling V4 shape selectivity: one based on a spectral receptive field (SRF) map in the orientation and spatial frequency domain and the other based on a map in an object-centered angular position and contour curvature space. We test the ability of these two characterizations to account for the responses of V4 neurons to a set of parametrically designed two-dimensional shapes recorded previously in the awake macaque. We report two lines of evidence suggesting that the SRF model does not capture the contour sensitivity of V4 neurons. First, the SRF model discards spatial phase information, which is inconsistent with the neuronal data. Second, the amount of variance explained by the SRF model was significantly less than that explained by the contour curvature model. Notably, cells best fit by the curvature model were poorly fit by the SRF model, the latter being appropriate for a subset of V4 neurons that appear to be orientation tuned. These limitations of the SRF model suggest that a full understanding of midlevel shape representation requires more complicated models that preserve phase information and perhaps deal with object segmentation.

Keywords: computational model; macaque monkey; object recognition; shape processing; ventral visual pathway.

Fig. 1.

Set of 51 shapes used to characterize V4 neurons. Each shape was presented at each of 8 orientations, or fewer for shapes with rotational symmetry. For example, the circles (top left) were shown in just 1 conformation because all rotations are identical. Shapes are numbered left to right, top to bottom, starting with 1 at top left. Gray arrow marks shape 24, referred to in results.

Fig. 2.

Shape stimuli and their spectral power representation. A: 2 rotations of shape 18 (Fig. 1) are shown at 128 × 128-pixel resolution. B: log of the Fourier power spectra of A. C: downsampled spectral power representation of B used for fitting. SF, spatial frequency.

Fig. 3.

Comparing responses to a shape and its 180° rotation. Mean responses to a shape and its 180° rotation are plotted against each other for 3 example neurons. Data points are derived from the 42 shapes for which 8 rotations were presented. Each 180° rotation pair contributes two points, (x,y) and (y,x), for a total of 336 points per neuron. The spectral receptive field (SRF) model predicts that y = x, up to noise, and thus a high positive correlation coefficient is expected. Nevertheless, our population contained neurons with positive correlation (r = 0.54; A), no correlation (r = 0.00; B) and negative correlation (r = −0.39; C). b1601, a8602, and b2601 indicate neuronal ID, where the first letter indicates the animal ID.

Fig. 4.

Computing baseline correlation between non-180° shape rotations. A: mean responses of neuron a6802 to pairings of stimuli that are spectrally identical, i.e., 180° rotation, (same analysis and plot format as in Fig. 3). B: similar to A, but responses are plotted for all pairings for a given shape that are not 180° rotations. There are 24 such pairings for each shape with 8 rotations, compared with only 4 pairings for 180° rotations. Correlation coefficients are similar in A and B, r = 0.46 and r = 0.47, respectively, suggesting that for this neuron average responses are higher for some shapes than for others, regardless of rotation.

Fig. 5.

Comparing response correlation for 180° rotations to baseline correlation. A: for each of the 109 neurons, the correlation coefficient for responses to spectrally identical shapes (180° rotations) is plotted against the baseline correlation value (the correlation between responses to non-180° pairings, see results). Points fall on both sides of the line of equality (dashed line). Points plotted as asterisks indicate neurons for which the y-axis deviates from the x-axis by >2 SD, based on a bootstrap estimate of the baseline correlation distribution. Points numbered 1–3 correspond to the 3 example neurons in Fig. 3, point 4 corresponds to the example in Fig. 4, and point 5 corresponds to an example neuron shown below in results. B: the data from A are replotted (gray filled circles) and are compared to predictions for an idealized SRF model and an idealized angular position and curvature (APC) model for each of the 109 neurons. As expected, points for the idealized SRF prediction have much higher correlation for spectrally identical stimuli, because the SRF model predicts identical mean responses for such stimuli.

Fig. 6.

Fit performance of SRF and APC models. A: mean explained variance is plotted for the SRF model and the 2D and 4D versions of the APC model for training data (left) and testing data (right). The training and testing partitions were 75% and 25% of the data, respectively. The SRF model explained more variance in the training data than the APC models, but both APC models out-performed the SRF model on the testing data on average. Error bars show SE. B: explained variance values for the 4D APC model are plotted against those for the SRF model for all 109 neurons. Three examples are indicated: b1601, which was better fit by the SRF model; a6701, which was better fit by the APC model; and b2002, which was about equally well fit by both models.

Fig. 7.

Shape tuning maps for 3 example neurons. A: mean firing rate of neuron b1601 to each shape (drawn in black) is indicated by the color surrounding the shape. Dark blue and dark red indicate the lowest and highest responses, respectively (see scale bar at bottom). This neuron responded best to shapes that had a horizontal alignment, and 180° rotations of the same shape often gave roughly similar responses (black arrow pairs). All rotations (up to 8) of each shape are arranged contiguously within a single column in 1 block. B: responses for example neuron b2002, which tended to prefer shapes with a vertical or right-leaning alignment. Sometimes responses to 180° rotations were similar (e.g., black arrow pair). C: responses for example neuron a6701, which was well fit by the APC model and poorly fit by the SRF model (Fig. 6B). Shapes associated with the strongest responses did not elicit strong responses when rotated by 180° (compare top and bottom arrows in each arrow pair).

Fig. 8.

SRF maps depend on regularization. SRF maps (image panels) are shown for the 3 example cells of Fig. 7 (A–C) and for 3 levels of the regularization parameter (low to high, top to bottom, respectively). Red indicates positive weights, and blue indicates negative weights (see scale bar near bottom of C). Top row shows maps for low λ (0.15). These maps produce the best performance on the training data (black line, top panels, described below) but substantially worse performance on the test data (red line, top panels) because of overfitting. Second row shows maps at the optimal λ (best test performance) for each neuron (λ = 1.52, 5.09, and 0.38 for A–C, respectively). Third row shows maps for high λ (16). Each map shows an example of the SRF given a random selection of training/testing partition (75/25%). Top panels plot the average performance (across 100 random training/test partitions) on the training and test data as a function of λ. Shaded area shows SD. D: average performance on the training and test data as a function of λ across all 109 neurons. Shaded area shows SD.