Contour Curvature As an Invariant Code for Objects in Visual Area V4

Abstract

Size-invariant object recognition-the ability to recognize objects across transformations of scale-is a fundamental feature of biological and artificial vision. To investigate its basis in the primate cerebral cortex, we measured single neuron responses to stimuli of varying size in visual area V4, a cornerstone of the object-processing pathway, in rhesus monkeys (Macaca mulatta). Leveraging two competing models for how neuronal selectivity for the bounding contours of objects may depend on stimulus size, we show that most V4 neurons (∼70%) encode objects in a size-invariant manner, consistent with selectivity for a size-independent parameter of boundary form: for these neurons, "normalized" curvature, rather than "absolute" curvature, provided a better account of responses. Our results demonstrate the suitability of contour curvature as a basis for size-invariant object representation in the visual cortex, and posit V4 as a foundation for behaviorally relevant object codes.

Significance statement: Size-invariant object recognition is a bedrock for many perceptual and cognitive functions. Despite growing neurophysiological evidence for invariant object representations in the primate cortex, we still lack a basic understanding of the encoding rules that govern them. Classic work in the field of visual shape theory has long postulated that a representation of objects based on information about their bounding contours is well suited to mediate such an invariant code. In this study, we provide the first empirical support for this hypothesis, and its instantiation in single neurons of visual area V4.

Keywords: area V4; neurophysiology; object recognition; primate; shape representation; vision.

Figure 1.

Experimental design. A, Parametric shapes with systematic variations in contour curvature. When a shape is scaled, the absolute curvature of a contour segment (convex, red; concave, blue) changes but its normalized curvature does not. Diagonal lines of constant absolute curvature are only illustrative; the empirical slope of these lines depends on the stimulus scales tested and the neuron's curvature sensitivity. Schematics for computing absolute and normalized curvature of a contour segment are shown. B, Overlaid convex and concave shapes (left and right) show that the convex projection and concave indentation at the top were varied systematically to span a range of curvature values while maintaining the rest of the boundary similar across stimuli. C, In the scale test, all shapes were presented at several scales within the RF (dashed circle), typically 0.4–1.0× RF size. D, In the position test, only convex shapes at a single intermediate scale (0.8×) were presented at different locations to control for the positional shifts that were induced by scaling. For concave shapes, the position of the concave indentation did not change as a function of stimulus size (C).

Figure 2.

Response predictions. These predictions are illustrated for a hypothetical neuron selective for shapes with a convex projection at the top. A, If the neuron encodes absolute curvature, we expect systematic shifts in stimulus preference across scale, accompanied by changes in the tuning centroid (triangles). B, If the neuron encodes normalized curvature, we do not expect systematic shifts in stimulus preference or the tuning centroid across scale. C, Tuning centroid across scale. The absolute curvature model predicts systematic shifts in the tuning centroid as a function of size, yielding a line with a non-zero slope, whereas the normalized curvature model predicts a line with a near-zero slope.

Figure 3.

Example responses to the scale test. A–D, Data from four example neurons. Average firing rates to shapes at the optimal stimulus rotation (left). Responses to stimuli at different scales are color-coded (grayscale; expressed as a fraction of RF size); baseline firing rates are also plotted (dashed lines). Tuning centroids across scale (right; also triangles on the left). The responses of neurons in A and B were consistent with the absolute curvature prediction. The responses of neurons in C and D were consistent with the normalized curvature prediction.

Figure 4.

Population analysis of changes in neuronal stimulus preferences across scale. A, Distribution of slopes of the tuning centroids as a function of size for all neurons recorded (N = 80). Only a small subset of neurons showed systematic shifts in their stimulus preferences, as indicated by significant linear regression slopes (N = 13/80; black). B, C, Distribution of the size separability index derived from responses at the optimal stimulus rotation and at all rotations (B and C, respectively). In both cases, neurons showed high separability indices (median = 0.97 in B; 0.95 in C; triangles).

Figure 5.

Absolute curvature model predictions for a neuron consistent with sensitivity to absolute curvature. A, Observed and predicted responses at all stimulus rotations (symbols and lines, respectively; same neuron as in Fig. 3B). Each panel shows the neuron's tuning curves at one of eight stimulus rotations (0°–360°; 45° steps). The preferred contour segment identified by the fitting procedure was a sharp convexity pointing to the lower left, adjoined by a shallow concavity in the counterclockwise direction (see inset shape). B, The model's predicted responses at the optimal stimulus rotation, normalized to the maximum predicted response for each stimulus scale. C, Observed and predicted tuning centroids (black and red); the tuning centroid shifted systematically across scale, for both the observed and predicted responses.

Figure 6.

Absolute curvature model predictions for a neuron consistent with sensitivity to normalized curvature. A, Observed and predicted responses at all stimulus rotations (symbols and lines, respectively; same neuron as in Fig. 3C; same format as in Fig. 5). The neuron's responses were strongly selective for stimulus rotation. The preferred contour segment identified by the fitting procedure was a sharp convexity pointing to the lower left, adjoined by a shallow concavity on either side (see inset shape). B, The model's predicted responses, normalized to the maximum predicted response for each stimulus scale; the model predicted small yet systematic horizontal shifts of the tuning curve across scale. C, Observed and predicted tuning centroids (black and red); the observed tuning centroids did not shift systematically, whereas those predicted by the model did.

Figure 7.

Additional example responses to the scale test. A, B, Observed and predicted responses for two neurons (left; symbols and lines); both neurons preferred the concave indentation in our stimulus set. Observed and predicted tuning centroids for the same neurons (right, black and red).

Figure 8.

Model comparisons. For each neuron, we plot the Z-transformed partial correlation of the normalized curvature model (Zn, ordinate) against the Z-transformed partial correlation of the absolute curvature model (Za, abscissa). Significance bounds (dashed lines) were used to classify neurons as selective for normalized curvature (red), or selective for absolute curvature (black); some neurons were unclassified (gray). Data from the two example neurons in Figures 5 and 6 are highlighted.

Figure 9.

Position test controls. A, B, Data from two example neurons (same as in Fig. 3A, B). Tuning curves for stimuli at different scales (left; replicated from Fig. 3), and tuning curves for stimuli at different positions (right); stimulus positions were chosen so as to match the positional shifts induced by scaling. Both neurons showed shifts in the tuning centroid across scale, but not across position (triangles). C, Population analysis of changes in neuronal stimulus preferences across position. The distribution of slopes of the tuning centroids across position for all convex-preferring neurons (N = 39). D, E, Distribution of the position separability index, derived from responses at the optimal stimulus rotation and at all rotations (D and E, respectively). In both cases, neurons showed high separability indices (median = 0.98 in D; 0.94 in E; triangles).

Figure 10.

Comparing size and position invariance in neurons tested for both stimulus transformations. For neurons that preferred the convex shapes in our stimulus set (N = 39), we compared the linear regression slopes derived from responses in the scale test (abscissa and marginal distribution, below) to the linear regression slopes derived from responses in the position test (ordinate and marginal distribution, left). There was no correlation between the linear regression slopes for the two test conditions.

Figure 11.

Response gain modulation as a function of size. A, Peak neuronal response at each scale, normalized by the maximum response, is plotted as a function of stimulus size. Most neurons (top) showed response enhancement; their responses were strongest for larger stimuli. A smaller subset of neurons (bottom) showed response suppression; their responses were strongest for smaller stimuli. B, Distribution of the ratio of responses to the largest and smallest scales tested, for all neurons; the distribution was broad, suggesting that many neurons carried information about stimulus size.