Spatiotemporal structure of nonlinear subunits in macaque visual cortex

Abstract

The primate visual system is arranged hierarchically, starting from the retina and continuing through a series of extrastriate visual areas. Selectivity for motion is first found in individual neurons in the primate visual cortex (V1), in which many simple cells respond selectively to the direction and speed of moving stimuli. Beyond simple cells, most studies of direction selectivity have focused on either V1 complex cells or neurons in the middle temporal area (MT/V5). To understand how visual information is transferred along this pathway, we have studied all three types of neurons, using a reverse correlation procedure to obtain high spatial and temporal resolution maps of activity for different motion stimuli. Most complex and MT cells showed strong second-order interactions, indicating that they were tuned for particular displacements of an apparent motion stimulus. The spatiotemporal structure of these interactions showed a high degree of similarity between the populations of V1 complex cells and MT cells, in terms of the spatiotemporal limits and preferences for motion and their two-dimensional spatial structure. Much of the structure in the V1 and MT second-order kernels could be accounted for on the basis of the first-order responses of V1 simple cells, under the assumption of a Reichardt or motion-energy type of computation.

Figure 1.

a, Example interaction function I(Δx, Δy, Δτ, τ₂). Each square contains a displacement map, which codes the response as a function of the spatial separation between two stimuli (Δx, Δy). The position of the second stimulus in any two-spot sequence is the origin of each map. Positive (facilitatory) responses are coded in orange, and negative (suppressive) responses are coded in blue. The response of the cell in the figure was therefore facilitated by a rightward apparent motion sequence and suppressed by a leftward sequence. Each row indicates a different temporal separation between stimuli (Δτ), and each column indicates a different correlation delay (τ₂). The displacement map outlined in thick black is the peak map, which is defined as the map that has the highest variance. Pixels that differ from baseline by±3 SDs (see Materials and Methods) are outlined by the dotted contour. b, Variance as a function of time for the neuron depicted in a. The solid curve indicates the variance in each displacement map as a function of delay (τ₂), when Δτ = 17 ms (1 monitor refresh). The dashed curve shows the same time course for the case in which Δτ = 33 ms (2 refreshes), and the dotted curve corresponds to Δτ = 50 ms (3 refreshes). The vertical dashed line indicates the peak response, which corresponds to the highlighted map in a. The horizontal dashed line is the threshold above which a response is considered significantly above the baseline noise. The vertical arrow indicates the interval over which the baseline noise is measured.

Figure 2.

a, The peak displacement map from the neuron in Figure 1a. The dashed line connects the peak of the facilitatory region to the origin of the map and is used to determine the spatial profile of the response in b. The dashed circle delineates the region over which profiles are measured. The dashed ovals show elliptical Gaussian fits to the facilitatory and suppressive response regions. b, The spatial profile of the map shown in a. Each dot corresponds to the response at one point along the profile. The solid blue curve is the Gabor fit to these points. The dashed vertical line indicates the point at which the spatial displacement (Δs) is 0. Different features of the Gabor fit are shown as dotted vertical lines, which indicate, from left to right, the zero-crossing for facilitation, the peak for facilitation, the peak for suppressive, and the zero-crossing for suppression.

Figure 3.

Correlations between aspects of the best-fitting Gabor functions and the retinal eccentricities of the neurons. Open circles indicate V1 neurons, and filled circles indicate MT neurons. The dashed lines show the regression lines. a, Correlation between retinal eccentricity and Gabor frequency. b, Correlation between retinal eccentricity and Gabor envelope width.

Figure 4.

a, Displacement maps computed at different eccentricities within the same MT receptive field. The dashed squares show the sizes and positions of random-dot patches used to measure speed tuning at the corresponding receptive field locations. b, Spatial profiles for the foveal (red) and peripheral (blue) displacement maps shown in a. Each dot corresponds to the response at a point along a line through the peak of each map. The solid curves are the best-fitting Gabor functions. c, Speed-tuning curves collected with random-dot fields centered on the foveal (red) and peripheral (blue) receptive field locations. Error bars indicate SD of the mean. d, Shift in frequency of the best-fitting Gabor as a function of change in retinal eccentricity for 37 displacement maps computed within the receptive fields of 17 MT neurons. Each dot corresponds to the difference in frequency for a given change in retinal eccentricity, and the dashed line is the result of a linear regression.

Figure 5.

a, Measurements of D_opt, which is the spatial displacement yielding the maximal facilitatory or suppressive response for each neuron. The maximum is defined from the peak (or trough) of the Gabor fit to the spatial profile, as shown in Figure 2 b. The population of V1 cells is shown on the left, and MT cells are on the right. Facilitation is on top, and suppression is on the bottom. b, The x-axis shows, for 94 MT neurons, the peak of a log-Gaussian fit to speed-tuning curves like those shown in Figure 4c. The y-axis shows D_opt for facilitation (filled circles) and suppression (open circles).

Figure 6.

Measurements of D_max, which is the maximal spatial displacement to which a neuron is sensitive. This is defined as the zero-crossing of the Gabor fit to the spatial profile shown in Figure 2b. Population histograms for D_max are shown for V1 (left) and MT (right), for facilitation (top) and suppression (bottom).

Figure 7.

Measurements of the subunit aspect ratio. Aspect ratio was defined as the ratio of length to width of the best-fitting elliptical Gaussian for facilitatory (top) and suppressive (bottom) fields in V1 (left) and MT (right).

Figure 8.

Examples of maps showing the spatial profile as a function of the temporal separation (Δτ) between stimuli for three MT cells (top) and three V1 cells (bottom). For each map, the rows are color-coded spatial profiles, with orange indicating facilitation and blue indicating suppression. The columns correspond to different values of Δτ, and the vertical dashed lines correspond to zero spatial displacement.

Figure 9.

Measures of separability in V1 (left) and MT (right). The top row shows population histograms of the separability index, calculated from a singular value decomposition of maps like those shown in Figure 8. The bottom row shows the tilt direction index, calculated from the discrete Fourier transform of the maps (see Materials and Methods).

Figure 10.

A simple model of velocity tuning (Levitt et al., 1994). a, The leftmost map corresponds to the example shown in Figure 8 (bottom left). The horizontal and vertical dotted lines intersect at the point of peak facilitation, and the curves shown in the horizontal and vertical boxes correspond to response profiles taken along these lines. The middle map indicates the “separable” prediction, defined as the outer product of the horizontal and vertical curves shown next to leftmost map. The rightmost map shows the “velocity-tuned” prediction, obtained by shifting the peak spatial profile by an appropriate amount to maintain the preferred velocity (Δs/Δτ). b, Same as a but for the example neuron shown in the bottom right of Figure 8.

Figure 11.

Distributions of correlation values for the separable and inseparable predictions for V1 (top left) and MT (top right). Each dot corresponds to a neuron, and the solid gray lines show thresholds for significance at the level of p < 0.05. Neurons that fall between the two lines cannot be categorized as being consistent with either prediction. The bottom row shows a comparison of the correlation values for 30 MT neurons for which displacements maps were computed using long bars (left) and small spots (right). b, Normalized response strength as a function of the temporal separation between stimuli (Δτ) for the population of neurons in V1 (black) and MT (gray).

Figure 12.

The effect of correlation delay (τ₂) on spatial response profiles. The top row shows maps in which the rows indicate response as a function of spatial displacement. The columns show how the responses change as a function of correlation delay. Beneath each map is the impulse response, taken through the peak of each facilitatory (red) or suppressive (blue) response. b, Population histograms of the biphasic index (see Materials and Methods) for V1 (left) and MT (right). Neurons with a biphasic index near 0 maintained their preferred direction throughout the duration of their response, whereas neurons with a biphasic index near 1 showed a reversal in preferred direction at long correlation delays.

Figure 13.

a, The first-order response of a V1 simple cell. Here, each map shows the response of a neuron as a function of spatial position (x,y) at a different correlation delay (τ). The solid lines connect pairs of maps that were cross-correlated to obtain simulated second-order maps. b, Simulated displacement maps showing predicted responses as a function of spatial displacement (Δx, Δy), with correlation delay τ₂ increasing from left to right. The maps were calculated from the simple cell responses shown in a. On the right is shown the direction-tuning curve for the same neuron in response to a long bar moving in a direction perpendicular to its orientation. c, Space–time map computed from the maps shown in a (left), along with a simulated Δs–Δτ map computed from the displacement maps shown in b (right).

Figure 14.

Population histograms for second-order kernels derived from first-order responses of V1 simple cells. The histograms show subunit aspect ratio (top left), D_opt for facilitation (top right), Gabor frequency (bottom left), and tilt direction index (bottom right) for the simulated interaction kernels. The same values for MT (solid lines) and V1 (dashed lines) are superimposed on each histogram.

Figure 15.

Predictions of full first- and second-order kernels. The first row shows an example V1 simple cell, along with its predicted first-order (solid line) and second-order (dotted line) direction-tuning curves. The combined tuning curve is shown in the second column, and the tuning curve measured with drifting random-dot fields is shown in the third column. The extent to which the primary kernel (first-order) and combined first- and second-order kernels capture the measured tuning curve are shown as correlation coefficients in the fourth column. Rows 2 and 3 show similar data for V1 complex cells and MT cells.