Disparity-based coding of three-dimensional surface orientation by macaque middle temporal neurons

Abstract

Gradients of binocular disparity across the visual field provide a potent cue to the three-dimensional (3-D) orientation of surfaces in a scene. Neurons selective for 3-D surface orientation defined by disparity gradients have recently been described in parietal cortex, but little is known about where and how this selectivity arises within the visual pathways. Because the middle temporal area (MT) has previously been implicated in depth perception, we tested whether MT neurons could signal the 3-D orientation (as parameterized by tilt and slant) of planar surfaces that were depicted by random-dot stereograms containing a linear gradient of horizontal disparities. We find that many MT neurons are tuned for 3-D surface orientation, and that tilt and slant generally have independent effects on MT responses. This separable coding of tilt and slant is reminiscent of the joint coding of variables in other areas (e.g., orientation and spatial frequency in V1). We show that tilt tuning remains unchanged when all coherent motion is removed from the visual stimuli, indicating that tilt selectivity is not a byproduct of 3-D velocity coding. Moreover, tilt tuning is typically insensitive to changes in the mean disparity (depth) of gradient stimuli, indicating that tilt tuning cannot be explained by conventional tuning for frontoparallel disparities. Finally, we explore the receptive field mechanisms underlying selectivity for 3-D surface orientation, and we show that tilt tuning arises through heterogeneous disparity tuning within the receptive fields of MT neurons. Our findings show that MT neurons carry high-level signals about 3-D surface structure, in addition to coding retinal image velocities.

Figure 1.

Schematic illustration of the 3-D orientation of planar surfaces, parameterized by tilt and slant. A, Tilt refers to the axis around which the plane is rotated away from frontoparallel, and slant defines the amount by which the plane is rotated (Gibson, 1950; Stevens, 1983). Zero slant corresponds to a frontoparallel surface for which the tilt is undefined. In this illustration, tilt and slant are defined by perspective and texture gradient cues. In our experiments, surface orientation was defined solely by the direction and magnitude of a linear gradient of horizontal disparity. B, An example of a 0 degree tilt stimulus depicted using a red-green anaglyph. Dots in our experiments were all red, and stereoscopic presentation was accomplished using ferroelectric liquid crystal shutters synchronized to the monitor refresh.

Figure 5.

Tilt tuning does not result from monocular cues. A, Data from an example neuron. Solid curves show binocular tilt-tuning measurements taken at three mean disparities ranging from -0.55 to 0.05°. Dashed curves show tilt tuning for monocular controls in which only the left or right half-image was presented to the monkey. Direction of motion, 280°; speed of motion, 17°/sec; aperture diameter, 4°; eccentricity, 8.5°; surround inhibition, 72%. B, For each neuron tested (n = 15), TDI values calculated from the left-eye (circles) and right-eye (triangles) half-images are plotted against TDI values calculated from binocular stimuli at each of three mean disparities. Thus, there are 90 data points in this plot: two eyes × three mean disparities × 15 neurons. Filled symbols denote monocular controls for which tilt tuning was statistically significant (ANOVA; p < 0.05). The dashed line has unity slope, and the solid line is the best fit to the data using linear regression.

Figure 3.

Horizontal disparity-tuning curves (left) and tilt-tuning curves (right) for four additional MT neurons. The format is similar to that of Figure 2, A and D, except that different mean disparities of the gradient stimulus are denoted here by different symbol types. Stimulus parameters were as follows: A, direction of motion, 70°; speed of motion, 1°/sec; aperture diameter, 4°; eccentricity, 5.5°; surround inhibition, 29%. B, Direction of motion, 161°; speed of motion, 1.5°/sec; aperture diameter, 27°; eccentricity, 10°; surround inhibition, 0%. C, Direction of motion, 135°; speed of motion, 12°/sec; aperture diameter, 27°; eccentricity, 15°; surround inhibition, 17%. D, Direction of motion, 250°; speed of motion, 8°/sec; aperture diameter, 24°; eccentricity, 11°; surround inhibition, 0%.

Figure 10.

Tilt tuning strength is not correlated with either the strength or spatial asymmetry of surround inhibition. A, The TDI is plotted against the percentage of surround inhibition for the 97 MT neurons in our sample. Neurons with significant surround inhibition are indicated by filled symbols (p < 0.05). B, TDI is plotted against the surround asymmetry index (see text) for 37 MT neurons that were tested with the stimulus configuration of Figure 8C.

Figure 8.

Tilt tuning can be predicted from three-dimensional receptive field substructure. All data in this Figure were taken from a single MT neuron. A, Receptive field map, conventions as in Figure 2C. B, Size-tuning curve, conventions as in Figure 2 B. C, Schematic illustration of the stimulus used to probe receptive field substructure. The receptive field (dashed circle, corresponding to that in A) was divided into seven subregions: a center patch containing dots at the preferred disparity, and six surrounding patches having variable disparities. A small (2°) patch of zero-disparity dots (yellow) was presented around the fixation point (white square) to help anchor vergence. D, Seven disparity-tuning curves are shown, corresponding to the stimulus locations in C. The six tuning curves around the perimeter show the disparity tuning of the neuron at each of the locations where the disparity of the surrounding patch was varied. The solid horizontal line in each of these plots shows the response of the neuron to the center patch when presented alone (at the optimal disparity). The dashed horizontal lines denote the level of spontaneous activity. The central disparity-tuning curve shows the measurement obtained with a large patch of dots that covered the entire receptive field (conventions as in Figs. 2 and 3). Arrowheads denote the mean disparities of the gradient stimulus used to test the neuron in E. E, Tilt tuning tested at five mean disparities ranging from -0.35 to 0.85° (conventions as in Fig. 3). Direction of motion, 230°; speed of motion, 2°/sec; aperture diameter, 24°; eccentricity, 19°; surround inhibition, 25%. F, Tilt-tuning curves predicted from a simple model based on linear summation of the responses in D (see in text for details).

Figure 9.

Summary of the quality of model predictions of tilt tuning. A, Tilt-tuning curves for an MT neuron taken at three mean disparities ranging from 0.5 to 1.1°. Direction of motion, 100°; speed of motion, 6°/sec; aperture diameter, 14°; eccentricity, 6.9°; surround inhibition, 75%. B, Model predictions for the neuron in A. C, Tilt tuning of a second MT neuron tested at three mean disparities ranging from -0.8 to 0.0°. Direction of motion, 255°; speed of motion, 8°/sec; aperture diameter, 26°; eccentricity, 11.2°; surround inhibition, 24%. D, Model predictions for neuron in B. E, Distribution of the absolute value of the difference in tilt preference,|ΔPref. Tilt|, between the predicted and the observed responses. Values of|ΔPref. Tilt| are shown for 24 mean disparities (with significant tilt tuning) from nine neurons. F, Distribution of correlation coefficients (R) between measured tilt-tuning curves and model predictions. One R value was computed for each of 24 means disparities from the same nine neurons as in E.

Figure 2.

A dataset for an example MT neuron that exhibits tilt selectivity. A, A conventional disparity-tuning curve measured using random-dot stereograms (i.e., slant was zero, and different uniform horizontal disparities were applied). Mean responses ± SE are shown for each different stimulus disparity, along with a spline fit. Colored arrowheads indicate the five mean disparities used for the disparity gradients in D. B, A size-tuning (area summation) curve. A frontoparallel (zero slant) surface was presented at the preferred disparity, and the diameter of the stimulus aperture varied. The response of this neuron was abolished at large sizes, indicating the presence of powerful (96%) surround inhibition. C, A quantitative receptive-field map was measured by presenting small (1.3 × 1.3°) patches of dots at 16 spatial locations on a 4 × 4 grid. Response strength is color-coded, from low (dark blue) to high (red); peak response was 45 spikes/sec. The dashed white circle shows the location and size of the stimulus aperture in which disparity gradient stimuli were presented. D, Tilt-tuning curves at five different mean disparities (color-coded). Smooth curves indicate the best fits of the modified sinusoid (Eqs. 1, 2). Stimulus parameters were as follows: direction of motion, 105° (convention: rightward, 0°; upward, 90°); speed of motion, 17°/sec; aperture diameter, 6°; eccentricity, 6.8°; and gradient magnitude, 0.2°/°.

Figure 4.

Population summary of tilt selectivity. A, Summary of tuning strength. The tilt discrimination index (Eq. 4) is plotted against the tilt modulation index (Eq. 3) for 97 MT neurons. Squares and triangles indicate data from monkeys B and J, respectively. Filled symbols indicate neurons for which the main effect of tilt was significant (two-way ANOVA; p < 0.05). Histograms along the margins of the scatter plot give distributions of the discrimination and modulation indices. B, Summary of the consistency of preferred tilts across mean disparities. For each neuron, we computed the absolute value of the difference in tilt preference,|ΔPref. Tilt|, between all unique pairs of mean disparities for which there was significant tilt tuning (ANOVA, p < 0.05). The scatter plot shows 219|ΔPref. Tilt| values (from 64 neurons) plotted against the tilt discrimination index for each neuron. There are up to 10 data points for each neuron, aligned vertically. Open symbols show the largest value of|ΔPref. Tilt| for each neuron. The histogram (right) shows the marginal distribution of|ΔPref. Tilt|.

Figure 6.

Joint coding of tilt and slant. A, Tilt-tuning measurements made at six different slants for the same MT neuron as in Figure 2. From top to bottom, the disparity gradient magnitudes are 0.002, 0.05, 0.1, 0.15, 0.2, and 0.25°/°. The corresponding slants are given along the right side of the plot. Smooth curves are the best fitting sinusoids (Eqs. 1, 2), and have been shifted vertically to minimize overlap and increase clarity. Calibration: 20 spikes/sec. B, TDI is plotted as a function of slant for all 29 neurons that were tested in the joint tilt-slant experiment. Each data point shows the TDI value for one slant, such that each neuron is represented four to six times in this plot. Open symbols indicate the slant value at which the maximum TDI was obtained for each neuron. C, Preferred tilt is plotted as a function of slant for the same 29 neurons. Preferred tilt values are only shown for slants at which the tilt tuning was significant (ANOVA; p < 0.05). Filled symbols denote neurons for which the tilt preference was statistically independent of slant (sequential F test; p > 0.05). Stars denote data for the example neuron from A.

Figure 7.

Tilt tuning does not require coherent motion. A, Ten MT neurons were tested (at three mean disparities each) using both coherent motion and noncoherent motion. Noncoherent stimuli were either stationary (gray symbols) or 0% coherence (black symbols). For each neuron, TDI values were computed at each mean disparity and for each motion condition; these values are compared across motion conditions in the scatter plot (n = 30). The solid diagonal line has unity slope. B, For each mean disparity that exhibited significant tilt tuning using both coherent and noncoherent motion (14/30), we computed the absolute difference between the preferred tilts, and these are plotted as a histogram. Gray and black filled bars denote the stationary and 0% coherence cases.