Optic flow processing in monkey STS: a theoretical and experimental approach

Abstract

How does the brain process visual information about self-motion? In monkey cortex, the analysis of visual motion is performed by successive areas specialized in different aspects of motion processing. Whereas neurons in the middle temporal (MT) area are direction-selective for local motion, neurons in the medial superior temporal (MST) area respond to motion patterns. A neural network model attempts to link these properties to the psychophysics of human heading detection from optic flow. It proposes that populations of neurons represent specific directions of heading. We quantitatively compared single-unit recordings in area MST with single-neuron simulations in this model. Predictions were derived from simulations and subsequently tested in recorded neurons. Neuronal activities depended on the position of the singular point in the optic flow. Best responses to opposing motions occurred for opposite locations of the singular point in the visual field. Excitation by one type of motion is paired with inhibition by the opposite motion. Activity maxima often occur for peripheral singular points. The averaged recorded shape of the response modulations is sigmoidal, which is in agreement with model predictions. We also tested whether the activity of the neuronal population in MST can represent the directions of heading in our stimuli. A simple least-mean-square minimization could retrieve the direction of heading from the neuronal activities with a precision of 4.3 degrees. Our results show good agreement between the proposed model and the neuronal responses in area MST and further support the hypothesis that area MST is involved in visual navigation.

Fig. 1.

Structure of the network model. In the first layer, direction-selective cells (A) modeled after properties of neurons in monkey visual area MT represent the optic flow input. At each receptive field position, columns of several neurons with different preferred directions—four in the drawing—encode the optic flow occurring at that location in the visual field. The combined activity in layer one encodes the optic flow field (B). In the second layer, the motion of the observer is recovered by neuronal populations that are tuned to preferred directions of heading. It contains a two-dimensional retinotopic map of heading directions. Each map position represents a specific direction of heading, given by the retinal projection of the movement of the observer. It is occupied by a column of neurons that receive inputs from different parts of the visual field and respond to optic flow motion patterns. Their response is a function of the direction of heading inherent in the flow field input (C). Individual neurons within a single column might carry different optic flow selectivities, but the combined activity of all neurons within one column is tuned to the direction of heading symbolized by this position in the map. Taken together, the population activity in layer two gives a map of heading directions (D). Connections between the two layers are randomly assigned. Connection strengths, however, are chosen in compliance with the heading detection scheme implemented (Lappe and Rauschecker, 1993a). Neurons from within one second-layer column may receive input from different, potentially overlapping, regions of the visual field and retain very large receptive fields.

Fig. 2.

Example of a network simulation. A, An observer moves on top of a ground plane into a direction indicated by the plus sign (+), which keeps a constant angle with the direction of gaze (x). B, The resulting optic flow field, which is used as input to the network, is a pure expansion with the singular point, i.e., the focus of expansion, located in the direction of the movement (+). C, The population activities in the second layer of the network as a grayscale map. Each square corresponds to one possible direction of heading. The brightness of the square indicates the activity of the population that represents this direction. The computed direction of heading corresponds to the brightness peak and is close to the correct direction (+). Note that the flow field and the output map are drawn on different scales. The diameter of the flow field is 100°, whereas the side length of the output map is only 40°.

Fig. 3.

Schematic illustration of the process of representing a certain direction of heading with a population of neurons. A, The response u(x,y) of a single cell from the second layer of the network to an optic flow input. The map position (x,y) denotes the azimuth and elevation of the direction of heading. The response u is a sigmoid function of the direction of heading. The single cell inA only responds when the direction of heading lies along the vertical meridian or in the left hemifield. B, A second cell from within the same subpopulation in the second layer of the network signals that the direction of heading is located in the right visual field. Both activity profiles overlap near the vertical meridian. Summing the activities of the neurons in A and Band of two more neurons (C, D), also from within the same column but with differently oriented response curves, results in a peak of population activity U for the total neuronal population in the second layer of the network. This peak of activity signals the direction of heading.

Fig. 4.

Simulated responsesu(x,y) of a single neuron from the second layer of the model to optic flow stimuli. The stimuli were pure expansions, contractions, clockwise rotations, and counterclockwise rotations as a function of the location (x,y) of the singular point within the visual field. The simulations show sigmoidal response profiles for all of these stimuli. A comparison of the responses to opposite stimuli reveals a complementary arrangement of areas of best response. Best responses to expansion are obtained in the lower left of the visual field, and best responses to contraction are obtained in the upper right of the visual field. For rotations, clockwise rotation is favored in the left and the center of the visual field, whereas counterclockwise rotation becomes more preferred in the right periphery of the visual field. The neuron is also direction-selective. It prefers full-field unidirectional frontoparallel translation toward the upper right. The receptive field of the neuron covers the lower left quadrant of the visual field and extends up to 10° into the other three quadrants. The neuron receives input from 32 locations from within this receptive field, which are indicated by black dots. At every such location, inputs from all possible local movement directions are present but are weighted according to an algorithm that allows the determination of the direction of heading. The receptive field of the neuron has a complex structure. Different parts of the receptive field can have different selectivities for local motion.

Fig. 5.

Example of a model neuron that is nonselective for rotational flow stimuli. The responsesu(x,y) to expansional and contractional flow stimuli depend in a complementary manner on the location (x,y) of the singular point. The responses to rotational stimuli are independent of the location of the singular point and are identical for both directions of rotation. However, the neuron is direction selective for full-field frontoparallel translation toward the lower right. The receptive field covers the central 90° × 90° of the simulated visual field.

Fig. 6.

Spike trains and peristimulus time histograms for a cell recorded in area MST show a reversal of selectivity depending on the location of the singular point. A, Neuronal activities during expansion (first phase of stimulus) and contraction (second phase) stimulation were recorded for nine locations of the singular point of the flow field. The arrangement of the histograms reflects the location of the singular point during the individual stimulations. One location was in the center of the visual field, and eight locations were arranged equidistantly on a circle of radius 15° around the center. The neuron favors the contraction stimulus when the singular point is placed in the upper left visual hemifield. In contrast, if the singular point is placed (Figure legend continues) to the right of the visual field center, the neuron favors expansion. B, Activities during rotational stimulation were recorded for 17 locations of the singular point. In addition to the 9 inner locations, 8 more locations were arranged on a second circle of radius 40°. Within the 9 central rotation stimuli (15° eccentric), no reversal of selectivity is observed. Instead, the neuron favors clockwise rotations (first stimulus phase) at most positions within the central 30° of the visual field. However, when the singular point is located 40° eccentric, it becomes apparent that the neuron favors counterclockwise rotation when the singular point is located in the upper left periphery of the visual field, and clockwise rotation when the singular point is located in the right or in the lower visual hemifield. C, Directional tuning for full-field frontoparallel translation is toward the left. The polar plot of the directional tuning was obtained by moving a full-field random dot pattern on a circular path in a frontoparallel plane, thereby covering all 360° of motion direction in a single trial. The receptive field covered the left half of the tangent screen, but an area of increased responsibility comprised the lower left hemifield.

Fig. 7.

Activities of a cell from area MST recorded during various optic flow stimulations. The plots are drawn in analogy to the ones used in the examples of model simulations in Figures 4 and 5. They show the activity u in spikes/sec for a number of locations (x,y) of the singular point. Activities during expansion and contraction stimulation were recorded for 17 locations of the singular point of the flow field, distributed around the center (0, 0) of the visual field. For the plots, individual activities recorded at these discrete positions were joined with nearest neighbors by linear triangular segments. Activities recorded during expansion and contraction display a smooth graded profile that conforms with the model predictions. A reversal of selectivity occurs roughly along the vertical meridian. Best responses to expansion and contractions were recorded from opposite areas of the visual field: expansion was favored when the singular point was in the upper visual hemifield (y > 0), and contraction was favored when the singular point was in the lower visual hemifield (y < 0). Responses to rotational flow stimuli were recorded for 9 locations of the singular point of the flow field, centered on the fovea, or 15° eccentric. For these stimuli, the neuron responded only to counterclockwise rotation. However, response strength is modulated strongly by the location of the singular point. Directional tuning for frontoparallel translation was toward the lower left. The receptive field covered the lower left quadrant of the visual field.

Fig. 8.

Examples of different shapes of the response function of neurons with peak response for centered optic flow stimuli. In both cases, the maximum response is reached when the stimulus is placed in the visual field center. The neuron in A, however, exhibits responses of almost the same strength for a number of eccentric positions that form a plateau in one part of the visual field. The neuron in B displays a bell-shaped response function with a clear single peak in the center.

Fig. 9.

Average response curves for different optic flow stimuli. The curves for expansion/contraction show an average of individual curves from 21 neurons that were recorded with 17 locations of the singular point each. Eighteen neurons contributed to the average response curves for rotation. Average response curves were generated by first aligning the response curves from the individual neurons and then averaging over the aligned curves. For the alignment, the gradient of the two-dimensional linear regression was used (see text).Left, Three-dimensional surface plots of the responseu depending on the location (x,y) of the singular point in the flow stimulus. Right, Cross sections through the midlines of the three-dimensional surface plots.

Fig. 10.

Distribution of angular differences between the directions of reversal of optic flow selectivity and the preferred direction for full-field, frontoparallel, unidirectional motion. The direction of reversal of the optic flow selectivity was obtained by fitting a two-dimensional linear regression to 9 or 18 data points. To determine the direction of maximum change from expansion to contraction (or clockwise to counterclockwise rotation, respectively), we performed a regression on the difference between the activity recorded during expansion and the activity recorded during contraction. The gradient of the regression indicates the direction in which the selectivity changes maximally from contraction to expansion. This direction was then subtracted from the preferred direction for full-field, frontoparallel, unidirectional motion. The experimental distributions (top graphs) show a clear correlation between the directions of reversal and the preferred direction: the distributions peak at 180° angular difference for expansion/contraction (top left) and 90° angular difference for rotation (top right). The distributions for 250 randomly selected model neurons (bottom graphs) display the same correlation. Thus, a correlation between the optic flow responses and the directional selectivity is obvious for most neurons, but it is consistent with an involvement in optic flow analysis.

Fig. 11.

Influence of motion parallax. Fifty-five neurons were tested with expansion/contraction stimuli in which all motion parallax was removed. This was obtained by assuming that all visible dots were distributed on a frontoparallel plane instead of in a random three-dimensional cloud. In this case, the distribution of speeds in the stimulus is uniform and depends only on visual eccentricity. For each neuron tested, we computed the direction of maximum change from expansion to contraction as the gradient of a regression on the difference between the nine activity recorded during expansion and those recorded during contraction. This gradient was compared to the one obtained using the stimulus set that did contain motion parallax. The graph in A shows the angular distribution of these differences between the gradients in these two conditions. Bshows the results for 84 randomly selected model neurons. In both cases, the distributions are centered around zero. This indicates that the response modulations in the case of stimuli lacking motion parallax are very similar to the case when motion parallax is present. This holds for the recorded as well as the simulated data.

Fig. 12.

Grayscale plots of computational heading maps obtained from the recorded neuronal activities. A least-square minimization scheme was used to derive the position (x,y) of the singular point of an expanding optic flow stimulus from the neuronal activities (see text for details). All neurons recorded with the 15° expansion stimuli contributed to the computation (N = 31). In the plots, the obtained least-square error for a specific heading direction (x,y) is coded by the gray value at that map location. Brighter gray levels indicate smaller values of the mean-square error. The most likely heading direction is given by the brightest square in the map. For comparison with the true heading direction, which is the focus of expansion in the case we studied, the optic flow stimuli used are plotted on top of the grayscale maps.