A set of high-order spatiotemporal stimuli that elicit motion and reverse-phi percepts

Abstract

Detection of motion is a crucial component of visual processing. To probe the computations underlying motion perception, we created a new class of non-Fourier motion stimuli, characterized by their third- and fourth-order spatiotemporal correlations. As with other non-Fourier stimuli, they lack second-order correlations, and therefore their motion cannot be detected by standard Fourier mechanisms. Additionally, these stimuli lack pairwise spatiotemporal correlation of edges or flicker-and thus, also cannot be detected by extraction of one of these features, followed by standard motion analysis. Nevertheless, many of these stimuli produced apparent motion in human observers. The pattern of responses-i.e., which specific spatiotemporal correlations led to a percept of motion-was highly consistent across subjects. For many of these stimuli, inverting the overall contrast of the stimulus reversed the direction of apparent motion. This "reverse-phi" phenomenon challenges existing models, including models that correlate low-level features and gradient models. Our findings indicate that current knowledge of the computations underlying motion processing is as yet incomplete, and that understanding how high-order spatiotemporal correlations lead to motion percepts will illuminate the computations underlying early motion processing.

Figure 1

Construction of a three-element spatiotemporal glider stimulus. The three-element glider (left) is represented by a wireframe cube with three of its corners colored. The wireframe cube represents a 2 × 2 × 2 region in spacetime, and the three colored corners are the three voxels that form the glider. The coloring indicates the time steps occupied by each voxel: two voxels (green) are at time t, and one (blue) is at time t + 1. The stimulus is constructed by applying an odd parity constraint (for black checks) to the colorings within all occurrences of the glider. That is, every placement of the glider contains an odd number (one or three) of black checks. The checks outlined in color in frame t and t + 1 on the right illustrate this parity constraint for three placements of the glider. The red arrows show, within a glider, how the color of a check in frame t + 1 is determined by the color of other checks in frame t. An example of the resulting movie is provided in the Supplementary data.

Figure 2

For some gliders, standard second-order motion can emerge from spatiotemporal correlations of features. (A) A four-element glider that induces second-order spatiotemporal correlation of edges orthogonal to the x-axis. (B) A four-element glider that induces second-order spatiotemporal correlation of flicker. (C) An x–t slice of the stimulus generated using the glider of (A). (D) An x–t slice of the stimulus generated using the glider of (B). (E) Spatiotemporal correlation of luminance, edges orthogonal to the x-axis, edges orthogonal to the y-axis, and flicker, from sample stimuli constructed by glider A (lower left 4 panels) and glider B (upper right 4 panels), both with even-parity rule. Black lines across the center of each panel separate leftward and rightward correlations.

Figure 3

For other gliders, spatiotemporal correlations do not arise by pairwise correlation of features. (A) A four-element glider that does not consist of two parallel pairs of voxels. (B) A three-element glider. (C) An x–t slice of the stimulus generated using the glider of (A). (D) An x–t slice of the stimulus generated using the glider of (B). (E) Spatiotemporal correlation of luminance, edges orthogonal to the x-axis, edges orthogonal to the y-axis, and flicker, from sample stimuli constructed by glider A (lower left 4 panels) and glider B (upper right 4 panels), both with even-parity rule. Black lines across the center of each panel separate leftward and rightward correlations.

Figure 4

Centroid directions for (A, B) 2 three-element gliders and (C, D, E) 3 four-element gliders. Within each glider, filled green circles represent voxels at time t and filled blue circles represent voxels at time t + 1. The centroid direction is, as indicated by the red arrow, the vector pointing from the centroid of voxels at time t (open green circle) to the centroid of voxels at time t + 1 (open blue circle). Note that when there is only one voxel at a time, the centroid is the voxel itself (as in (A)–(C)).

Figure 5

Results for 10 three-element glider stimuli, tested on 5 subjects. Within the results shown over one glider, responses for the even-parity rule are on the left; responses for the odd-parity rule are on the right. Fraction in centroid direction >0.6 or <0.4 is significant (p < 0.05, two-tailed). The gliders are ordered in time-reversal pairs. That is, the glider in column 2 is the time reversal of the glider in column 1 (and vice versa), the glider in column 4 is the time reversal of the glider in column 3, and so on.

Figure 6

Results for 14 four-element gliders. (A) Gliders with 3 elements at one time, 1 at the adjacent time. (B) Gliders with 2 elements at one time, 2 at the adjacent time. Within the results for one glider, responses for the even-parity rule are on the left; responses for the odd-parity rule are on the right, except for the last glider of (B), where only the even parity was tested. Fraction in centroid direction >0.6 or <0.4 is significant (p < 0.05, two-tailed). As in Figure 5, the gliders are ordered in time-reversal pairs, except for the last two gliders of (B).

Figure 7

Spatiotemporal correlations can arise from interactions of different kinds of features. (A) A four-element glider that shows interaction of an edge in one location and flicker in another location at a separate time. (B) A stimulus generated by the glider in (A) and the even-parity rule showing spatiotemporal correlation of edge and flicker. Half-integer values of the coordinates arise in (B) because we assign a flicker’s time to the time halfway between the frames. (C) A three-element glider showing interaction of an edge in one location and the luminance in another location at a separate time. (D) A stimulus generated by the glider in (C) and the even-parity rule showing spatiotemporal correlation of edge and luminance. Half-integer values arise in (D) because we assign an edge’s location to the position halfway between the corresponding voxels. Note that in (B) and (D), the correlation peak is offset from the origin in both space and time.