Spatial scale and distribution of neurovascular signals underlying decoding of orientation and eye of origin from fMRI data

Abstract

Multivariate pattern analysis of functional magnetic resonance imaging (fMRI) data is widely used, yet the spatial scales and origin of neurovascular signals underlying such analyses remain unclear. We compared decoding performance for stimulus orientation and eye of origin from fMRI measurements in human visual cortex with predictions based on the columnar organization of each feature and estimated the spatial scales of patterns driving decoding. Both orientation and eye of origin could be decoded significantly above chance in early visual areas (V1-V3). Contrary to predictions based on a columnar origin of response biases, decoding performance for eye of origin in V2 and V3 was not significantly lower than that in V1, nor did decoding performance for orientation and eye of origin differ significantly. Instead, response biases for both features showed large-scale organization, evident as a radial bias for orientation, and a nasotemporal bias for eye preference. To determine whether these patterns could drive classification, we quantified the effect on classification performance of binning voxels according to visual field position. Consistent with large-scale biases driving classification, binning by polar angle yielded significantly better decoding performance for orientation than random binning in V1-V3. Similarly, binning by hemifield significantly improved decoding performance for eye of origin. Patterns of orientation and eye preference bias in V2 and V3 showed a substantial degree of spatial correlation with the corresponding patterns in V1, suggesting that response biases in these areas originate in V1. Together, these findings indicate that multivariate classification results need not reflect the underlying columnar organization of neuronal response selectivities in early visual areas.NEW & NOTEWORTHY Large-scale response biases can account for decoding of orientation and eye of origin in human early visual areas V1-V3. For eye of origin this pattern is a nasotemporal bias; for orientation it is a radial bias. Differences in decoding performance across areas and stimulus features are not well predicted by differences in columnar-scale organization of each feature. Large-scale biases in extrastriate areas are spatially correlated with those in V1, suggesting biases originate in primary visual cortex.

Keywords: extrastriate visual cortex; functional magnetic resonance imaging; human visual cortex; multivariate pattern classification analysis; ocular dominance columns; orientation columns; primary visual cortex.

Fig. 1.

Experimental design and analysis. A: event-related fMRI design (single trial shown). Stimulus images consisted of 1 cycle/deg sinusoidal luminance gratings shown within an annular aperture against a uniform gray background. On each trial, sinusoidal gratings of a single orientation were displayed monoptically for 6 s (the other eye was shown a blank gray background). The spatial phase of the gratings changed randomly every 100 ms. Trials were separated by intervals (gray background) varying randomly in length between 12 and 24 s. Subjects performed a luminance-change detection task on a central fixation cross shown dichoptically. B: measurement of perceptual eye dominance (2 consecutive trials shown). Stimuli consisted of 1 cycle/deg sinusoidal luminance gratings oriented 45° left or right of vertical, displayed within circular patches at each of 25 locations across the visual field. On each 6-s trial, a grating stimulus patch was shown for duration of the trial at one location, with the two eyes being shown orthogonal orientations (randomly chosen). Subjects continuously pressed one of two keys to indicate the perceived orientation of the stimulus. Eye preference at each location was computed as the fraction of time dominated by the right eye stimulus. C: time course of stimulus-evoked BOLD response for individual subjects. Each time series shows the stimulus-evoked response (averaged across stimulus conditions and ROIs) estimated by linear deconvolution and averaged across visual areas V1–hV4. Error bars represent average SE of the estimate (square root of average error variance across ROIs) for each time point. D: classification performance for orientation and eye of origin in visual areas V1–hV4. Height of bars indicates proportion of correctly classified trials for each stimulus type. In areas V1–V3, classification performance is significantly above chance level (dotted line) for both orientation and eye of origin for both stimulus orientations; in hV4, classification performance is only significant for oblique stimuli. Error bars represent SE of the mean across subjects.

Fig. 2.

Spatial distribution of orientation preference measured by fMRI. A: spatial distribution of orientation preference plotted in visual field coordinates for each voxel for areas V1–V3 across all subjects. Each plot symbol corresponds to a single voxel, with color representing t values indicating relative preference for horizontal (0°) vs. vertical (90°) stimuli (inset: color map). The size of each plot symbol indicates goodness of fit (R²) of voxel time series. Dotted circles show locations of inner and outer boundaries of stimuli in visual field coordinates. For each area, preference for horizontal orientations predominates along the horizontal meridian, whereas preference for vertical orientations is found mainly along the vertical meridian, consistent with bias for radial orientations. B: same as A but for oblique orientation stimuli. Color represents t values indicating relative preference for rightward oblique (45°) vs. leftward oblique (135°) orientation. Orientation preference shows a radial bias with voxels preferring 45° orientation having receptive fields (RFs) centered on the upper right and lower left quadrants, whereas voxels preferring the orthogonal orientation have RF centers in the other two quadrants. C: radial bias index for areas V1–V3 for cardinal and oblique stimuli. Radial bias is defined as the correlation between the spatial distribution of orientation preference (A and B) and the radial bias map for each pair of orientations (polar sinusoid pattern shown in inset circular color maps in A and B; see materials and methods for details). In all three areas, the average radial bias index was significantly greater than predicted by chance (t-test, P < 0.02, FDR corrected for multiple comparisons). Error bars represent SE of the mean across subjects. D: cortical flat maps from the left (LH) and right hemisphere (RH) of a representative subject (S2) showing the distribution of relative orientation preference for oblique orientation stimuli in visual cortical areas V1–hV4. Color coded as in B. Voxels preferring rightward oblique orientation (45°) predominate in the lower hemifield representations of V1–V3 in the right hemisphere and upper hemifield representations in the left hemisphere, corresponding to the spatial distribution of orientation preference shown in B; voxels preferring the orthogonal orientation predominate in the other two quadrants.

Fig. 3.

Spatial distribution of eye preference measured behaviorally and by fMRI. A: spatial distribution of perceptual eye dominance within 6° eccentricity for each subject. Color indicates relative preference for right or left eye stimulation (measured at 25 different visual field locations as the proportion of stimulus duration dominated by the corresponding eye; see materials and methods for details). Dotted lines indicate inner and outer boundaries of stimuli used in fMRI experiments. B: spatial distribution of eye preference measured by fMRI for areas V1–V3 for each subject. Color represents t values indicating relative preference for right vs. left eye stimulation (inset: t color map) for each voxel; size of plot symbols indicates goodness of fit (R²) of linear model fit to each voxel time series. Dotted lines indicate inner and outer boundaries of stimuli. Numbers and asterisks next to each plot indicate strength and significance (*P < 0.05; **P < 0.01; ***P < 0.001) of correlation with perceptual eye dominance in A (see text for details). C: nasotemporal eye preference in V1-V3. Height of bars shows average eye preference across subjects (t value of contrast between right and left eye stimulation averaged across orientations) for right hemifield (RH; black bars) and left hemifield (LH; white bars) voxels (corresponding to left and right hemispheres, respectively). Numbers at right of each plot indicate the proportion of subjects showing significant contralateral preference (resampling test, P < 0.05, FDR corrected for multiple comparisons). Error bars represent SE of the mean across subjects.

Fig. 4.

Effect of binning by visual polar angle and eccentricity on classification performance for decoding orientation and eye of origin in areas V1–V3. A: classification performance (proportion reduction in error) for decoding cardinal stimulus orientation (0° or 90°) in area V1 as a function of number of bins. Filled symbols represent binning of voxels by polar angle; open symbols represent binning of voxels randomly. Decoding performance is significantly higher for binning by polar angle than random binning, consistent with a radial bias in orientation preference. Error bars represent SE of the mean across subjects. B and C: threshold performance (log₂ number of bins) for decoding orientation in V1–V3 for cardinal and oblique orientation stimuli for binning by polar angle (filled bars) and random binning (open bars). Decoding performance is significantly higher for binning by polar angle than random binning in all 3 areas (in V2 and V3 for oblique orientations only), as predicted by radial bias for orientation in these areas. *P < 0.05; **P < 0.01; ***P < 0.001. Error bars indicate 68% confidence intervals estimated by a bootstrapping procedure (see materials and methods). D: same as A, but for decoding eye of origin. Binning by polar angle does not improve decoding performance compared with random binning in V1, suggesting eye preference (unlike orientation preference) is not organized in a large-scale radial pattern. E: effect of binning by eccentricity on orientation decoding. Binning by eccentricity does not improve decoding performance relative to random binning, indicating that orientation preference does not show a large-scale eccentricity bias. F: same as E, but for decoding eye of origin. Binning by eccentricity does not improve performance for decoding eye of origin relative to random binning, suggesting that the distribution of eye preference is not systematically related to eccentricity.

Fig. 5.

Effect of binning by visual hemifield on classification performance for decoding orientation and eye of origin in V1–V3. Plot symbols, error bars, and conventions are as defined in Fig. 4. A–C: binning by hemifield significantly improves classification performance for decoding orientation in all areas, except V3 for cardinal orientations. D–F: binning by visual hemifield significantly improves decoding performance for eye of origin compared with random binning for both stimulus orientations in V1, and for at least one orientation in V2 and V3, consistent with a large-scale left-right hemifield organization in eye preference.

Fig. 6.

Effect of regressing out large-scale spatial patterns on decoding performance.as function of voxel inclusion threshold (R²). A–C: regressing out angular position significantly reduces decoding performance for orientation in V1–V3. Classification performance (proportion reduction in error) for decoding stimulus orientation (averaged across cardinal and oblique orientations) in areas V1–V3 as a function of voxel inclusion threshold (R²). Filled symbols represent decoding performance computed on data with polar angle component removed; open symbols represent decoding performance on original data. P values indicate significance of difference between thresholds (computed using a resampling procedure; see materials and methods). Error bars represent SE of the mean across subjects. D–F: regressing out visual hemifield significantly reduces decoding performance for eye of origin in V1 and V2, but not V3. Plot symbols, error bars, and conventions are as defined in A–C.

Fig. 7.

Effect of spatial filtering of voxel responses on decoding performance. A–C: decoding performance for orientation in V1–V3 as a function of low-pass (filled symbols) and high-pass (open symbols) filter size. Error bars represent SE of the mean across subjects. D–F: same as A–C but for decoding eye of origin.

Fig. 8.

Orientation and eye preference patterns in V2 and V3 are significantly correlated with patterns of V1 stimulus preference. A and B: eye and orientation preference in V2 as a function of eye preference in V1. Each plot symbol corresponds to the stimulus preference (t value of contrast between right and left eye stimulation, data collapsed across stimulus orientations) of a single voxel in V2 plotted against V1 stimulus preference at the corresponding visual field location. Different plot symbols represent different subjects; size of each plot symbol indicates goodness of fit of voxel time series (R²). The spatial patterns of both eye and orientation preferences in V2 are significantly correlated with V1 eye and orientation preference patterns, respectively. C: average correlation (Pearson) between V1 and V2/V3 stimulus preference patterns. Error bars represent SE of the mean across subjects. D and E: binning by V1 eye preference significantly improves decoding performance for eye of origin compared with random binning in V2 and V3. F: binning by V1 orientation preference significantly improves decoding performance for orientation compared with random binning in V2 and V3. **P < 0.01; ***P < 0.001.