A perceptual space of local image statistics

Abstract

Local image statistics are important for visual analysis of textures, surfaces, and form. There are many kinds of local statistics, including those that capture luminance distributions, spatial contrast, oriented segments, and corners. While sensitivity to each of these kinds of statistics have been well-studied, much less is known about visual processing when multiple kinds of statistics are relevant, in large part because the dimensionality of the problem is high and different kinds of statistics interact. To approach this problem, we focused on binary images on a square lattice - a reduced set of stimuli which nevertheless taps many kinds of local statistics. In this 10-parameter space, we determined psychophysical thresholds to each kind of statistic (16 observers) and all of their pairwise combinations (4 observers). Sensitivities and isodiscrimination contours were consistent across observers. Isodiscrimination contours were elliptical, implying a quadratic interaction rule, which in turn determined ellipsoidal isodiscrimination surfaces in the full 10-dimensional space, and made predictions for sensitivities to complex combinations of statistics. These predictions, including the prediction of a combination of statistics that was metameric to random, were verified experimentally. Finally, check size had only a mild effect on sensitivities over the range from 2.8 to 14min, but sensitivities to second- and higher-order statistics was substantially lower at 1.4min. In sum, local image statistics form a perceptual space that is highly stereotyped across observers, in which different kinds of statistics interact according to simple rules.

Keywords: Intermediate vision; Local features; Metamers; Multipoint correlations; Visual textures.

Fig. 1

(A) The gamuts of the 10 image statistics. In each case, the coordinate value (vertical scale) indicates the correlation strength, which can range from −1 to 1. (B) Timecourse of a typical trial. Stimuli were typically presented for 120 ms, followed by a random mask. (C) The stimulus was a 64 × 64 array of checks that contained a 16 × 64 target positioned 8 checks away from one of the four edges. Stimuli were of two types: top, random background with structured target (here, θ_┘ = 1), or bottom, structured background with random target. Red rectangles indicate the target, and were not visible during the trials. Panels B and C reproduced, with permission of the copyright holder, The Association for Research in Vision and Ophthalmology, from (Victor, Thengone, & Conte, 2013). (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Fig. 2

Psychometric functions along five kinds of coordinates: first-order (γ), cardinal second-order (β_{_}), diagonal second-order (β_/), third-order (θ_┘), and fourth-order (α). For each coordinate, psychometric functions are shown for negative excursions (left element of each pair) and positive excursions (right element of each pair). Chance performance is 0.25; error bars indicate 95% confidence limits based on binomial statistics. The patches above the psychometric functions show typical 32 × 32 samples of images along each coordinate axis, constructed with γ = ±0.2, β_{_} = ±0.4, β_/ = ±0.4, θ_┘= ±0.8, and α = ±0.8. Subjects MC, DT, JD, DF. Data for subjects MC and DT for β_{_}, β_/, and θ_┘ reproduced, with permission of the copyright holder, The Association for Research in Vision and Ophthalmology, from (Victor, Thengone, & Conte, 2013).

Fig. 3

Thresholds for positive and negative excursions for the five kinds of coordinates shown in Fig. 2, for N = 16 subjects. All subjects had the same rank-order of thresholds. γ < β_{_} < β_\ < θ_┘ < α.

Fig. 4

Isodiscrimination contours in the 15 coordinate planes. Black: locus corresponding to a fraction correct of 0.625, halfway between chance and perfect. Blue: 95% confidence limits via bootstrap. Red: isodiscrimination contours for the ellipsoidal model fitted to each subject. The 15 coordinate planes include, up to symmetry, all pairwise combinations of the 10 coordinates {γ, β_{_}, β_|, β_\, β_/, θ_┘, θ_└, θ_┌, θ_┐, α}. Subjects MC, JD, DF, DT. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

Fig. 5

(A) Eigenvalues of the sensitivity matrix Q for each subject of Fig. 4 (colored bars) and the average across these subjects (gray bar); each eigenvalue is normalized by dividing by the subject’s largest eigenvalue (sym1). Error bars indicate 95% confidence limits, via parametric bootstrap of the sensitivity measurements. The pie graphs show the fractional contribution of each order of statistic to the eigenvectors. Eigenvectors are named according to symmetry class (see Eigenvector Classes in Methods), and, within each symmetry class (sym, hvi, dii, and rot), are labeled in order of decreasing eigenvalue. Texture samples are the eigenvectors of the averaged Q-matrix at distances of 0.18 (sym1 and sym2) and 0.36 (other eigenvectors) from the origin. (B) Dot-products of the eigenvectors of the Q-matrices for each of the four subjects of Fig. 4, with the eigenvectors of the average Q-matrix of the other three subjects. Dot-products are not shown for the last three eigenvectors, since symmetry considerations force these values to 1. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

Fig. 6

Measured and predicted sensitivities along the eigenvectors (Table 3) and in the Minkowski directions (Table 1) for four subjects. Thresholds are measured for positive (filled symbols) and negative (open symbols) directions, where the sign is defined relative to the eigenvectors tabulated in Table 3 and the Minkowski directions in Table 1. The prediction of the ellipsoidal model is necessarily the same for both directions, and is shown in gray. Error bars show 95% confidence limits via bootstrap. Texture samples are at distances of 0.18 for sym1 and sym2, 0.36 for the other eigenvectors, and 0.3 for the Minkowski directions.

Fig. 7

Sensitivity as a function of check size for the five kinds of coordinates: first-order (γ), cardinal second-order (β_{_}), diagonal second-order (β_\), third-order (θ_┘), and fourth-order (α). N = 6 subjects for γ, β_{_}, θ_┘, and α; N = 5 for β_\ (all but DT). For α, sensitivities for positive and negative excursions are plotted separately as α₊ and α₋, since these were systematically different across subjects. Subjects MC, SR, KP, SP, DT, RS.

Fig. 8

Isodiscrimination contours as a function of check size for three pairwise combinations of the image statistics, in N = 4 subjects. Heavy lines indicate the loci corresponding to a fraction correct of 0.625, halfway between chance and perfect, for each check size. Thin lines indicate 95% confidence limits via bootstrap, and are omitted where some bootstrap resamplings had thresholds greater than 1. Subjects MC, SR, KP, and SP.

Fig. 9

Thresholds as a function of stimulus duration in planes sampling pairwise combinations of the image statistics ((γ, α) and (θ_┘, θ_└)-plane, 50–180 ms; (β_\, β_/)-plane, 30–120 ms). Each plot shows threshold vs. time in one direction of the indicated coordinate plane. Red trace: along the first coordinate; green trace: along the second coordinate; blue trace: along the diagonal in which both coordinates have the same sign; brown: along the diagonal in which coordinates have opposite signs. Subjects MC, DT, DF. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)