Optimal inference of sameness

Abstract

Deciding whether a set of objects are the same or different is a cornerstone of perception and cognition. Surprisingly, no principled quantitative model of sameness judgment exists. We tested whether human sameness judgment under sensory noise can be modeled as a form of probabilistically optimal inference. An optimal observer would compare the reliability-weighted variance of the sensory measurements with a set size-dependent criterion. We conducted two experiments, in which we varied set size and individual stimulus reliabilities. We found that the optimal-observer model accurately describes human behavior, outperforms plausible alternatives in a rigorous model comparison, and accounts for three key findings in the animal cognition literature. Our results provide a normative footing for the study of sameness judgment and indicate that the notion of perception as near-optimal inference extends to abstract relations.

Fig. 1.

Experimental procedure and psychometric curves. Error bars indicate SEM over subjects. (A) Subjects reported through a key press whether the orientations of the ellipses were identical. (B) Experimental conditions. In experiment 1, set size was 2, 4, or 8. In experiment 2, set size was 6 and stimuli all had high reliability (HIGH), all low reliability (LOW), or were mixed (MIXED). (C) Proportion of “different” responses in experiment 1 as a function of set size. (D) Proportion of “different” responses in experiment 1 as a function of the SD of the presented orientations. All “same” trials have an SD of 0.

Fig. 2.

Statistical structure of the task and a geometrical interpretation of the optimal decision rule. (A) Each node represents a random variable, each arrow a conditional probability distribution. This diagram specifies the distribution of measurements, x. The optimal observer “inverts” the generative model and computes the probability of C given x. (B) Geometrical interpretation of the optimal decision rule at N = 3. The axes represent the observed stimulus orientations, x. Each dot represents the set of measurements on a single trial. On “same” trials (red), the dots lie—on average—closer to the diagonal than on “different” trials (blue). The optimal strategy is to respond “same” when x lies within the green cylinder, whose axis is the diagonal.

Fig. 3.

Comparison of models in experiment 1. Circles and error bars represent mean and SEM of subject data. Shaded areas represent SEM of model fits. (A) Proportion “different” responses as a function of sample SD for optimal and suboptimal models. (B) Bayesian model comparison. Each bar represent the log likelihood of the optimal model minus that of a suboptimal model. (C) The decision criteria (Left) and the internal noise levels (Right) for the best-fitting BC model are nearly identical to those of the best-fitting optimal-observer model. This suggests that the human criteria are close to optimal.

Fig. 4.

Comparison of models in experiment 2. Circles and error bars represent mean and SEM of subject data. Shaded areas represent SEM of model fits. (A) Proportion “different” responses as a function of sample SD for optimal and suboptimal models. (B) Proportion of “different” responses as a function of the number of high-reliability stimuli in a display (N_high). In both BC models, LOW (N_high = 0) and HIGH (N_high = 6) were fitted separately. (C) Decision criterion in the optimal model as a function of N_high. (D) Bayesian model comparison. Each bar represents the log likelihood of the optimal model minus that of a suboptimal model.

Fig. 5.

Comparison between sameness judgment in animals and the optimal observer. (A) Upper: Data from Young and Wasserman (24). The proportion of “different” responses in pigeons correlates strongly with the entropy of the stimulus set. Lower: Results from an optimal-observer simulation. (B) Upper: Data replotted from Wasserman, Young, and Fagot (16). Thin lines represent individual experiments, the thick line their mean. Performance of baboons increases with set size. Lower: Results from an optimal-observer simulation. (C) Upper: Data from Wasserman and Young (28). Increasing the amount of blur in the stimulus set results in more “same” responses on “different” trials, but leaves the responses on “same” trials largely unaffected. Lower: Black, results from an optimal-observer simulation with a prior p_same = 0.6 and a guessing rate of 0.25. Gray, same with p_same = 0.5.