Economic irrationality is optimal during noisy decision making

Abstract

According to normative theories, reward-maximizing agents should have consistent preferences. Thus, when faced with alternatives A, B, and C, an individual preferring A to B and B to C should prefer A to C. However, it has been widely argued that humans can incur losses by violating this axiom of transitivity, despite strong evolutionary pressure for reward-maximizing choices. Here, adopting a biologically plausible computational framework, we show that intransitive (and thus economically irrational) choices paradoxically improve accuracy (and subsequent economic rewards) when decision formation is corrupted by internal neural noise. Over three experiments, we show that humans accumulate evidence over time using a "selective integration" policy that discards information about alternatives with momentarily lower value. This policy predicts violations of the axiom of transitivity when three equally valued alternatives differ circularly in their number of winning samples. We confirm this prediction in a fourth experiment reporting significant violations of weak stochastic transitivity in human observers. Crucially, we show that relying on selective integration protects choices against "late" noise that otherwise corrupts decision formation beyond the sensory stage. Indeed, we report that individuals with higher late noise relied more strongly on selective integration. These findings suggest that violations of rational choice theory reflect adaptive computations that have evolved in response to irreducible noise during neural information processing.

Keywords: choice optimality; decision making; evidence accumulation; irrationality; selective integration.

Fig. 1.

Selective integration and intransitivity. (A) Schematic of the selective integration model. On each time step, the values of two different alternatives on a single attribute are considered. Input samples (I_A, I_B), corresponding to attribute values, feed to a bottleneck that discounts the gain of the weakest sample (via selective gating, w) before relaying the inputs to the accumulators (Y_A, Y_B). Noise can arise both at the input (σ) and accumulation levels (ξ). (B) Choice probability for different values of w and for σ = ξ = 2, for pairwise comparisons between three equally valued multiattribute alternatives (table). A→B: A wins in more samples than B. WST is violated for w > 0 [i.e., P(A|{A,B}) > 0.5, P(B|{B,C}) > 0.5, P(A|{A,C}) < 0.5, with P(X|{X,Y}) denoting the probability of choosing X over Y].

Fig. 2.

Behavioral task and selective integration. (A) Trial schematic in experiment 1. Participants (n = 28) viewed two streams of bars and had to choose which stream was overall highest. (B) Mean choices in cyclic trials in experiment 1 (Fig. 1B) revealed a frequent-winner effect: A was chosen more often over B, B over C, and C over A. (C) Accuracy in increment trials, where the samples of either the frequently winning or the frequently losing stream were increased by a constant (A+ vs. B and A vs. B+, respectively). The difference in accuracy between the A+ vs. B and the A vs. B+ trials also revealed a frequent-winner effect. (D) The frequent-winner effect in cyclic ([P(A|{A,B})+P(B|{B, C})+P(C|{A,C})]/3–0.5) and increment trials [P(A₊|{A₊,B}) – P(B₊|{A,B₊})] correlated positively to each other. Filled circles correspond to different participants. Grey curve is the linear regression line. Error bars are 2 SEM. ***P < 0.001.

Fig. S1.

Recency effect and model comparison. (A) A logistic regression assessed the impact of the temporal position of each pair on choice and revealed a recency effect in experiments 1–4. This recency effect necessitated the use of a leak parameter in the models (Methods). (B) Percentage of participants that a given model offered the lowest BIC in experiment 1. Selective integration with late noise only (labeled L; i.e., omitting early noise) scored better. Majority of confirming dimensions (MCD) variants had much higher BIC scores than selective integration variants and consequently failed to score the lowest BIC in any participant. (C) In experiments 2–4, the variant of selective integration with late noise only (L) also scored lower BIC values in more participants compared with the other variants (E: early noise only; F: full model with early and late noise). (D) This three-parameter model that omits early noise is consistent with “late-noise dominance” in the full model (F), which included both early and late noise (mean ± SE of late- vs. early-noise parameter values in the full model across all four experiments: 14.38 ± 1.91 vs. 5.59 ± 0.58). Error bars correspond to 2 SEM. ***P < 0.001.

Fig. S2.

Behavioral results in experiments 2–4. (A) Accuracy in increment and standard trials in experiment 2 (n = 17) where the presentation speed was manipulated within participants (experiment 2). (B) Accuracy in increment trials and standard trials in experiment 3 (n = 27), where the sequence length changed randomly within an experimental block (experiment 3). (C) Time course of a trial in experiment 4 (n = 21). The colors–dimensions mapping was randomized across participants (SI Methods). (D) Choices in cyclic trials in experiment 4 replicated the frequent-winner effect in experiment 1. WST was significantly violated in 11 out of 21 participants who chose A more often than B, B more often than C, and C more often than A (Table S5). (E) Participants in experiment 4 had above-chance accuracy in both increment conditions, and the frequent-winner effect of experiment 1 was replicated [$P\left(A_{+}|\left\{A_{+},B\right\}\right)>P\left(B_{+}|\left\{A,B_{+}\right\}\right)$]. Error bars correspond to 2 SEM. ***P < 0.001; **P < 0.010; *P < 0.050.

Fig. S3.

Intransitivity predictions of noisy leaky integrator. (A) Probability of choice in the cyclic trials obtained by simulating a noisy leaky integrator (in 5,000 sessions using input sequences as per experiment 4). Leak and noise varied within the range of the corresponding best-fitting parameters. When leak is present, intransitivity can be observed due to the randomization of the sequences and the recency bias. This pattern weakens and eventually dissipates in the presence of noise. (B) Predictions of the noisy leaky integrator (5,000 sessions) using the best-fitting noise and leak parameters per participant and after setting w = 0. Probability of choice on y axes is given for the first alternative in the corresponding binary set (i.e., A in {A, B}).

Fig. 3.

Selective integration and decision accuracy. (A) The input distributions in a typical two-alternative forced-choice scenario. The SD of the distributions (σ) corresponds to early noise. (B) Decision accuracy in the model for the scenario in A, as a function of w, for different levels of late noise (ξ; curves) and after 12 accumulation steps (t). Black circles indicate the value of w that maximizes accuracy for a given level of late noise. (C) Example input (Top) and single-trial accumulator states (Bottom) for lossless (Left) and selective integration (Right). The input parameters are as in A, and late noise was absent. (D) Bivariate end-state (t = 12) accumulator distributions for the choice problem in A, for lossless (Left) and selective integration (Right). (Top) ξ = 0. (Bottom) ξ = 15. Density to the Left of dashed diagonal corresponds to accuracy (in percentage). Higher density is depicted with red, and lower density with blue.

Fig. S4.

Optimal selective gating for early/late noise and parameter recovery in the model. (A) The optimal selective gating increases (brighter shades) as late noise increases (Fig. 3B) and decreases (darker shades) as early noise increases. The latter effect is due to the fact that, as early noise increases, the local winner becomes less and less predictive of the identity of the correct alternative. As a consequence, selective integration operates almost randomly and at a high cost, which is not balanced out in the absence of strong late noise. Simulation details are described in SI Methods. (B) Generative parameters plotted against recovered parameters for late noise and (C) selective gating in simulated datasets (SI Methods). Red curves are linear regression lines. ***P < 0.001.

Fig. 4.

Relationship between selective gating and late noise. (A) Estimated selective gating parameters for each individual (circles) in all four experiments (n = 93) plotted against estimated late noise parameters. (B) Same as A, but for simulated data (see Fig. S4 B and C and SI Methods). Gray curves are linear regression lines. ***P < 0.001.