Reward probability and timing uncertainty alter the effect of dorsal raphe serotonin neurons on patience

Abstract

Recent experiments have shown that optogenetic activation of serotonin neurons in the dorsal raphe nucleus (DRN) in mice enhances patience in waiting for future rewards. Here, we show that serotonin effect in promoting waiting is maximized by both high probability and high timing uncertainty of reward. Optogenetic activation of serotonergic neurons prolongs waiting time in no-reward trials in a task with 75% food reward probability, but not with 50 or 25% reward probabilities. Serotonin effect in promoting waiting increases when the timing of reward presentation becomes unpredictable. To coherently explain the experimental data, we propose a Bayesian decision model of waiting that assumes that serotonin neuron activation increases the prior probability or subjective confidence of reward delivery. The present data and modeling point to the possibility of a generalized role of serotonin in resolving trade-offs, not only between immediate and delayed rewards, but also between sensory evidence and subjective confidence.

Fig. 1

Schematic of the sequential tone-food waiting task. a Diagram of the test in which optogenetic stimulation was applied during the reward-delay period (experiments 1 and 2). b Time sequence of serotonin activation trials and serotonin no-activation trials. In serotonin activation trials, 0.8 s of blue light was delivered at the onset of the reward delay. In serotonin no-activation trials, 0.8 s of yellow light was applied at the onset of the reward delay. In each trial, 1 s of yellow light was used at the onset of food presentation or at the reward wait error. Blue and yellow bars denote blue and yellow light stimulation, respectively. Brown- and red-shaded regions denote tone- and reward-delay periods, respectively. Orange-shaded regions denote duration of tone presentation. c Locations of optical fibers in the DRN. Light blue bars in the DRN represent tracks of implanted optical fibers. Coronal drawings were adapted from ref. with permission

Fig. 2

Optogenetic activation of DRN serotonin neurons enhances waiting in 75% reward tests, but not in 25 or 50% reward tests. a Distribution of waiting time during omission trials in the 75% one-pellet test. b Distribution of waiting time during omission trials in the 75% two-pellet test. c Distribution of waiting time during omission trials in the 25% one-pellet test. d Distribution of waiting time during omission trials in the 25% three-pellet test. e Distribution of waiting time during omission trials in the 50% one-pellet test. f Distribution of waiting time during omission trials in the 50% three-pellet test. Orange circles illustrate the timing and numbers of food pellets presented in rewarded trials. White circles denote omission trials

Fig. 3

Average waiting time during omission trials in the 75, 50, and 25% reward tests. a Average waiting time in serotonin no-activation (yellow) and activation (blue) during the 75% one-pellet test for individual ChR2-expressing (blue thin lines) and WT (green thin lines) mice and for population of ChR2-expressing (blue line) and WT (green line) mice. b Average waiting time in serotonin no-activation (yellow) and activation (blue) during the 75% two-pellet test for individual ChR2-expressing mice (gray lines) and for population of mice (blue line). c Average waiting time in serotonin no-activation (yellow) and activation (blue) during the 25% one-pellet test for individual ChR2-expressing mice (gray lines) and for population of mice (blue line). d Average waiting time in serotonin no-activation (yellow) and activation (blue) during the 25% three-pellet test for individual ChR2-expressing mice (gray lines) and for population of mice (blue line). e Average waiting time in serotonin no-activation (yellow) and activation (blue) during the 50% one-pellet test for individual ChR2-expressing mice (gray lines) and for population of mice (blue line). f Average waiting time in serotonin no-activation (yellow) and activation (blue) during the 50% three-pellet test for individual ChR2-expressing mice (gray lines) and for population of mice (blue line). ***P < 0.001 by paired t-test. Error bars represent the s.e.m. In some case, the error bars are too small to be visible. n.s. not significant

Fig. 4

The role of serotonin in promoting patience is modulated by reward probability, but not by reward value. Waiting time ratios: 75% one-pellet test (1.13 ± 0.01, n = 57 tests from 6 mice), 75% two-pellet test (1.14 ± 0.02, n = 30 tests from 5 mice), 25% one-pellet test (1.00 ± 0.01, n = 38 tests from 6 mice), 25% three-pellet test (1.00 ± 0.01, n = 41 tests from 6 mice), 50% one-pellet test (1.03 ± 0.01, n = 21 tests from 3 mice), and 50% three-pellet test (1.04 ± 0.02, n = 19 tests from 3 mice). Waiting time ratios were significantly larger in 75% reward tests compared with tests having the same expected reward value. **P < 0.01, ***P < 0.001 by post hoc Bonferroni correction. n.s. not significant. Error bars represent the s.e.m.

Fig. 5

Optogenetic activation of DRN serotonin neurons enhances waiting for temporally uncertain rewards. a Distribution of waiting time during omission trials in the D6 test. b Distribution of waiting time during omission trials in the D4-6-8 test. c Distribution of waiting time during omission trials in the D2-6-10 test. d Distribution of waiting time during omission trials in the D10 test. Orange circles illustrate the timing and number of food pellets presented in rewarded trials. White circles denote omission trials

Fig. 6

The role of serotonin in promoting patience for future rewards with uncertain timing. a Average waiting time in serotonin no-activation (yellow) and activation (blue) during the D6 test for individual ChR2-expressing (blue thin lines) and WT (green thin lines) mice and for population of ChR2-expressing (blue line) and WT (green line) mice. b Average waiting time in serotonin no-activation (yellow) and activation (blue) during the D4-6-8 test for individual ChR2-expressing mice (gray lines) and for population of mice (blue line). c Average waiting time in serotonin no-activation (yellow) and activation (blue) during the D2-6-10 test for individual ChR2-expressing (blue thin lines) and WT (green thin lines) mice and for population of ChR2-expressing (blue line) and WT (green line) mice. d Average waiting time in serotonin no-activation (yellow) and activation (blue) during the D10 test for individual ChR2-expressing mice (gray lines) and for population of mice (blue line). ***P < 0.001 by paired t-test. Error bars represent the s.e.m. In some case, the error bars are too small to be visible. e Waiting time ratios in the 75% one-pellet tests in which food pellets were delivered with uncertain timing. The waiting time ratio in the D2-6-10 test was the largest among the five tests. The waiting time ratio in the D6 test was not significantly different from the waiting time ratios in the D3 test and in the D10 test. ***P < 0.001 by post hoc Bonferroni correction. Error bars present the s.e.m. n.s. not significant

Fig. 7

A Bayesian decision making model for waiting reproduces features of effects of reward probability and timing uncertainty on promotion of patience by serotonin. a Top panel: the model assumes that the subject has a probabilistic model of reward delivery timing (magenta line), which is assumed to be Gaussian with μ = 3 s and σ = 2 s in this example. As the time passes without reward delivery, the likelihood of a reward trial diminishes according to the cumulative density function (green line). Middle panel: the posterior probability for a reward trial goes down along with the likelihood, but persists longer if the prior probability for a reward trial is higher. Bottom panel: the timing of quitting is shifted later with a higher prior probability (Methods). b We assume that dorsal raphe serotonin neuron stimulation causes an overestimation of the prior probability when the reward probability is higher (p′ = p + p² − p³ in this example). The yellow and blue lines show the time of quitting without and with increased prior probability, respectively. The effect of serotonin neuron stimulation is largest with a reward probability p = 0.75 (top panel; μ = 3 s and σ = 2 s). c With a larger uncertainty σ of reward timing, the waiting time distribution shifts later and the effect of serotonin neuron stimulation (increase of prior probability from 0.75 to 0.95 in this example) increases. A shift in the average reward timing (bottom panel; μ = 10 s and σ = 3 s) does not cause a large increase in waiting time with serotonin neuron stimulation