Manipulating the revision of reward value during the intertrial interval increases sign tracking and dopamine release

Abstract

Recent computational models of sign tracking (ST) and goal tracking (GT) have accounted for observations that dopamine (DA) is not necessary for all forms of learning and have provided a set of predictions to further their validity. Among these, a central prediction is that manipulating the intertrial interval (ITI) during autoshaping should change the relative ST-GT proportion as well as DA phasic responses. Here, we tested these predictions and found that lengthening the ITI increased ST, i.e., behavioral engagement with conditioned stimuli (CS) and cue-induced phasic DA release. Importantly, DA release was also present at the time of reward delivery, even after learning, and DA release was correlated with time spent in the food cup during the ITI. During conditioning with shorter ITIs, GT was prominent (i.e., engagement with food cup), and DA release responded to the CS while being absent at the time of reward delivery after learning. Hence, shorter ITIs restored the classical DA reward prediction error (RPE) pattern. These results validate the computational hypotheses, opening new perspectives on the understanding of individual differences in Pavlovian conditioning and DA signaling.

Fig 1. Sign tracking is more prominent during sessions with 120-s ITIs.

(A) Response bias, which is (Lever Presses − Food Cup Entries) / (Lever Presses + Food Cup Entries). (B) The probability difference, which is (P_lever − P_receptacle). (C) Latency index, which is ($\overline{\mathrm{x}}$ Cup Entry Latency − $\overline{\mathrm{x}}$ Lever Press Latency) / 8. (D) PCA index = average of response bias, probability difference, and latency difference indices described in A–C. All of these ratios range from −1.0 to +1.0 and are more positive and negative for animals that sign track and goal track, respectively. All behavioral indices are derived from sessions during which DA was recorded (60-s ITI groups = 7 rats; 120-s ITI group = 12 rats) and used behavior during the entire 8-s CS epoch. Each of the above distributions was computed by session. Supporting material reports results by rat. (E-F) PCA index computed using just the first 4 s (E) and last 4 s (F) of the CS period;120-s ITI group = red; 60-s ITI group = blue. (G) Average beam break (solid) and lever press (dashed) rate for 120-s (red) and 60-s (blue) ITI sessions. (H) Average lever press rate for 120-s (red) and 60-s (blue) ITI sessions. Data are the same as in G but with a smaller scale so that differences and timing can be better visualized. Error bars represent SEM. (I) Green lines are the difference between solid blue and solid red lines from “G” (food cup entries for the 120-s ITI group minus food cup entries for the 60-s ITI group) during the cue period. Thus, negative deflections illustrate more food cup entries during sessions with a 60-s ITI. Orange lines represent the differences between the 120-s ITI group lever pressing and the 60-s ITI group lever pressing (i.e., red dashed minus blue dashed from “H”). Thus, positive deflections represent time during the cue period when rats in the 120-s ITI group lever pressed more than those in the 60-s ITI group. Orange and green tick marks represent 100-ms bins in which there was a significant difference between 120-s and 60-s ITI groups (t test; p < 0.05). Percentages for each session were computed and then averaged across sessions. Underlying data for Fig 1 can be found in S1 Data. For analysis averaged within session and rat, and then averaged across rats, please see S2 Fig. CS, conditioned stimulus; DA, dopamine; GT, goal tracking; ITI, intertrial interval; P_lever, probability of pressing lever; P_receptacle, probability of entering the receptacle; PCA, Pavlovian Conditioned Approach; ST, sign tracking; US, unconditioned stimulus.

Fig 2. DA release to the CS and US is stronger during sessions with 120-s ITIs.

(A-B) Average DA release over time for 120-s (red) and 60-s (blue) ITI groups sessions. “A” represents DA release averaged within the session first and then across sessions, whereas DA release in “B” represents averages taken within each session and rat, and then averaged across rats. Error bars represent SEM, with “n” being session and rat for “A” and “B,” respectively. (C-D) Average DA release during the US period (y-axis) relative to time the photobeam in the food cup was broken during the ITI for the 120-s (red) and 60-s (blue) ITI groups for each session. (E-F) Average DA release during the CS period (y-axis) relative to time the photobeam in the food cup was broken during the ITI for the 120-s (red) and 60-s (blue) ITI groups. Underlying data for Fig 2 can be found in S2 Data. CS, conditioned stimulus; DA, dopamine; ITI, intertrial interval; US, unconditioned stimulus.

Fig 3. Development of sign tracking and DA signals over training.

(A-B) Average beam break (solid) and lever press (dashed) rate for 120-s (A) and 60-s (B) ITI sessions. (C-D) Average lever press rate for 120-s (C) and 60-s (D) ITI sessions. Percentages for each session were computed and then averaged across sessions. Data are the same as in A and B but with a smaller scale so that differences and timing can be better visualized. (E-F) Average DA release over time for 120-s (E) and 60-s (F) ITI sessions. In each of the above (A-F), data are broken down into averages from sessions 1–3 (pale colors, pink [120 s] and turquoise [60s]) and sessions 4–10 (dark colors, red [120 s] and blue [60 s]); 60-s ITI group = 7 rats; 120-s ITI group = 12 rats. DA release for each session was computed and then averaged across sessions. For analysis averaged within session and rat and then averaged across rat, please see S2 Fig. (G-H) Average DA release over time for each of the 10 sessions for the 120-s ITI group (G) and the 60-s ITI group (H). For breakdown of behavior by session for each of the 10 sessions, please see S4 Fig. (I-J) Average DA release over time for 120-s (I) and 60-s (J) ITI groups broken down into sessions in which rats either lever pressed (120-s ITI = 4 rats, 20 sessions; 60-s ITI = 2 rats, 7 sessions) or entered the food cup during the CS (120-s ITI = 3 rats, 13 sessions; 60-s ITI = 4 rats, 18 sessions). Error bars represent SEM. Underlying data for Fig 3 can be found in S3 Data. CS, conditioned stimulus; DA, dopamine; GT, goal tracking; ITI, intertrial interval; ST, sign tracking; US, unconditioned stimulus.