Subcortical Substrates of Explore-Exploit Decisions in Primates

Neuron. 2019 Aug 7;103(3):533-545.e5. doi: 10.1016/j.neuron.2019.05.017. Epub 2019 Jun 10.

Abstract

The explore-exploit dilemma refers to the challenge of deciding when to forego immediate rewards and explore new opportunities that could lead to greater rewards in the future. While motivational neural circuits facilitate learning based on past choices and outcomes, it is unclear whether they also support computations relevant for deciding when to explore. We recorded neural activity in the amygdala and ventral striatum of rhesus macaques as they solved a task that required them to balance novelty-driven exploration with exploitation of what they had already learned. Using a partially observable Markov decision process (POMDP) model to quantify explore-exploit trade-offs, we identified that the ventral striatum and amygdala differ in how they represent the immediate value of exploitative choices and the future value of exploratory choices. These findings show that subcortical motivational circuits are important in guiding explore-exploit decisions.

Publication types

  • Research Support, N.I.H., Intramural

MeSH terms

  • Amygdala / physiology*
  • Animals
  • Choice Behavior / physiology*
  • Computer Simulation
  • Conditioning, Operant / physiology
  • Decision Making / physiology*
  • Delay Discounting / physiology*
  • Exploratory Behavior / physiology*
  • Macaca mulatta
  • Male
  • Markov Chains
  • Models, Neurological
  • Motivation / physiology*
  • Reward
  • Ventral Striatum / physiology*