The role of the basal ganglia in exploration in a neural model based on reinforcement learning

Int J Neural Syst. 2006 Apr;16(2):111-24. doi: 10.1142/S0129065706000548.

Abstract

We present a computational model of basal ganglia as a key player in exploratory behavior. The model describes exploration of a virtual rat in a simulated water pool experiment. The virtual rat is trained using a reward-based or reinforcement learning paradigm which requires units with stochastic behavior for exploration of the system's state space. We model the Subthalamic Nucleus-Globus Pallidus externa (STN-GPe) segment of the basal ganglia as a pair of neuronal layers with oscillatory dynamics, exhibiting a variety of dynamic regimes such as chaos, traveling waves and clustering. Invoking the property of chaotic systems to explore state-space, we suggest that the complex exploratory dynamics of STN-GPe system in conjunction with dopamine-based reward signaling from the Substantia Nigra pars compacta (SNc) present the two key ingredients of a reinforcement learning system.

MeSH terms

  • Animals
  • Basal Ganglia / physiology*
  • Computer Simulation*
  • Conditioning, Psychological
  • Dopamine / physiology
  • Exploratory Behavior / physiology*
  • Models, Neurological*
  • Nonlinear Dynamics
  • Rats
  • Reinforcement, Psychology
  • Reward
  • Stochastic Processes
  • User-Computer Interface

Substances

  • Dopamine