Integration of reinforcement learning and optimal decision-making theories of the basal ganglia.

Bogacz R
Larsen T
Scientific Abstract

This article seeks to integrate two sets of theories describing action selection in the basal ganglia: reinforcement learning theories describing learning which actions to select to maximize reward and decision-making theories proposing that the basal ganglia selects actions on the basis of sensory evidence accumulated in the cortex. In particular, we present a model that integrates the actor-critic model of reinforcement learning and a model assuming that the cortico-basal-ganglia circuit implements a statistically optimal decision-making procedure. The values of cortico-striatal weights required for optimal decision making in our model differ from those provided by standard reinforcement learning models. Nevertheless, we show that an actor-critic model converges to the weights required for optimal decision making when biologically realistic limits on synaptic weights are introduced. We also describe the model's predictions concerning reaction times and neural responses during learning, and we discuss directions required for further integration of reinforcement learning and optimal decision-making theories.

Citation

2011.Neural Comput, 23(4):817-51.

Related Content
Publication
Keuken MC, Van Maanen L, Bogacz R, Schäfer A, Neumann J, Turner R, Forstmann BU

2015.Hum Brain Mapp, 36(10):4041-4052.

Publication
Groschner LN, Chan Wah Hak L, Bogacz R, DasGupta S, Miesenböck G
2018. Cell, 173(4):894-905.
Publication
Mikhael JG, Bogacz R

2016.PLoS Comput. Biol., 12(9):e1005062.

Publication
Herz DM, Bange M, Gonzalez-Escamilla G, Auer M, Ashkan K, Fischer P, Tan H, Bogacz R, Muthuraman M, Groppa S, Brown P

2022. Nat Commun, 13(1):7530.

Publication
van Swieten MMH, Bogacz R
2020. PLoS Comput. Biol., 16(5):e1007465.