Learning the payoffs and costs of actions.

Möller M
Bogacz R

How do we decide what to do in any given situation? We need to work out the payoffs and costs of actions – what we will gain and what it might cost us. But what is the exact nature of the mechanism in the brain which weighs up these payoffs and costs? (Continue reading accessible summary.)

Video Abstract
Scientific Abstract

A set of sub-cortical nuclei called basal ganglia is critical for learning the values of actions. The basal ganglia include two pathways, which have been associated with approach and avoid behavior respectively and are differentially modulated by dopamine projections from the midbrain. Inspired by the influential opponent actor learning model, we demonstrate that, under certain circumstances, these pathways may represent learned estimates of the positive and negative consequences (payoffs and costs) of individual actions. In the model, the level of dopamine activity encodes the motivational state and controls to what extent payoffs and costs enter the overall evaluation of actions. We show that a set of previously proposed plasticity rules is suitable to extract payoffs and costs from a prediction error signal if they occur at different moments in time. For those plasticity rules, successful learning requires differential effects of positive and negative outcome prediction errors on the two pathways and a weak decay of synaptic weights over trials. We also confirm through simulations that the model reproduces drug-induced changes of willingness to work, as observed in classical experiments with the D2-antagonist haloperidol.

Basal ganglia as a scale: weighing payoffs against costs
Basal ganglia as a scale: weighing payoffs against costs
Citation
2019. PLoS Comput. Biol., 15(2):e1006285.
Related Content
News
Image
A woman, a man and another woman stand facing the camera, the man in the middle holds a metal award- a sculpture of two film strips in a double helix.

Rob Key (middle; representing Oxford Sparks) and Demi Brizee (right; representing the MRC Brain Network Dynamics Unit) receive the award for Best Film Produced by Universities and Research Institutes.

Photo credit: Manuel Castells Clemente / #LabMeCrazy! International Film Festival

News
Image
Portrait photos of Charlotte Collingwood (left) and Vivienne Collins (right)

Charlotte Collingwood and Dr Vivienne Collins, co-winners of the 2022 Director’s Award for Public Engagement.