An association between prediction errors and risk-seeking: Theory and behavioral evidence.

Moeller M
Grohn J
Manohar S
Bogacz R

Unexpected rewards lead to release of the chemical messenger dopamine in the brain. Interestingly, dopamine is also known control our willingness to take risks. We theorized that, since pleasant surprises cause dopamine release, they might change risk preferences as well. In this study, we tested this idea and discovered that participants are more likely to make a risky choice just after they experienced an outcome that was better than expected.

Scientific Abstract

Reward prediction errors (RPEs) and risk preferences have two things in common: both can shape decision making behavior, and both are commonly associated with dopamine. RPEs drive value learning and are thought to be represented in the phasic release of striatal dopamine. Risk preferences bias choices towards or away from uncertainty; they can be manipulated with drugs that target the dopaminergic system. Based on the common neural substrate, we hypothesize that RPEs and risk preferences are linked on the level of behavior as well. Here, we develop this hypothesis theoretically and test it empirically. First, we apply a recent theory of learning in the basal ganglia to predict how RPEs influence risk preferences. We find that positive RPEs should cause increased risk-seeking, while negative RPEs should cause risk-aversion. We then test our behavioral predictions using a novel bandit task in which value and risk vary independently across options. Critically, conditions are included where options vary in risk but are matched for value. We find that our prediction was correct: participants become more risk-seeking if choices are preceded by positive RPEs, and more risk-averse if choices are preceded by negative RPEs. These findings cannot be explained by other known effects, such as nonlinear utility curves or dynamic learning rates.

Reward prediction error enhances emphasis on potential gains
Reward prediction error enhances emphasis on potential gains. Basal ganglia include two main pathways which facilitate (Go) and inhibit (NoGo) movements. It has been proposed that they encode potential gains and costs connected with actions, respectively. Dopamine activates Go pathway and inhibits the NoGo pathway. Therefore, we hypothesized that positive rewards prediction error should enhance processing of information about potential gains, suppress information about possible costs, and hence lead to a higher probability of risky choices.
2021. PLoS Comput. Biol., 17(7):e1009213.
Malhotra G, Leslie DS, Ludwig CJ, Bogacz R
2018.Psychon Bull Rev, 25(3):971-996.
Bogacz R
2020. eLife, 9:e53262
Roberts BM, Doig NM, Brimblecombe KR, Lopes EF, Siddorn RE, Threlfell S, Connor-Robson N, Bengoa-Vergniory N, Pasternack N, Wade-Martins R, Magill PJ, Cragg SJ
2020. Nat. Commun., 11(1):4958.
Leimbach F, Georgiev D, Litvak V, Antoniades C, Limousin P, Jahanshahi M, Bogacz R
2018.J Cogn Neurosci, 30(6):876-884.
Mikhael JG, Bogacz R

2016.PLoS Comput. Biol., 12(9):e1005062.