Recurrent predictive coding models for associative memory employing covariance learning.

Tang M
Salvatori T
Millidge B
Song Y
Lukasiewicz T
Bogacz R

The hippocampus is a brain region involved in learning and memory. Its role in memory is thought to be based on brain cells predicting the activity of other brain cells, a process called predictive coding. We propose a computer model for the activity of the hippocampus that uses predictive coding for the storage and retrieval of images. This demonstrates how memory based on predictive coding can be achieved in the hippocampus.

Scientific Abstract

The computational principles adopted by the hippocampus in associative memory (AM) tasks have been one of the most studied topics in computational and theoretical neuroscience. Recent theories suggested that AM and the predictive activities of the hippocampus could be described within a unitary account, and that predictive coding underlies the computations supporting AM in the hippocampus. Following this theory, a computational model based on classical hierarchical predictive networks was proposed and was shown to perform well in various AM tasks. However, this fully hierarchical model did not incorporate recurrent connections, an architectural component of the CA3 region of the hippocampus that is crucial for AM. This makes the structure of the model inconsistent with the known connectivity of CA3 and classical recurrent models such as Hopfield Networks, which learn the covariance of inputs through their recurrent connections to perform AM. Earlier PC models that learn the covariance information of inputs explicitly via recurrent connections seem to be a solution to these issues. Here, we show that although these models can perform AM, they do it in an implausible and numerically unstable way. Instead, we propose alternatives to these earlier covariance-learning predictive coding networks, which learn the covariance information implicitly and plausibly, and can use dendritic structures to encode prediction errors. We show analytically that our proposed models are perfectly equivalent to the earlier predictive coding model learning covariance explicitly, and encounter no numerical issues when performing AM tasks in practice. We further show that our models can be combined with hierarchical predictive coding networks to model the hippocampo-neocortical interactions. Our models provide a biologically plausible approach to modelling the hippocampal network, pointing to a potential computational mechanism during hippocampal memory formation and recall, which employs both predictive coding and covariance learning based on the recurrent network structure of the hippocampus.

Four rectangles on top of each other showing different layers of a network
Our predictive coding network can model the recurrent hippocampal network and its interactions with the neocortical hierarchy in memory tasks
Citation

2023. PLoS Comput Biol, 19(4)e1010719.

DOI
10.1371/journal.pcbi.1010719
Related Content
Publication
Salvatori T, Song Y, Hong Y, Sha L, Frieder S, Xu Z, Bogacz R, Lukasiewicz T
2021. Adv. Neural Inf. Process Syst., 34:3874-3886.
Publication
Koolschijn RS, Shpektor A, Clarke WT, Ip IB, Dupret D, Emir UE, Barron HC

2021. eLife, 10:e70071

Publication
Barron HC, Auksztulewicz R, Friston KJ
2020. Prog. Neurobiol., 192:101821.
Publication
Tang M, Barron HC, Bogacz R
Publication
Salvatori T, Song Y, Yordanov Y, Millidge B, Sha L, Emde C, Xu Z, Bogacz R, Lukasiewicz T
2024. The Twelveth International Conference on Learning Representations