Learning on Arbitrary Graph Topologies via Predictive Coding

Salvatori T
Pinchetti L
Millidge B
Song Y
Bao T
Bogacz R
Lukasiewicz T

To solve distinct learning tasks, artificial neural networks typically need to have different architectures and be trained in different ways. This paper shows that a model of information processing in the brain can solve all this tasks within a single network.

Scientific Abstract

Training with backpropagation (BP) in standard deep learning consists of two main steps: a forward pass that maps a data point to its prediction, and a backward pass that propagates the error of this prediction back through the network. This process is highly effective when the goal is to minimize a specific objective function. However, it does not allow training on networks with cyclic or backward connections. This is an obstacle to reaching brain-like capabilities, as the highly complex heterarchical structure of the neural connections in the neocortex are potentially fundamental for its effectiveness. In this paper, we show how predictive coding (PC), a theory of information processing in the cortex, can be used to perform inference and learning on arbitrary graph topologies. We experimentally show how this formulation, called PC graphs, can be used to flexibly perform different tasks with the same network by simply stimulating specific neurons. This enables the model to be queried on stimuli with different structures, such as partial images, images with labels, or images without labels. We conclude by investigating how the topology of
the graph influences the final performance, and comparing against simple baselines trained with BP.

Different learning tasks
Examples of different tasks solved by a predictive coding network
2022. Advances in Neural Information Processing Systems, 35, 38232-38244
Related Content
Salvatori T, Song Y, Xu Z, Lukasiewicz T, Bogacz R
In Proceedings of the 36th AAAI Conference on Artificial Intelligence‚ AAAI 2022 ‚Vancouver, BC, Canada, February 22--March 1‚ 2022 (Vol. 10177, pp. 507-524). AAAI Press.
Richards BA, Lillicrap TP, Beaudoin P, Bengio Y, Bogacz R, Christensen A, Clopath C, Costa RP, de Berker A, Ganguli S, Gillon CJ, Hafner D, Kepecs A, Kriegeskorte N, Latham P, Lindsay GW, Miller KD, Naud R, Pack CC, Poirazi P, Roelfsema P, Sacramento J, Saxe A, Scellier B, Schapiro AC, Senn W, Wayne G, Yamins D, Zenke F, Zylberberg J, Therien D, Kording KP
2019. Nat. Neurosci., 22:1761-1770.
Song Y, Lukasiewicz T, Xu Z, Bogacz R
2020. Adv Neural Inf Process Syst, 33:22566-22579.
Salvatori T, Song Y, Hong Y, Sha L, Frieder S, Xu Z, Bogacz R, Lukasiewicz T
2021. Adv. Neural Inf. Process Syst., 34:3874-3886.
Barron HC, Auksztulewicz R, Friston KJ
2020. Prog. Neurobiol., 192:101821.