Dendritic normalisation improves learning in sparsely connected artificial neural networks

PLoS Comput Biol. 2021 Aug 9;17(8):e1009202. doi: 10.1371/journal.pcbi.1009202. eCollection 2021 Aug.

Abstract

Artificial neural networks, taking inspiration from biological neurons, have become an invaluable tool for machine learning applications. Recent studies have developed techniques to effectively tune the connectivity of sparsely-connected artificial neural networks, which have the potential to be more computationally efficient than their fully-connected counterparts and more closely resemble the architectures of biological systems. We here present a normalisation, based on the biophysical behaviour of neuronal dendrites receiving distributed synaptic inputs, that divides the weight of an artificial neuron's afferent contacts by their number. We apply this dendritic normalisation to various sparsely-connected feedforward network architectures, as well as simple recurrent and self-organised networks with spatially extended units. The learning performance is significantly increased, providing an improvement over other widely-used normalisations in sparse networks. The results are two-fold, being both a practical advance in machine learning and an insight into how the structure of neuronal dendritic arbours may contribute to computation.

Publication types

  • Research Support, Non-U.S. Gov't

MeSH terms

  • Action Potentials / physiology
  • Animals
  • Computational Biology
  • Deep Learning
  • Dendrites / physiology*
  • Humans
  • Machine Learning*
  • Models, Neurological*
  • Nerve Net / physiology
  • Neural Networks, Computer*
  • Neurons / physiology
  • Stochastic Processes

Grants and funding

We acknowledge funding through BMBF grants 01GQ1406 (Bernstein Award 2013) to HC and 031L0229 to PJ. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.