AUTHOR=Gammell Jimmy , Buckley Sonia , Nam Sae Woo , McCaughan Adam N. TITLE=Layer-Skipping Connections Improve the Effectiveness of Equilibrium Propagation on Layered Networks JOURNAL=Frontiers in Computational Neuroscience VOLUME=15 YEAR=2021 URL=https://www.frontiersin.org/journals/computational-neuroscience/articles/10.3389/fncom.2021.627357 DOI=10.3389/fncom.2021.627357 ISSN=1662-5188 ABSTRACT=

Equilibrium propagation is a learning framework that marks a step forward in the search for a biologically-plausible implementation of deep learning, and could be implemented efficiently in neuromorphic hardware. Previous applications of this framework to layered networks encountered a vanishing gradient problem that has not yet been solved in a simple, biologically-plausible way. In this paper, we demonstrate that the vanishing gradient problem can be mitigated by replacing some of a layered network's connections with random layer-skipping connections in a manner inspired by small-world networks. This approach would be convenient to implement in neuromorphic hardware, and is biologically-plausible.