AUTHOR=Kubo Yoshimasa , Chalmers Eric , Luczak Artur TITLE=Combining backpropagation with Equilibrium Propagation to improve an Actor-Critic reinforcement learning framework JOURNAL=Frontiers in Computational Neuroscience VOLUME=16 YEAR=2022 URL=https://www.frontiersin.org/journals/computational-neuroscience/articles/10.3389/fncom.2022.980613 DOI=10.3389/fncom.2022.980613 ISSN=1662-5188 ABSTRACT=

Backpropagation (BP) has been used to train neural networks for many years, allowing them to solve a wide variety of tasks like image classification, speech recognition, and reinforcement learning tasks. But the biological plausibility of BP as a mechanism of neural learning has been questioned. Equilibrium Propagation (EP) has been proposed as a more biologically plausible alternative and achieves comparable accuracy on the CIFAR-10 image classification task. This study proposes the first EP-based reinforcement learning architecture: an Actor-Critic architecture with the actor network trained by EP. We show that this model can solve the basic control tasks often used as benchmarks for BP-based models. Interestingly, our trained model demonstrates more consistent high-reward behavior than a comparable model trained exclusively by BP.