AUTHOR=Sun Yinqian , Zeng Yi , Li Yang 

TITLE=Solving the spike feature information vanishing problem in spiking deep Q network with potential based normalization

JOURNAL=Frontiers in Neuroscience

VOLUME=16

YEAR=2022

URL=https://www.frontiersin.org/journals/neuroscience/articles/10.3389/fnins.2022.953368

DOI=10.3389/fnins.2022.953368

ISSN=1662-453X

ABSTRACT=<p>Brain-inspired spiking neural networks (SNNs) are successfully applied to many pattern recognition domains. The SNNs-based deep structure has achieved considerable results in perceptual tasks, such as image classification and target detection. However, applying deep SNNs in reinforcement learning (RL) tasks is still a problem to be explored. Although there have been previous studies on the combination of SNNs and RL, most focus on robotic control problems with shallow networks or using the ANN-SNN conversion method to implement spiking deep Q networks (SDQN). In this study, we mathematically analyzed the problem of the disappearance of spiking signal features in SDQN and proposed a potential-based layer normalization (pbLN) method to train spiking deep Q networks directly. Experiment shows that compared with state-of-art ANN-SNN conversion method and other SDQN works, the proposed pbLN spiking deep Q networks (PL-SDQN) achieved better performance on Atari game tasks.</p>