AUTHOR=Chen Jieneng , Chen Jingye , Zhang Ruiming , Hu Xiaobin 

TITLE=Toward a Brain-Inspired System: Deep Recurrent Reinforcement Learning for a Simulated Self-Driving Agent

JOURNAL=Frontiers in Neurorobotics

VOLUME=Volume 13 - 2019

YEAR=2019

URL=https://www.frontiersin.org/journals/neurorobotics/articles/10.3389/fnbot.2019.00040

DOI=10.3389/fnbot.2019.00040

ISSN=1662-5218

ABSTRACT=An effective way to achieve intelligence is to simulate the various intelligent behaviors in the human brain. In recent years, bio-inspired learning methods have emerged, which are different from the classical mathematical programming principle. In the perspective of brain inspiration, reinforcement learning has gained additional interest in solving decision-making tasks as increasing neuroscientific research demonstrates that there are significant links between reinforcement learning and specific neural substrates. Due to the increasing research of human brains and reinforcement learning, scientists have investigated how robots can autonomously tackle complex tasks, e.g. in the form of self-driving agent control in a human-like way. In this study, we propose an end-to-end architecture using novel Deep Q Network architecture combined with a recurrence so as to resolve the problem in the field of simulated self-driving. A main contribution in this study is that we trained the agent using a brain-inspired trial-and-error way, which was more in line with the real world situation. Besides, there are three innovations in the proposed learning network: raw screen outputs is the only information the agent can rely on, a weighting layer that enhances the impact differences of the lengthy episode and a modified relay mechanism that overcomes the problem of sparsity and accelerates learning. The proposed network was trained and tested under the Mario Kart64 environment. After training for several epochs, the resulting agent was capable to perform advanced behaviors in the given scene. We wish the proposed brain-inspired learning system would motivate the real-world self-driving control solutions in the future.