AUTHOR=Kim Youngseok , Gokmen Tayfun , Miyazoe Hiroyuki , Solomon Paul , Kim Seyoung , Ray Asit , Doevenspeck Jonas , Khan Raihan S. , Narayanan Vijay , Ando Takashi 

TITLE=Neural network learning using non-ideal resistive memory devices

JOURNAL=Frontiers in Nanotechnology

VOLUME=4

YEAR=2022

URL=https://www.frontiersin.org/journals/nanotechnology/articles/10.3389/fnano.2022.1008266

DOI=10.3389/fnano.2022.1008266

ISSN=2673-3013

ABSTRACT=<p>We demonstrate a modified stochastic gradient (<italic>Tiki-Taka v2</italic> or <italic>TTv2</italic>) algorithm for deep learning network training in a cross-bar array architecture based on ReRAM cells. There have been limited discussions on cross-bar arrays for training applications due to the challenges in the switching behavior of nonvolatile memory materials. <italic>TTv2</italic> algorithm is known to overcome the device non-idealities for deep learning training. We demonstrate the feasibility of the algorithm for a linear regression task using 1R and 1T1R ReRAM devices. Using the measured device properties, we project the performance of a long short-term memory (LSTM) network with 78 K parameters. We show that <italic>TTv2</italic> algorithm relaxes the criteria for symmetric device update response. In addition, further optimization of the algorithm increases noise robustness and significantly reduces the required number of states, thereby drastically improving the model accuracy even with non-ideal devices and achieving the test error close to that of the conventional learning algorithm with an ideal device.</p>