Abstract:The training of neural network (NN) is usually time-consuming and resource intensive. Memristor has shown its potential in computation of NN. Especially for the metal-oxide resistive random access memory (RRAM), its crossbar structure and multi-bit characteristic can perform the matrix-vector product in high precision, which is the most common operation of NN. However, there exist two challenges on realizing the training of NN. Firstly, the current architecture can only support the inference phase of training and cannot perform the backpropagation (BP), the weights update of NN. Secondly, the training of NN requires enormous iterations and constantly updates the weights to reach the convergence, which leads to large energy consumption because of lots of write and read operations. In this work, we propose a novel architecture, TIME, and peripheral circuit designs to enable the training of NN in RRAM. TIME supports the BP and the weights update while maximizing the reuse of peripheral circuits for the inference operation on RRAM. Meanwhile, a variability-free tuning scheme and gradually-write circuits are designed to reduce the cost of tuning RRAM. We explore the performance of both SL (supervised learning) and DRL ( deep reinforcement learning) in TIME, and a specific mapping method of DRL is also introduced to further improve the energy efficiency. Experimental results show that, in SL, TIME can achieve 5.3x higher energy efficiency on average compared with the most powerful application-specific integrated circuits (ASIC) in the literature. In DRL, TIME can perform averagely 126x higher than GPU in energy efficiency. If the cost of tuning RRAM can be further reduced, TIME have the potential of boosting the energy efficiency by 2 orders of magnitude compared with ASIC.

Oxide-based filamentary RRAM for deep learning

Impacts of State Instability and Retention Failure of Filamentary Analog RRAM on the Performance of Deep Neural Network

RRAM-based Coprocessors for Deep Learning

Overview of Resistive Random Access Memory (RRAM): Materials, Filament Mechanisms, Performance Optimization, and Prospects

Filament-Free Bulk Resistive Memory Enables Deterministic Analogue Switching

Hardware Implementation Of Rram Based Binarized Neural Networks

RRAM based learning acceleration.

Device and circuit optimization of RRAM for neuromorphic computing

A Methodology to Improve Linearity of Analog RRAM for Neuromorphic Computing

Analytic Model for Statistical State Instability and Retention Behaviors of Filamentary Analog RRAM Array and Its Applications in Design of Neural Network

Optimized Programming Scheme Enabling Symmetric Conductance Modulation in HfO₂ Resistive Random-Access Memory (RRAM) for Neuromorphic Systems

Advances of RRAM Devices: Resistive Switching Mechanisms, Materials and Bionic Synaptic Application

Multi-level, forming and filament free, bulk switching trilayer RRAM for neuromorphic computing at the edge

Suppress Variations Of Analog Resistive Memory For Neuromorphic Computing By Localizing V-O Formation

Atomic Layer Deposition Films for Resistive Random‐Access Memories (Adv. Mater. Technol. 16/2024)

TIME: A Training-in-Memory Architecture for RRAM-Based Deep Neural Networks

Novel 2D Materials-based Resistive Devices

High-Throughput In-Memory Computing for Binary Deep Neural Networks with Monolithically Integrated RRAM and 90nm CMOS

TaOx Synapse Array Based on Ion Profile Engineering for High Accuracy Neuromorpic Computing

Resistive random access memory: introduction to device mechanism, materials and application to neuromorphic computing

Graphene-based RRAM devices for neural computing