Abstract:The ever-increasing demand to extract temporal correlations across sequential data and perform context-based learning in this era of big data has led to the development of long short-term memory (LSTM) networks. Furthermore, there is an urgent need to perform these time-series data-dependent applications including speech/video processing and recognition, language modelling and translation, etc. on compact internet-of-things (IoT) edge devices with limited energy. To this end, in this work, for the first time, we propose an extremely area- and energy-efficient LSTM network implementation exploiting the passive resistive random access memory (RRAM) crossbar array. We developed a hardware-aware LSTM network simulation framework and performed an extensive analysis of the proposed LSTM implementation considering the non-ideal hardware artifacts such as spatial (device-to-device) and temporal variations, non-linearity, noise, etc. utilizing an experimentally calibrated comprehensive phenomenological model for passive RRAM crossbar array. Our results indicate that the proposed passive RRAM crossbar-based LSTM network implementation not only outperforms the prior digital and active 1T-1R crossbar-based LSTM implementations by more than three orders of magnitude in terms of area and two orders of magnitude in terms of training energy for identical network accuracy, but also exhibits robustness against spatial and temporal variations and noise, and a faster convergence rate. Our work may provide the incentive for experimental realization of LSTM networks on passive RRAM crossbar arrays.

Design of CMOS-memristor Circuits for LSTM architecture

Generalised Analog LSTMs Recurrent Modules for Neural Computing

Memristive LSTM network hardware architecture for time-series predictive modeling problem

Circuit Design of Memristor-based GRU and Its Applications in SOC Estimation

Long short-term memory networks in memristor crossbars

Long short-term memory networks in memristor crossbar arrays

Memristor-based LSTM Network with in Situ Training and Its Applications

Memristor-Based Lstm Network For Text Classification

Long Short-Term Memory Implementation Exploiting Passive RRAM Crossbar Array

Memristive LSTM Network for Sentiment Analysis

Memristor-based circuit design of BiLSTM network

Wafer Quality Inspection using Memristive LSTM, ANN, DNN and HTM

A Compact and Configurable Long Short-Term Memory Neural Network Hardware Architecture.

In Situ Training of Feed-Forward and Recurrent Convolutional Memristor Networks

Memristor-Based Circuit Demonstration of Gated Recurrent Unit for Predictable Neural Network

Design of CMOS Memristor Emulator and Logic Gates For Neuromorphic Circuits

GST-memristor-based online learning neural networks

Machine Learning with Memristors via Thermodynamic RAM

Memristor Neural Network Design

Dynamics of Dual Memristors-based Neuron Circuit for Pattern Recognition

Design of Analog CMOS-Memristive Neural Network Circuits for Pattern Recognition