Abstract:Memristors, emerging non-volatile memory devices, have shown promising potential in neuromorphic hardware designs, especially in spiking neural network (SNN) hardware implementation. Memristor-based SNNs have been successfully applied in a wide range of various applications, including image classification and pattern recognition. However, implementing memristor-based SNNs in text classification is still under exploration. One of the main reasons is that training memristor-based SNNs for text classification is costly due to the lack of efficient learning rules and memristor non-idealities. To address these issues and accelerate the research of exploring memristor-based spiking neural networks in text classification applications, we develop a simulation framework with a virtual memristor array using an empirical memristor model. We use this framework to demonstrate a sentiment analysis task in the IMDB movie reviews dataset. We take two approaches to obtain trained spiking neural networks with memristor models: 1) by converting a pre-trained artificial neural network (ANN) to a memristor-based SNN, or 2) by training a memristor-based SNN directly. These two approaches can be applied in two scenarios: offline classification and online training. We achieve the classification accuracy of 85.88% by converting a pre-trained ANN to a memristor-based SNN and 84.86% by training the memristor-based SNN directly, given that the baseline training accuracy of the equivalent ANN is 86.02%. We conclude that it is possible to achieve similar classification accuracy in simulation from ANNs to SNNs and from non-memristive synapses to data-driven memristive synapses. We also investigate how global parameters such as spike train length, the read noise, and the weight updating stop conditions affect the neural networks in both approaches.

Memristor-Based Lstm Network For Text Classification

Long short-term memory networks in memristor crossbars

Memristor-based circuit design of BiLSTM network

Long short-term memory networks in memristor crossbar arrays

Text Classification in Memristor-based Spiking Neural Networks

Design of CMOS-memristor Circuits for LSTM architecture

Evolving Long Short-Term Memory Network-Based Text Classification

Memristor-Based MobileNetV3 Circuit Design for Image Classification

Long Short-Term Memory Implementation Exploiting Passive RRAM Crossbar Array

Reduction 93.7% time and power consumption using a memristor-based imprecise gradient update algorithm

Memristive LSTM network hardware architecture for time-series predictive modeling problem

A Compact and Configurable Long Short-Term Memory Neural Network Hardware Architecture.

Long-term and short-term memory networks based on forgetting memristors

Memristive Circuit Design of Sequencer Network for Human Emotion Classification

Enabling Energy-Efficient Deployment of Large Language Models on Memristor Crossbar: A Synergy of Large and Small

Memristor‐Based Analog Computation and Neural Network Classification with a Dot Product Engine

A C-LSTM Neural Network for Text Classification

Wafer Quality Inspection using Memristive LSTM, ANN, DNN and HTM

In Situ Training of Feed-Forward and Recurrent Convolutional Memristor Networks

Local Bidirectional Long Short Term Memory for Text Classification