Abstract:Recurrent neural networks (RNNs) have achieved the state-of-the-art performance on various sequence learning tasks due to their powerful sequence modeling capability. However, RNNs usually require a large number of parameters and high computational complexity. Hence, it is quite challenging to implement complex RNNs on embedded devices with stringent memory and latency requirement. In this paper, we first present a novel hybrid compression method for a widely used RNN variant, long-short term memory (LSTM), to tackle these implementation challenges. By properly using circulant matrices, forward nonlinear function approximation, and efficient quantization schemes with a retrain-based training strategy, the proposed compression method can reduce more than 95% of memory usage with negligible accuracy loss when verified under language modeling and speech recognition tasks. An efficient scalable parallel hardware architecture is then proposed for the compressed LSTM. With an innovative chessboard division method for matrix-vector multiplications, the parallelism of the proposed hardware architecture can be freely chosen under certain latency requirement. Specifically, for the circulant matrix-vector multiplications employed in the compressed LSTM, the circulant matrices are judiciously reorganized to fit in with the chessboard division and minimize the number of memory accesses required for the matrix multiplications. The proposed architecture is modeled using register transfer language (RTL) and synthesized under the TSMC 90-nm CMOS technology. With 518.5-kB on-chip memory, we are able to process a 512x512 compressed LSTM in 1.71 mu s, corresponding to 2.46 TOPS on the uncompressed one, at a cost of 30.77-mm(2) chip area. The implementation results demonstrate that the proposed design can achieve significantly high flexibility and area efficiency, which satisfies many real-time applications on embedded devices. It is worth mentioning that the memory-efficient approach of accelerating LSTM developed in this paper is also applicable to other RNN variants.

An Energy-Efficient Architecture for Accelerating Inference of Memory-Augmented Neural Networks

A Low-Power In-Memory Multiplication and Accumulation Array with Modified Radix-4 Input and Canonical Signed Digit Weights

In-memory Implementation of On-chip Trainable and Scalable ANN for AI/ML Applications

Exploiting Near-Memory Processing Architectures for Bayesian Neural Networks Acceleration

Flash-Based Content Addressable Memory with L2 Distance for Memory-Augmented Neural Network

A Fast and Power Efficient Architecture to Parallelize LSTM based RNN for Cognitive Intelligence Applications.

A 3.89-Gops/mw Scalable Recurrent Neural Network Processor with Improved Efficiency on Memory and Computation

ARMIN: Towards a More Efficient and Light-weight Recurrent Memory Network.

NeuroNAS: A Framework for Energy-Efficient Neuromorphic Compute-in-Memory Systems using Hardware-Aware Spiking Neural Architecture Search

Quantized Memory-Augmented Neural Networks

HiMA: A Fast and Scalable History-based Memory Access Engine for Differentiable Neural Computer

Distributed Associative Memory Network with Memory Refreshing Loss

Memory-Computing Decoupling: A DNN Multitasking Accelerator with Adaptive Data Arrangement.

Energy-efficient Task Adaptation for NLP Edge Inference Leveraging Heterogeneous Memory Architectures

Sparsity-Oriented MRAM-Centric Computing for Efficient Neural Network Inference

An In-memory Booth Multiplier Based on Non-volatile Memory for Neural Network Applications.

Efficient Memory Management for Deep Neural Net Inference

Survey on Memory-Augmented Neural Networks: Cognitive Insights to AI Applications

Accelerating Recurrent Neural Networks: A Memory-Efficient Approach

Reconfigurable Architecture for Neural Approximation in Multimedia Computing.

An Energy-Efficient Near-Data Processing Accelerator for DNNs that Optimizes Data Accesses