Abstract:Pruning deep neural networks (DNNs) can reduce the model size and thus save hardware resources of a resistive-random-access-memory (ReRAM)-based DNN accelerator. For the tightly coupled crossbar structure, existing ReRAM-based pruning techniques prune the weights of a DNN in a structured manner, thereby attaining low pruning ratios. This article presents a novel pruning technique, SegPrune, for pruning the weights of a DNN flexibly on crossbar architectures in order to maximize the pruning ratio achieved while preserving crossbar efficiency. We observe that different filters of a weight matrix share a large number of matrix subcolumns (in the same rows), called segments, that can be pruned by using the same segment shape in the sense that the weights at the same column position of these segments are either simultaneously accuracy-sensitive (and should thus be reserved) or simultaneously accuracy-insensitive (and can thus be pruned). Due to the bit-line exchangeability in the crossbar, segments with the same pruning shape can be assembled together into the same crossbar to ensure crossbar execution efficiency. We propose a projection-based shape voting algorithm to select suitable segment shapes to drive the weight pruning process. Accordingly, we also introduce a low-overhead data path that can be easily integrated into any existing ReRAM-based DNN accelerator, achieving a high pruning ratio and a high execution efficiency. Our evaluation shows that SegPrune outperforms the state-of-the-art, Hybrid-P, and FORMAS, by up to $14.6\times $ and $3.6\times $ in pruning ratio, $13.9\times $ and $3.4\times $ in inference speedup, and $12.5\times $ and $3.1\times $ in energy reduction, respectively, while achieving an even higher accuracy at the cost of less than 0.27% extra hardware area overhead.

Pruned RNN-T for fast, memory-efficient ASR training

Class-Aware Pruning for Efficient Neural Networks

Exploring RNN-Transducer for Chinese Speech Recognition

Accurate and Structured Pruning for Efficient Automatic Speech Recognition

Improving RNN transducer with normalized jointer network

Structured Pruning of Recurrent Neural Networks through Neuron Selection

Reshaping deep neural network for fast decoding by node-pruning

Alignment Restricted Streaming Recurrent Neural Network Transducer.

TST: Time-Sparse Transducer for Automatic Speech Recognition

Exploiting Symmetric Temporally Sparse BPTT for Efficient RNN Training

Numerical Pruning for Efficient Autoregressive Models

BAT: Boundary aware transducer for memory-efficient and low-latency ASR

TODM: Train Once Deploy Many Efficient Supernet-Based RNN-T Compression For On-device ASR Models

Attention-based Transducer for Online Speech Recognition

Developing RNN-T Models Surpassing High-Performance Hybrid Models with Customization Capability.

Accelerating RNN-T Training and Inference Using CTC Guidance

LRTD: A Low-rank Transformer with Dynamic Depth and Width for Speech Recognition.

A Flexible Yet Efficient DNN Pruning Approach for Crossbar-Based Processing-in-Memory Architectures.

ConvRNN-T: Convolutional Augmented Recurrent Neural Network Transducers for Streaming Speech Recognition

Knowledge Distillation Via Module Replacing for Automatic Speech Recognition with Recurrent Neural Network Transducer