Abstract:Recently, deep neural networks (DNNs) significantly outperform Gaussian mixture models in acoustic modeling for speech recognition. However, the substantial increase in computational load during the inference stage makes deep models difficult to directly deploy on low-power embedded devices. To alleviate this issue, structure sparseness and low precision fixed-point quantization have been applied widely. In this work, binary neural networks for speech recognition are developed to reduce the computational cost during the inference stage. A fast implementation of binary matrix multiplication is introduced. On modern central processing unit (CPU) and graphics processing unit (GPU) architectures, a 5–7 times speedup compared with full precision floatingpoint matrix multiplication can be achieved in real applications. Several kinds of binary neural networks and related model optimization algorithms are developed for large vocabulary continuous speech recognition acoustic modeling. In addition, to improve the accuracy of binary models, knowledge distillation from the normal full precision floating-point model to the compressed binary model is explored. Experiments on the standard Switchboard speech recognition task show that the proposed binary neural networks can deliver 3–4 times speedup over the normal full precision deep models. With the knowledge distillation from the normal floating-point models, the binary DNNs or binary convolutional neural networks (CNNs) can restrict the word error rate (WER) degradation to within 15.0%, compared to the normal full precision floating-point DNNs or CNNs, respectively. Particularly for the binary CNN with binarization only on the convolutional layers, the WER degradation is very small and is almost negligible with the proposed approach.

1-Bit WaveNet: Compressing a Generative Neural Network in Speech Recognition with Two Binarized Methods

Binary Neural Networks for Speech Recognition

Compact Convolutional Recurrent Neural Networks Via Binarization For Speech Emotion Recognition

Bipolar Population Threshold Encoding for Audio Recognition with Deep Spiking Neural Networks

Speech Recognition Model Compression

BiPointNet: Binary Neural Network for Point Clouds

BiFSMNv2: Pushing Binary Neural Networks for Keyword Spotting to Real-Network Performance.

Bi-Real Net V2: Rethinking Non-linearity for 1-Bit CNNs and Going Beyond

Highly Efficient Neural Network Language Model Compression Using Soft Binarization Training

Lowbit Neural Network Quantization for Speaker Verification

Effective SVD-Based Deep Network Compression for Automatic Speech Recognition.

Neural Network Language Model Compression with Product Quantization and Soft Binarization

Bit-Quantized-Net: an Effective Method for Compressing Deep Neural Networks.

Sub-bit Neural Networks: Learning to Compress and Accelerate Binary Neural Networks

Wavenet Based Low Rate Speech Coding

An Energy and Memory Efficient Speaker Verification System Based on Binary Neural Networks

Bi-Real Net: Binarizing Deep Network Towards Real-Network Performance

Binary Convolutional Neural Network with High Accuracy and Compression Rate

Speech enhancement deep-learning architecture for efficient edge processing

Efficient Non-Autoregressive GAN Voice Conversion using VQWav2vec Features and Dynamic Convolution

Deep learning model compression using network sensitivity and gradients