Abstract:In this paper, we propose a novel teacher-student learning framework for the preprocessing of a speech recognizer, leveraging the online noise tracking capabilities of improved minima controlled recursive averaging IMCRA and deep learning of nonlinear interactions between speech and noise. First, a teacher model with deep architectures is built to learn the target of ideal ratio masks IRMs using simulated training pairs of clean and noisy speech data. Next, a student model is trained to learn an improved speech presence probability by incorporating the estimated IRMs from the teacher model into the IMCRA approach. The student model can be compactly designed in a causal processing mode having no latency with the guidance of a complex and noncausal teacher model. Moreover, the clean speech requirement, which is difficult to meet in real-world adverse environments, can be relaxed for training the student model, implying that noisy speech data can be directly used to adapt the regression-based enhancement model to further improve speech recognition accuracies for noisy speech collected in such conditions. Experiments on the CHiME-4 challenge task show that our best student model with bidirectional gated recurrent units BGRUs can achieve a relative word error rate WER reduction of 18.85% for the real test set when compared to unprocessed system without acoustic model retraining. However, the traditional teacher model degrades the performance of the unprocessed system in this case. In addition, the student model with a deep neural network DNN in causal mode having no latency yields a relative WER reduction of 7.94% over the unprocessed system with 670 times less computing cycles when compared to the BGRU-equipped student model. Finally, the conventional speech enhancement and IRM-based deep learning method destroyed the ASR performance when the recognition system became more powerful. While our proposed approach could still improve the ASR performance even in the more powerful recognition system.

Discriminative Learning of Additive Noise and Channel Distortions for Robust Speech Recognition

Robust Speech Recognition Method Based on Discriminative Environment Feature Extraction

Robust Speech Recognition Method Based on Discriminative Learning of Environmental Features

Discriminative environment feature extraction for robust speech recognition

Compensation of Speech Enhancement Distortion for Robust Speech Recognition

Performance of Discriminative HMM Training in Noise.

Mean and Covariance Adaptation Based on Minimum Classification Error Linear Regression for Continuous Density HMMs

An Algorithm of Model Compensation Based on the Estimation of Additive Noise and Channel Function for Speech Recognition

Noisy speech recognition performance of discriminative HMMs

Teacher-Student Training Approach Using an Adaptive Gain Mask for LSTM-Based Speech Enhancement in the Airborne Noise Environment

Discriminative Dynamic Gaussian Mixture Selection with Enhanced Robustness and Performance for Multi-Accent Speech Recognition

Speech-enhanced and Noise-aware Networks for Robust Speech Recognition

A Discriminative Training Approach for Text-Independent Speaker Recognition.

SUBTRACTION OF ADDITIVE NOISE FROM CORRUPTED SPEECH FOR ROBUST SPEECH RECOGNITION

Stereo Hidden Markov Modeling for Noise Robust Speech Recognition

Error Modeling Via Asymmetric Laplace Distribution for Deep Neural Network Based Single-Channel Speech Enhancement

Autoregressive Model-Based Robust Speech Recognition in Additive Noise Environment

Unified adaptation approach for robust speech recognition

Transfer Learning for Acoustic Modeling of Noise Robust Speech Recognition

Speech Enhancement Based on Teacher–Student Deep Learning Using Improved Speech Presence Probability for Noise-Robust Speech Recognition

Channel and speaker adaptation techniques for robust speech recognition