Abstract:In this paper, we propose a novel teacher-student learning framework for the preprocessing of a speech recognizer, leveraging the online noise tracking capabilities of improved minima controlled recursive averaging IMCRA and deep learning of nonlinear interactions between speech and noise. First, a teacher model with deep architectures is built to learn the target of ideal ratio masks IRMs using simulated training pairs of clean and noisy speech data. Next, a student model is trained to learn an improved speech presence probability by incorporating the estimated IRMs from the teacher model into the IMCRA approach. The student model can be compactly designed in a causal processing mode having no latency with the guidance of a complex and noncausal teacher model. Moreover, the clean speech requirement, which is difficult to meet in real-world adverse environments, can be relaxed for training the student model, implying that noisy speech data can be directly used to adapt the regression-based enhancement model to further improve speech recognition accuracies for noisy speech collected in such conditions. Experiments on the CHiME-4 challenge task show that our best student model with bidirectional gated recurrent units BGRUs can achieve a relative word error rate WER reduction of 18.85% for the real test set when compared to unprocessed system without acoustic model retraining. However, the traditional teacher model degrades the performance of the unprocessed system in this case. In addition, the student model with a deep neural network DNN in causal mode having no latency yields a relative WER reduction of 7.94% over the unprocessed system with 670 times less computing cycles when compared to the BGRU-equipped student model. Finally, the conventional speech enhancement and IRM-based deep learning method destroyed the ASR performance when the recognition system became more powerful. While our proposed approach could still improve the ASR performance even in the more powerful recognition system.

DNN Training Based on Classic Gain Function for Single-channel Speech Enhancement and Recognition.

Speech Enhancement Based on Teacher–Student Deep Learning Using Improved Speech Presence Probability for Noise-Robust Speech Recognition

A Time Domain Progressive Learning Approach with SNR Constriction for Single-Channel Speech Enhancement and Recognition

Single-Channel Speech Enhancement Algorithm Based on ME-MGCRN in Low Signal-to-Noise Scenario

2D-to-2d Mask Estimation for Speech Enhancement Based on Fully Convolutional Neural Network

An Experimental Study on Speech Enhancement Based on Deep Neural Networks

A regression approach to speech enhancement based on deep neural networks

Deep Neural Network Based Noised Asian Speech Enhancement and Its Implementation on a Hearing Aid App.

Robust Speech Recognition With Speech Enhanced Deep Neural Networks

Improving Deep Neural Network Based Speech Enhancement in Low SNR Environments

Speech enhancement from fused features based on deep neural network and gated recurrent unit network

Joint Training Of Front-End And Back-End Deep Neural Networks For Robust Speech Recognition

SNR-Based Progressive Learning of Deep Neural Network for Speech Enhancement

Error Modeling Via Asymmetric Laplace Distribution for Deep Neural Network Based Single-Channel Speech Enhancement

Dynamic noise aware training for speech enhancement based on deep neural networks.

SNR-Based Features and Diverse Training Data for Robust DNN-Based Speech Enhancement

Joint Noise and Mask Aware Training for DNN-based Speech Enhancement with SUB-band Features

Double Adversarial Network Based Monaural Speech Enhancement for Robust Speech Recognition.

A Novel Training Strategy Using Dynamic Data Generation for Deep Neural Network Based Speech Enhancement.

A Regression Approach to Single-Channel Speech Separation Via High-Resolution Deep Neural Networks.

Acceleration Strategies for Speech Recognition Based on Deep Neural Networks