Abstract:In this paper, we propose a novel teacher-student learning framework for the preprocessing of a speech recognizer, leveraging the online noise tracking capabilities of improved minima controlled recursive averaging IMCRA and deep learning of nonlinear interactions between speech and noise. First, a teacher model with deep architectures is built to learn the target of ideal ratio masks IRMs using simulated training pairs of clean and noisy speech data. Next, a student model is trained to learn an improved speech presence probability by incorporating the estimated IRMs from the teacher model into the IMCRA approach. The student model can be compactly designed in a causal processing mode having no latency with the guidance of a complex and noncausal teacher model. Moreover, the clean speech requirement, which is difficult to meet in real-world adverse environments, can be relaxed for training the student model, implying that noisy speech data can be directly used to adapt the regression-based enhancement model to further improve speech recognition accuracies for noisy speech collected in such conditions. Experiments on the CHiME-4 challenge task show that our best student model with bidirectional gated recurrent units BGRUs can achieve a relative word error rate WER reduction of 18.85% for the real test set when compared to unprocessed system without acoustic model retraining. However, the traditional teacher model degrades the performance of the unprocessed system in this case. In addition, the student model with a deep neural network DNN in causal mode having no latency yields a relative WER reduction of 7.94% over the unprocessed system with 670 times less computing cycles when compared to the BGRU-equipped student model. Finally, the conventional speech enhancement and IRM-based deep learning method destroyed the ASR performance when the recognition system became more powerful. While our proposed approach could still improve the ASR performance even in the more powerful recognition system.

Single-channel Speech Enhancement Student under Multi-channel Speech Enhancement Teacher

Audio-Visual Speech Enhancement with Deep Multi-modality Fusion

Speech Enhancement Based on Teacher–Student Deep Learning Using Improved Speech Presence Probability for Noise-Robust Speech Recognition

SNR-Based Teachers-Student Technique for Speech Enhancement

Densely Connected Multi-Stage Model with Channel Wise Subband Feature for Real-Time Speech Enhancement.

Shared Network for Speech Enhancement Based on Multi-Task Learning.

An Attention-Based Neural Network Approach For Single Channel Speech Enhancement

On monoaural speech enhancement for automatic recognition of real noisy speech using mixture invariant training

Supervised Attention Multi-Scale Temporal Convolutional Network for monaural speech enhancement

Deep Neural Network Based Noised Asian Speech Enhancement and Its Implementation on a Hearing Aid App.

A Time Domain Progressive Learning Approach with SNR Constriction for Single-Channel Speech Enhancement and Recognition

Neural Speech Enhancement with Unsupervised Pre-Training and Mixture Training

Deep Noise Tracking Network: A Hybrid Signal Processing/Deep Learning Approach to Speech Enhancement

A Feature Integration Network for Multi-Channel Speech Enhancement

Improving Audio-visual Speech Recognition Performance with Cross-modal Student-teacher Training

Multi-task single channel speech enhancement using speech presence probability as a secondary task training target

CompNet: Complementary Network for Single-Channel Speech Enhancement.

Multi-Stage Progressive Speech Enhancement Network

Distortionless Multi-Channel Target Speech Enhancement for Overlapped Speech Recognition

Dynamic noise aware training for speech enhancement based on deep neural networks.

DNN Training Based on Classic Gain Function for Single-channel Speech Enhancement and Recognition.