Abstract:Although great progress has been made in automatic speech recognition, significant performance degradation still exists in noisy environments. Our previous work has demonstrated the superior noise robustness of very deep convolutional neural networks (VDCNN). Based on our work on VDCNNs, this paper proposes a more advanced model referred to as the very deep convolutional residual network (VDCRN). This new model incorporates batch normalization and residual learning, showing more robustness than previous VDCNNs. Then, to alleviate the mismatch between the training and testing conditions, model adaptation and adaptive training are developed and compared for the new VDCRN. This paper focuses on factor aware training (FAT) and cluster adaptive training (CAT). For FAT, a unified framework is explored. For CAT, two schemes are first explored to construct the bases in the canonical model; furthermore, a factorized version of CAT is designed to address multiple nonspeech variabilities in one model. Finally, a complete multipass system is proposed to achieve the best system performance in the noisy scenarios. The proposed new approaches are evaluated on three different tasks: Aurora4 (simulated data with additive noise and channel distortion), CHiME4 (both simulated and real data with additive noise and reverberation), and the AMI meeting transcription task (real data with significant reverberation). The evaluation not only includes different noisy conditions, but also covers both simulated and real noisy data. The experiments show that the new VDCRN is more robust, and the adaptation on this model can further significantly reduce the word error rate (WER). The proposed best architecture obtains consistent and very large improvements on all tasks compared to the baseline VDCNN or long short-term memory. Particularly, on Aurora4 a new milestone 5.67% WER is achieved by only improving acoustic modeling.

Noisy training for deep neural networks in speech recognition

DENOISPEECH: DENOISING TEXT TO SPEECH WITH FRAME-LEVEL NOISE MODELING

Noise is the Fatal Poison: A Noise-aware Network for Noisy Dataset Classification

Noisy-target Training: A Training Strategy for DNN-based Speech Enhancement without Clean Speech

Iterative Noisy-Target Approach: Speech Enhancement Without Clean Speech

Dynamic noise aware training for speech enhancement based on deep neural networks.

A regression approach to speech enhancement based on deep neural networks

Deep Neural Network Based Noised Asian Speech Enhancement and Its Implementation on a Hearing Aid App.

Multi-task Joint-Learning of Deep Neural Networks for Robust Speech Recognition

SNR-Based Features and Diverse Training Data for Robust DNN-Based Speech Enhancement

Adaptive Very Deep Convolutional Residual Network for Noise Robust Speech Recognition

Transfer Learning for Acoustic Modeling of Noise Robust Speech Recognition

A Novel Training Strategy Using Dynamic Data Generation for Deep Neural Network Based Speech Enhancement.

Building DNN acoustic models for large vocabulary speech recognition

Training Recurrent Neural Networks against Noisy Computations during Inference

Denoising Noisy Neural Networks: A Bayesian Approach with Compensation.

Noisy Training Improves E2E ASR for the Edge

Incoherent training of deep neural networks to de-correlate bottleneck features for speech recognition

Target Speech Signal Enhancement Based on Deep Neural Networks

Boosting Noise Robustness of Acoustic Model via Deep Adversarial Training

Control System and Speech Recognition of Exhibition Hall Digital Media Based on Computer Technology