Abstract:Although great progress has been made in automatic speech recognition (ASR), significant performance degradation still exists in noisy environments. In this paper, a novel factor-aware training framework, named neural network-based multifactor aware joint training, is proposed to improve the recognition accuracy for noise robust speech recognition. This approach is a structured model which integrates several different functional modules into one computational deep model. We explore and extract speaker, phone, and environment factor representations using deep neural networks (DNNs), which are integrated into the main ASR DNN to improve classification accuracy. In addition, the hidden activations in the main ASR DNN are used to improve factor extraction, which in turn helps theASRDNN. All the model parameters, including those in the ASR DNN and factor extraction DNNs, are jointly optimized under the multitask learning framework. Unlike prior traditional techniques for the factor-aware training, our approach requires no explicit separate stages for factor extraction and adaptation. Moreover, the proposed neural network-based multifactor aware joint training can be easily combined with the conventional factor-aware training which uses the explicit factors, such as i-vector, noise energy, and T 60 value to obtain additional improvement. The proposed method is evaluated on two main noise robust tasks: the AMI single distant microphone task in which reverberation is the main concern, and the Aurora4 task in which multiple noise types exist. Experiments on both tasks show that the proposed model can significantly reduce word error rate (WER). The best configuration achieved more than 15% relative reduction in WER over the baselines on these two tasks.

Training method of end-to-end neural network speech recognition model

Neural network language model training method and device and voice recognition method

Automatic speech recognition based on time domain modeling

Research on End-to-end Voiceprint Recognition Model Based on Convolutional Neural Network

End-to-End Feature Learning for Text-Independent Speaker Verification

A New Neural Network Oriented Speech Recognition

Performance Optimization of Speech Recognition System with Deep Neural Network Model

Continuous speech recognition method and continuous speech recognition system

Method for training speaker recognition network model, speaker recognition method and system

A Chinese Acoustic Model Based On Convolutional Neural Network

Noisy training for deep neural networks in speech recognition

An Overview Of End-To-End Automatic Speech Recognition

Cascaded CNN-resBiLSTM-CTC: an End-to-End Acoustic Model for Speech Recognition.

Deep neural network construction method for voice command word recognition and recognition method and device

A Novel Training Strategy Using Dynamic Data Generation for Deep Neural Network Based Speech Enhancement.

Speech-enhanced and Noise-aware Networks for Robust Speech Recognition

Target Speech Signal Enhancement Based on Deep Neural Networks

Speech Enhancement Method Based on LSTM Neural Network for Speech Recognition

Neural Network Based Multi-Factor Aware Joint Training for Robust Speech Recognition.

A regression approach to speech enhancement based on deep neural networks

Learning the Front-End Speech Feature with Raw Waveform for End-to-End Speaker Recognition