Abstract:We propose an integrated end-to-end automatic speech recognition (ASR) paradigm by joint learning of the front-end speech signal processing and back-end acoustic modeling. We believe that only good signal processing can lead to top ASR performance in challenging acoustic environments. This notion leads to a unified deep neural network (DNN) framework for distant speech processing that can achieve both high-quality enhanced speech and high-accuracy ASR simultaneously. Our goal is accomplished by two techniques, namely: (i) a reverberation-time-aware DNN based speech dereverberation architecture that can handle a wide range of reverberation times to enhance speech quality of reverberant and noisy speech, followed by (ii) DNN-based multicondition training that takes both clean-condition and multicondition speech into consideration, leveraging upon an exploitation of the data acquired and processed with multichannel microphone arrays, to improve ASR performance. The final end-to-end system is established by a joint optimization of the speech enhancement and recognition DNNs. The recent REverberant Voice Enhancement and Recognition Benchmark (REVERB) Challenge task is used as a test bed for evaluating our proposed framework. We first report on superior objective measures in enhanced speech to those listed in the 2014 REVERB Challenge Workshop on the simulated data test set. Moreover, we obtain the best single-system word error rate (WER) of 13.28 on the 1-channel REVERB simulated data with the proposed DNN-based pre-processing algorithm and clean-condition training. Leveraging upon joint training with more discriminative ASR features and improved neural network based language models, a low single-system WER of 4.46 is attained. Next, a new multi-channel-condition joint learning and testing scheme delivers a state-of-the-art WER of 3.76 on the 8-channel simulated data with a single ASR system. Finally, we also report on a preliminary yet promising experimentation with the REVERB real test data.

Early Reflections Based Speech Enhancement

Preserving Early Reflections to Improve Speech Quality of Reverberant Speech Separation

On phase recovery and preserving early reflections for deep-learning speech dereverberation

Direct source and early reflections localization using deep deconvolution network under reverberant environment

An End-to-End Deep Learning Approach to Simultaneous Speech Dereverberation and Acoustic Modeling for Robust Speech Recognition

Monaural Speech Dereverberation using Deformable Convolutional Networks

Direct source and early reflections localization using deep deconvolution network under reverbrate environment

Speech enhancement with frequency domain auto-regressive modeling

Echo-aware room impulse response generation

Efficient High-Performance Bark-Scale Neural Network for Residual Echo and Noise Suppression

RVAE-EM: Generative speech dereverberation based on recurrent variational auto-encoder and convolutive transfer function

Frequency-domain Dereverberation on Speech Signal Using Surround Retinex

Predicting reflection patterns from binaural activity maps using deep neural networks

Multi-channel target speech enhancement based on ERB-scaled spatial coherence features

AEC in a NetShell: On Target and Topology Choices for FCRN Acoustic Echo Cancellation

Acoustic Echo Cancellation by Combining Adaptive Digital Filter and Recurrent Neural Network

Efficient Acoustic Echo Suppression with Condition-Aware Training

Improving Deep Neural Network Based Speech Enhancement in Low SNR Environments

Dereverberation using joint estimation of dry speech signal and acoustic system

Translate Reverberated Speech to Anechoic Ones: Speech Dereverberation with BERT

End-to-End Classification of Reverberant Rooms using DNNs