Abstract:We propose an integrated end-to-end automatic speech recognition (ASR) paradigm by joint learning of the front-end speech signal processing and back-end acoustic modeling. We believe that only good signal processing can lead to top ASR performance in challenging acoustic environments. This notion leads to a unified deep neural network (DNN) framework for distant speech processing that can achieve both high-quality enhanced speech and high-accuracy ASR simultaneously. Our goal is accomplished by two techniques, namely: (i) a reverberation-time-aware DNN based speech dereverberation architecture that can handle a wide range of reverberation times to enhance speech quality of reverberant and noisy speech, followed by (ii) DNN-based multicondition training that takes both clean-condition and multicondition speech into consideration, leveraging upon an exploitation of the data acquired and processed with multichannel microphone arrays, to improve ASR performance. The final end-to-end system is established by a joint optimization of the speech enhancement and recognition DNNs. The recent REverberant Voice Enhancement and Recognition Benchmark (REVERB) Challenge task is used as a test bed for evaluating our proposed framework. We first report on superior objective measures in enhanced speech to those listed in the 2014 REVERB Challenge Workshop on the simulated data test set. Moreover, we obtain the best single-system word error rate (WER) of 13.28 on the 1-channel REVERB simulated data with the proposed DNN-based pre-processing algorithm and clean-condition training. Leveraging upon joint training with more discriminative ASR features and improved neural network based language models, a low single-system WER of 4.46 is attained. Next, a new multi-channel-condition joint learning and testing scheme delivers a state-of-the-art WER of 3.76 on the 8-channel simulated data with a single ASR system. Finally, we also report on a preliminary yet promising experimentation with the REVERB real test data.

Environment-Dependent Attention-Driven Recurrent Convolutional Neural Network for Robust Speech Enhancement

A Fast Convolutional Self-Attention Based Speech Dereverberation Method For Robust Speech Recognition

Reverberant speech enhancement based on phase and reverberation aware DNN

Environmental Attention-Guided Branchy Neural Network for Speech Enhancement

Supervised Attention Multi-Scale Temporal Convolutional Network for monaural speech enhancement

An End-to-End Deep Learning Approach to Simultaneous Speech Dereverberation and Acoustic Modeling for Robust Speech Recognition

Improving Deep Neural Network Based Speech Enhancement in Low SNR Environments

Phase and Reverberation Aware DNN for Distant-Talking Speech Enhancement

Speech enhancement from fused features based on deep neural network and gated recurrent unit network

Densely Connected Multi-Stage Model with Channel Wise Subband Feature for Real-Time Speech Enhancement.

Single Channel Speech Enhancement Using Temporal Convolutional Recurrent Neural Networks.

Single-Channel Speech Enhancement Algorithm Based on ME-MGCRN in Low Signal-to-Noise Scenario

Speech Dereverberation for Enhancement and Recognition Using Dynamic Features Constrained Deep Neural Networks and Feature Adaptation

A Dual-Staged Context Aggregation Method Towards Efficient End-To-End Speech Enhancement

An Attention-Based Neural Network Approach For Single Channel Speech Enhancement

A Recursive Network with Dynamic Attention for Monaural Speech Enhancement

Parallel Gated Neural Network With Attention Mechanism For Speech Enhancement

Convolutional Recurrent Neural Network with Attention for 3D Speech Enhancement

Deep Encoder/decoder Dual-Path Neural Network for Speech Separation in Noisy Reverberation Environments

NAAGN: Noise-Aware Attention-Gated Network for Speech Enhancement

Speech-enhanced and Noise-aware Networks for Robust Speech Recognition