Abstract:We propose an integrated end-to-end automatic speech recognition (ASR) paradigm by joint learning of the front-end speech signal processing and back-end acoustic modeling. We believe that only good signal processing can lead to top ASR performance in challenging acoustic environments. This notion leads to a unified deep neural network (DNN) framework for distant speech processing that can achieve both high-quality enhanced speech and high-accuracy ASR simultaneously. Our goal is accomplished by two techniques, namely: (i) a reverberation-time-aware DNN based speech dereverberation architecture that can handle a wide range of reverberation times to enhance speech quality of reverberant and noisy speech, followed by (ii) DNN-based multicondition training that takes both clean-condition and multicondition speech into consideration, leveraging upon an exploitation of the data acquired and processed with multichannel microphone arrays, to improve ASR performance. The final end-to-end system is established by a joint optimization of the speech enhancement and recognition DNNs. The recent REverberant Voice Enhancement and Recognition Benchmark (REVERB) Challenge task is used as a test bed for evaluating our proposed framework. We first report on superior objective measures in enhanced speech to those listed in the 2014 REVERB Challenge Workshop on the simulated data test set. Moreover, we obtain the best single-system word error rate (WER) of 13.28 on the 1-channel REVERB simulated data with the proposed DNN-based pre-processing algorithm and clean-condition training. Leveraging upon joint training with more discriminative ASR features and improved neural network based language models, a low single-system WER of 4.46 is attained. Next, a new multi-channel-condition joint learning and testing scheme delivers a state-of-the-art WER of 3.76 on the 8-channel simulated data with a single ASR system. Finally, we also report on a preliminary yet promising experimentation with the REVERB real test data.

Robust In-Car Speech Recognition Based on Nonlinear Multiple Regressions

Adaptive Nonlinear Regression Using Multiple Distributed Microphones for In-Car Speech Recognition

Optimizing Regression for In-Car Speech Recognition Using Multiple Distributed Microphones

Robust Log-Energy Estimation and Its Dynamic Change Enhancement for In-car Speech Recognition

Sub-band Based Log-Energy and Its Dynamic Range Stretching for Robust In-Car Speech Recognition

Feature Denoising Using Joint Sparse Representation for In-Car Speech Recognition.

Non-linear Spectral Contrast Stretching for In-Car Speech Recognition.

In-car speech enhancement using ensemble empirical mode decomposition

Speech Selection and Environmental Adaptation for Asynchronous Speech Recognition

In-car Speech Enhancement Based on Ensemble Empirical Mode Decomposition.

Wavoice: A mmWave-assisted Noise-resistant Speech Recognition SystemJust Accepted

Low-SNR Speech Enhancement and Separation in Driving Environment

DualSep: A Light-weight dual-encoder convolutional recurrent network for real-time in-car speech separation

Autoregressive Model-Based Robust Speech Recognition in Additive Noise Environment

Wavoice: an Mmwave-Assisted Noise-Resistant Speech Recognition System

Wavoice: an Mmwave-Assisted Noise-Resistant Speech Recognition System.

In-Vehicle Environment Noise Speech Enhancement Using Lightweight Wave-U-Net

Robust Speaker Recognition in Cross-Channel Condition

An End-to-End Deep Learning Approach to Simultaneous Speech Dereverberation and Acoustic Modeling for Robust Speech Recognition

An Experimental Performance Assessment of Temporal Convolutional Networks for Microphone Virtualization in a Car Cabin

Efficient Embedded Speech Recognition for Very Large Vocabulary Mandarin Car-Navigation Systems