Abstract:Speech enhancement and automatic speech recognition (ASR) are most often evaluated in matched (or multi-condition) settings where the acoustic conditions of the training data match (or cover) those of the test data. Few studies have systematically assessed the impact of acoustic mismatches between training and test data, especially concerning recent speech enhancement and state-of-the-art ASR techniques. In this article, we study this issue in the context of the CHiME-3 dataset, which consists of sentences spoken by talkers situated in challenging noisy environments recorded using a 6-channel tablet based microphone array. We provide a critical analysis of the results published on this dataset for various signal enhancement, feature extraction, and ASR backend techniques and perform a number of new experiments in order to separately assess the impact of different noise environments, different numbers and positions of microphones, or simulated vs. real data on speech enhancement and ASR performance. We show that, with the exception of minimum variance distortionless response (MVDR) beamforming, most algorithms perform consistently on real and simulated data and can benefit from training on simulated data. We also find that training on different noise environments and different microphones barely affects the ASR performance, especially when several environments are present in the training data: only the number of microphones has a significant impact. Based on these results, we introduce the CHiME-4 Speech Separation and Recognition Challenge, which revisits the CHiME-3 dataset and makes it more challenging by reducing the number of microphones available for testing.

On Design of Robust Deep Models for CHiME-4 Multi-Channel Speech Recognition with Multiple Configurations of Array Microphones

An Iterative Mask Estimation Approach to Deep Learning Based Multi-Channel Speech Recognition

An information fusion framework with multi-channel feature concatenation and multi-perspective system combination for the deep-learning-based robust recognition of microphone array speech

An information fusion approach to recognizing microphone array speech in the CHiME-3 challenge based on a deep learning framework

Robust speech recognition using beamforming with adaptive microphone gains and multichannel noise reduction

Acoustic Modeling for Multi-Array Conversational Speech Recognition in the Chime-6 Challenge

A Speaker-Dependent Approach to Separation of Far-Field Multi-Talker Microphone Array Speech for Front-End Processing in the CHiME-5 Challenge

Channel selection using neural network posterior probability for speech recognition with distributed microphone arrays in everyday environments

Automatic channel selection and spatial feature integration for multi-channel speech recognition across various array topologies

A Space-and-Speaker-Aware Iterative Mask Estimation Approach to Multi-Channel Speech Recognition in the CHiME-6 Challenge.

Deep Long Short-Term Memory Adaptive Beamforming Networks For Multichannel Robust Speech Recognition

LSTM-Based Iterative Mask Estimation and Post-Processing for Multi-Channel Speech Enhancement

The THU-SPMI CHiME-4 system : Lightweight design with advanced multi-channel processing , feature enhancement , and language modeling

A Two-stage Single-channel Speaker-dependent Speech Separation Approach for Chime-5 Challenge.

Multi-Channel Automatic Speech Recognition Using Deep Complex Unet

The JHU Multi-Microphone Multi-Speaker ASR System for the CHiME-6 Challenge

Building state-of-the-art distant speech recognition using the CHiME-4 challenge with a setup of speech enhancement baseline

Multi-Channel Feature Adaptation for Robust Speech Recognition

Acoustic Model Ensembling Using Effective Data Augmentation for CHiME-5 Challenge.

2D-to-2d Mask Estimation for Speech Enhancement Based on Fully Convolutional Neural Network

An analysis of environment, microphone and data simulation mismatches in robust speech recognition