Abstract:We propose a novel speaker-dependent speech separation framework for the challenging CHiME-5 acoustic environments, exploiting advantages of both deep learning based and conventional preprocessing techniques to prepare data effectively for separating target speech from multi-talker mixed speech collected with multiple microphone arrays. First, a series of multi-channel operations is conducted to reduce existing reverberation and noise, and a single-channel deep learning based speech enhancement model is used to predict speech presence probabilities. Next, a two-stage supervised speech separation approach, using oracle speaker diarization information from CHiME-5, is proposed to separate speech of a target speaker from interference speakers in mixed speech. Given a set of three estimated masks of the background noise, the target speaker and the interference speakers from single-channel speech enhancement and separation models, a complex Gaussian mixture model based generalized eigenvalue beam-former is then used for enhancing the signal at the reference array while avoiding the speaker permutation issue. Furthermore, the proposed front-end can generate a large variety of processed data for an ensemble of speech recognition results. Experiments on the development set have shown that the proposed two-stage approach can yield significant improvements of recognition performance over the official baseline system and achieved top accuracies in all four competing evaluation categories among all systems submitted to the CHiME-5 Challenge.

Target Speaker Extraction for Overlapped Multi-Talker Speaker Verification

Target Speaker Verification with Selective Auditory Attention for Single and Multi-talker Speech

Audio-Visual Active Speaker Extraction for Sparsely Overlapped Multi-talker Speech

Speaker Verification based on Single Channel Speech Separation

Distortionless Multi-Channel Target Speech Enhancement for Overlapped Speech Recognition

Speaker Mask Transformer for Multi-talker Overlapped Speech Recognition

Audio-visual End-to-end Multi-channel Speech Separation, Dereverberation and Recognition

Margin-Mixup: A Method for Robust Speaker Verification in Multi-Speaker Audio

Royalflush Speaker Diarization System for ICASSP 2022 Multi-channel Multi-party Meeting Transcription Challenge

Towards Robust Speaker Verification with Target Speaker Enhancement

Overlapped speech recognition from a jointly learned multi-channel neural speech extraction and representation

Simultaneous Speech Extraction for Multiple Target Speakers under the Meeting Scenarios

End-to-End Multi-Speaker Speech Recognition using Speaker Embeddings and Transfer Learning

Unified Modeling of Multi-Talker Overlapped Speech Recognition and Diarization with a Sidecar Separator

A Two-stage Single-channel Speaker-dependent Speech Separation Approach for Chime-5 Challenge.

Speaker-conditioning Single-channel Target Speaker Extraction using Conformer-based Architectures

Speaker Overlap-aware Neural Diarization for Multi-party Meeting Analysis

A Hybrid Continuity Loss to Reduce Over-Suppression for Time-domain Target Speaker Extraction

Target Speaker Extraction Using Attention-Enhanced Temporal Convolutional Network

A Speaker-Dependent Approach to Separation of Far-Field Multi-Talker Microphone Array Speech for Front-End Processing in the CHiME-5 Challenge