Abstract:Voice Activity Detection (VAD) and Overlapped Speech Detection (OSD) are key pre-processing tasks for speaker diarization. In the meeting context, it is often easier to capture speech with a distant device. This consideration however leads to severe performance degradation. We study a unified supervised learning framework to solve distant multi-microphone joint VAD and OSD (VAD+OSD). This paper investigates various multi-channel VAD+OSD front-ends that weight and combine incoming channels. We propose three algorithms based on the Self-Attention Channel Combinator (SACC), previously proposed in the literature. Experiments conducted on the AMI meeting corpus exhibit that channel combination approaches bring significant VAD+OSD improvements in the distant speech scenario. Specifically, we explore the use of learned complex combination weights and demonstrate the benefits of such an approach in terms of explainability. Channel combination-based VAD+OSD systems are evaluated on the final back-end task, i.e. speaker diarization, and show significant improvements. Finally, since multi-channel systems are trained given a fixed array configuration, they may fail in generalizing to other array set-ups, e.g. mismatched number of microphones. A channel-number invariant loss is proposed to learn a unique feature representation regardless of the number of available microphones. The evaluation conducted on mismatched array configurations highlights the robustness of this training strategy.

Eigenchannel Space Combination Method of Joint Factor Analysis

Orthogonal subspace combination based on the joint factor analysis for text-independent speaker recognition

Applying Emotional Factor Analysis And I-Vector To Emotional Speaker Recognition

Maximum Likelihood I-Vector Space Using PCA for Speaker Verification.

Cross-modal Mask Fusion and Modality-Balanced Audio-Visual Speech Recognition

Multi-feature Combination for Speaker Recognition

Channel-Combination Algorithms for Robust Distant Voice Activity and Overlapped Speech Detection

Joint DOA Estimation and Dereverberation Based on Multi-Channel Linear Prediction Filtering and Azimuth Sparsity

A Two-Stage Approach for the Estimation of Multiple Input Multiple Output Acoustic Channels

Compensation of Intrinsic Variability with Factor Analysis Modeling for Robust Speaker Verification

A Two-Stage Approach for the Estimation of Doubly Spread Acoustic Channels

Mixture Encoder for Joint Speech Separation and Recognition

Channel Adversarial Training for Cross-channel Text-independent Speaker Recognition

Multi-resolution Time Frequency Feature and Complementary Combination for Short Utterance Speaker Recognition

Combining Eigenvoice Speaker Modeling And Vts-Based Environment Compensation For Robust Speech Recognition

Joint Speaker Features Learning for Audio-visual Multichannel Speech Separation and Recognition

Methods of Combining Multiple Classifiers with Different Features and Their Applications to Text-Independent Speaker Identification

Multichannel blind speech source separation with a disjoint constraint source model

Joint speaker encoder and neural back-end model for fully end-to-end automatic speaker verification with multiple enrollment utterances

A multiple eigenspaces constructing method and its application to face recognition