Abstract:Recent works show that speech separation guided diarization (SSGD) is an increasingly promising direction, mainly thanks to the recent progress in speech separation. It performs diarization by first separating the speakers and then applying voice activity detection (VAD) on each separated stream. In this work we conduct an in-depth study of SSGD in the conversational telephone speech (CTS) domain, focusing mainly on low-latency streaming diarization applications. We consider three state-of-the-art speech separation (SSep) algorithms and study their performance both in online and offline scenarios, considering non-causal and causal implementations as well as continuous SSep (CSS) windowed inference. We compare different SSGD algorithms on two widely used CTS datasets: CALLHOME and Fisher Corpus (Part 1 and 2) and evaluate both separation and diarization performance. To improve performance, a novel, causal and computationally efficient leakage removal algorithm is proposed, which significantly decreases false alarms. We also explore, for the first time, fully end-to-end SSGD integration between SSep and VAD modules. Crucially, this enables fine-tuning on real-world data for which oracle speakers sources are not available. In particular, our best model achieves 8.8% DER on CALLHOME, which outperforms the current state-of-the-art end-to-end neural diarization model, despite being trained on an order of magnitude less data and having significantly lower latency, i.e., 0.1 vs. 1 s. Finally, we also show that the separated signals can be readily used also for automatic speech recognition, reaching performance close to using oracle sources in some configurations.

GPU-accelerated Guided Source Separation for Meeting Transcription

Advances in Online Audio-Visual Meeting Transcription

INTEGRATION OF SPEECH SEPARATION, DIARIZATION, AND RECOGNITION FOR MULTI-SPEAKER MEETINGS: SYSTEM DESCRIPTION, COMPARISON, AND ANALYSIS

GPU Accelerated GMM Supervectors for Speaker and Language Recognition

SkiM: Skipping Memory LSTM for Low-Latency Real-Time Continuous Speech Separation

GSEP: A robust vocal and accompaniment separation system using gated CBHG module and loudness normalization

Simultaneous Speech Extraction for Multiple Target Speakers under the Meeting Scenarios

GASS: Generalizing Audio Source Separation with Large-scale Data

End-to-End Integration of Speech Separation and Voice Activity Detection for Low-Latency Diarization of Telephone Conversations

PGSS: Pitch-Guided Speech Separation.

Unifying Speech Enhancement and Separation with Gradient Modulation for End-to-End Noise-Robust Speech Separation

Multi-channel Conversational Speaker Separation via Neural Diarization

Meeting Recognition with Continuous Speech Separation and Transcription-Supported Diarization

Mixture Encoder Supporting Continuous Speech Separation for Meeting Recognition

A Modular-based Strategy for Mitigating Gradient Conflicts in Simultaneous Speech Translation

SURT 2.0: Advances in Transducer-based Multi-talker Speech Recognition

Implementing contextual biasing in GPU decoder for online ASR

Simultaneous Diarization and Separation of Meetings through the Integration of Statistical Mixture Models

Separator-Transducer-Segmenter: Streaming Recognition and Segmentation of Multi-party Speech

A Deep Analysis of Speech Separation Guided Diarization Under Realistic Conditions