Abstract:Recent works show that speech separation guided diarization (SSGD) is an increasingly promising direction, mainly thanks to the recent progress in speech separation. It performs diarization by first separating the speakers and then applying voice activity detection (VAD) on each separated stream. In this work we conduct an in-depth study of SSGD in the conversational telephone speech (CTS) domain, focusing mainly on low-latency streaming diarization applications. We consider three state-of-the-art speech separation (SSep) algorithms and study their performance both in online and offline scenarios, considering non-causal and causal implementations as well as continuous SSep (CSS) windowed inference. We compare different SSGD algorithms on two widely used CTS datasets: CALLHOME and Fisher Corpus (Part 1 and 2) and evaluate both separation and diarization performance. To improve performance, a novel, causal and computationally efficient leakage removal algorithm is proposed, which significantly decreases false alarms. We also explore, for the first time, fully end-to-end SSGD integration between SSep and VAD modules. Crucially, this enables fine-tuning on real-world data for which oracle speakers sources are not available. In particular, our best model achieves 8.8% DER on CALLHOME, which outperforms the current state-of-the-art end-to-end neural diarization model, despite being trained on an order of magnitude less data and having significantly lower latency, i.e., 0.1 vs. 1 s. Finally, we also show that the separated signals can be readily used also for automatic speech recognition, reaching performance close to using oracle sources in some configurations.

QDM-SSD: Quality-Aware Dynamic Masking for Separation-Based Speaker Diarization

Improving Separation-Based Speaker Diarization Via Iterative Model Refinement And Speaker Embedding Based Post-Processing

Separation Guided Speaker Diarization in Realistic Mismatched Conditions

Unsupervised Adaptation with Quality-Aware Masking to Improve Target-Speaker Voice Activity Detection for Speaker Diarization

A Spatial Long-Term Iterative Mask Estimation Approach for Multi-Channel Speaker Diarization and Speech Recognition.

A Deep Analysis of Speech Separation Guided Diarization Under Realistic Conditions

DCF-DS: Deep Cascade Fusion of Diarization and Separation for Speech Recognition under Realistic Single-Channel Conditions

Progressive Multi-Target Network Based Speech Enhancement with Snr-Preselection for Robust Speaker Diarization

Exploring Speaker-Related Information in Spoken Language Understanding for Better Speaker Diarization

Multi-channel Conversational Speaker Separation via Neural Diarization

So-DAS: A Two-Step Soft-Direction-Aware Speech Separation Framework

End-to-end integration of speech separation and voice activity detection for low-latency diarization of telephone conversations

A Real-time Speaker Diarization System Based on Spatial Spectrum

Scenario-Dependent Speaker Diarization for DIHARD-III Challenge

Speaker Diarization with Enhancing Speech for the First DIHARD Challenge

Target-speaker Voice Activity Detection with Improved I-Vector Estimation for Unknown Number of Speaker

A Quick and Effective Speaker Diarization System.

One model to rule them all ? Towards End-to-End Joint Speaker Diarization and Speech Recognition

Online speaker diarization of meetings guided by speech separation

Using Iterative Adaptation and Dynamic Mask for Child Speech Extraction under Real-World Multilingual Conditions