Abstract:Speaker diarization is usually referred to as the task that determines ``who spoke when'' in a recording. Until a few years ago, all competitive approaches were modular. Systems based on this framework reached state-of-the-art performance in most scenarios but had major difficulties dealing with overlapped speech. More recently, the advent of end-to-end models, capable of dealing with all aspects of speaker diarization with a single model and better performing regarding overlapped speech, has brought high levels of attention. This thesis is framed during a period of co-existence of these two trends. We describe a system based on a Bayesian hidden Markov model used to cluster x-vectors (speaker embeddings obtained with a neural network), known as VBx, which has shown remarkable performance on different datasets and challenges. We comment on its advantages and limitations and evaluate results on different relevant corpora. Then, we move towards end-to-end neural diarization (EEND) methods. Due to the need for large training sets for training these models and the lack of manually annotated diarization data in sufficient quantities, the compromise solution consists in generating training data artificially. We describe an approach for generating synthetic data which resembles real conversations in terms of speaker turns and overlaps. We show how this method generating ``simulated conversations'' allows for better performance than using a previously proposed method for creating ``simulated mixtures'' when training the popular EEND with encoder-decoder attractors (EEND-EDA). We also propose a new EEND-based model, which we call DiaPer, and show that it can perform better than EEND-EDA, especially when dealing with many speakers and handling overlapped speech. Finally, we compare both VBx-based and DiaPer systems on a wide variety of corpora and comment on the advantages of each technique.

Systematic Evaluation of Online Speaker Diarization Systems Regarding their Latency

A Review of Common Online Speaker Diarization Methods

An approach to optimize inference of the DIART speaker diarization pipeline

An Experimental Review of Speaker Diarization methods with application to Two-Speaker Conversational Telephone Speech recordings

A lightweight approach to real-time speaker diarization: from audio toward audio-visual data streams

Dissecting User-Perceived Latency of On-Device E2E Speech Recognition

Sequence-to-Sequence Neural Diarization with Automatic Speaker Detection and Representation

Online speaker diarization of meetings guided by speech separation

One model to rule them all ? Towards End-to-End Joint Speaker Diarization and Speech Recognition

End-to-End Integration of Speech Separation and Voice Activity Detection for Low-Latency Diarization of Telephone Conversations

From Modular to End-to-End Speaker Diarization

An Integrated Top-Down/Bottom-Up Approach To Speaker Diarization

End-to-End Evaluation for Low-Latency Simultaneous Speech Translation

A Quick and Effective Speaker Diarization System.

INTEGRATION OF SPEECH SEPARATION, DIARIZATION, AND RECOGNITION FOR MULTI-SPEAKER MEETINGS: SYSTEM DESCRIPTION, COMPARISON, AND ANALYSIS

LS-EEND: Long-Form Streaming End-to-End Neural Diarization with Online Attractor Extraction

Latent Class Model with Application to Speaker Diarization

Leveraging Self-Supervised Learning for Speaker Diarization

Investigating Confidence Estimation Measures for Speaker Diarization

End-to-end Online Speaker Diarization with Target Speaker Tracking