Abstract:Speaker diarization is usually referred to as the task that determines ``who spoke when'' in a recording. Until a few years ago, all competitive approaches were modular. Systems based on this framework reached state-of-the-art performance in most scenarios but had major difficulties dealing with overlapped speech. More recently, the advent of end-to-end models, capable of dealing with all aspects of speaker diarization with a single model and better performing regarding overlapped speech, has brought high levels of attention. This thesis is framed during a period of co-existence of these two trends. We describe a system based on a Bayesian hidden Markov model used to cluster x-vectors (speaker embeddings obtained with a neural network), known as VBx, which has shown remarkable performance on different datasets and challenges. We comment on its advantages and limitations and evaluate results on different relevant corpora. Then, we move towards end-to-end neural diarization (EEND) methods. Due to the need for large training sets for training these models and the lack of manually annotated diarization data in sufficient quantities, the compromise solution consists in generating training data artificially. We describe an approach for generating synthetic data which resembles real conversations in terms of speaker turns and overlaps. We show how this method generating ``simulated conversations'' allows for better performance than using a previously proposed method for creating ``simulated mixtures'' when training the popular EEND with encoder-decoder attractors (EEND-EDA). We also propose a new EEND-based model, which we call DiaPer, and show that it can perform better than EEND-EDA, especially when dealing with many speakers and handling overlapped speech. Finally, we compare both VBx-based and DiaPer systems on a wide variety of corpora and comment on the advantages of each technique.

Speaker Diarisation and Longitudinal Linking in Multi-Genre Broadcast Data.

Supplementary data for" Speaker Diarisation and Linking in Multi Genre Broadcast Data"

Cambridge University Transcription Systems for the Multi-Genre Broadcast Challenge.

Incorporating Spatial Cues in Modular Speaker Diarization for Multi-channel Multi-party Meetings

Microsoft Speaker Diarization System for the VoxCeleb Speaker Recognition Challenge 2020

A Spatial Long-Term Iterative Mask Estimation Approach for Multi-Channel Speaker Diarization and Speech Recognition.

From Modular to End-to-End Speaker Diarization

Multi-scale speaker embedding-based graph attention networks for speaker diarisation

An Integrated Top-Down/Bottom-Up Approach To Speaker Diarization

Neural Speaker Diarization Using Memory-Aware Multi-Speaker Embedding with Sequence-to-Sequence Architecture

The Development of the Cambridge University Alignment Systems for the Multi-Genre Broadcast Challenge.

Speaker Diarization Using Latent Space Clustering in Generative Adversarial Network

The 2015 Sheffield System for Transcription of Multi-Genre Broadcast Media

Speaker-turn aware diarization for speech-based cognitive assessments

Multi-Speaker End-to-End Multi-Modal Speaker Diarization System for the MISP 2022 Challenge

Semi-supervised multi-channel speaker diarization with cross-channel attention

The xmuspeech system for multi-channel multi-party meeting transcription challenge

Progressive Multi-Target Network Based Speech Enhancement with Snr-Preselection for Robust Speaker Diarization

A Real-time Speaker Diarization System Based on Spatial Spectrum

Summary of the DISPLACE Challenge 2023 - DIarization of SPeaker and LAnguage in Conversational Environments

Sequence-to-Sequence Neural Diarization with Automatic Speaker Detection and Representation