Abstract:In clustering-based speaker diarization systems, the embedding clusters for distinctive speakers exhibit wide variability in size and density, posing difficulty for clustering accuracy. In spite of this, with the assistance of the overall distance relationships among speaker embeddings, most of the embeddings can be grouped to the correct cluster by sophisticated offline clustering algorithms. However, in online scenarios, such a complete distance relationships of the embeddings can not be obtained due to the incremental arrival of embeddings. Consequently, determining the number of clusters and then correctly grouping the embeddings become challenging in an online fashion. Furthermore, errors would accumulate quickly over time if the online clustering algorithm assigns the embeddings into clusters erroneously in the beginning. To address these problems, we designed a novel framework for online clustering. To reduce the high variability of speaker embeddings, we proposed the clustering guided embedding extractor training (CGEET) algorithm to encourage similarity between the size of the embedding space for different speakers in attempt to simplify the distance relationships of embeddings. The CGEET algorithm can grasp the distance information of the entire speaker embedding space and provide it to the online clustering algorithm. With this preliminary information, the distance thresholds guided online clustering (DTGOC) algorithm then processes incoming embeddings using a divide-and-conquer approach. It first handles the embeddings with explicit distance relationships and then searches for possible path combination they have with remaining embeddings in an online fashion. Moreover, in order to utilize the distance relationships of embeddings that are far apart in time, an online re-clustering strategy is incorporated in our DTGOC algorithm, which can alleviate error accumulation during online clustering. By implementing the above innovations, our proposed online clustering system achieves 14.00 DER with collar 0.25 at 2.5 s latency on the AISHELL-4, while the DER of the offline agglomerative hierarchical clustering system is 14.54.

Discriminative Neural Clustering for Speaker Diarisation.

SOT Triggered Neural Clustering for Speaker Attributed ASR

Deep Self-Supervised Hierarchical Clustering for Speaker Diarization

Spectral Clustering-Aware Learning of Embeddings for Speaker Diarisation

Interrelate Training and Searching: A Unified Online Clustering Framework for Speaker Diarization

Supervised Hierarchical Clustering using Graph Neural Networks for Speaker Diarization

A Quick and Effective Speaker Diarization System.

Speaker Diarization Using Latent Space Clustering in Generative Adversarial Network

Assessing the Robustness of Spectral Clustering for Deep Speaker Diarization

Meta-Learning With Latent Space Clustering in Generative Adversarial Network for Speaker Diarization

Overlap-aware End-to-End Supervised Hierarchical Graph Clustering for Speaker Diarization

Highly Efficient Real-Time Streaming and Fully On-Device Speaker Diarization with Multi-Stage Clustering

A Spatial Long-Term Iterative Mask Estimation Approach for Multi-Channel Speaker Diarization and Speech Recognition.

A new DP-like speaker clustering algorithm

Content-Aware Speaker Embeddings for Speaker Diarisation

Towards Unsupervised Speaker Diarization System for Multilingual Telephone Calls Using Pre-trained Whisper Model and Mixture of Sparse Autoencoders

Interrelate Training and Clustering for Online Speaker Diarization

Self-Tuning Spectral Clustering for Speaker Diarization

Speaker Diarization with Lexical Information

Speaker Recognition Based on Pre-Trained Model and Deep Clustering

Combination of Deep Speaker Embeddings for Diarisation