Abstract:Although fully end-to-end speaker diarization systems have made significant progress in recent years, modular systems often achieve superior results in real-world scenarios due to their greater adaptability and robustness. Historically, modular speaker diarization methods have seldom discussed how to leverage spatial cues from multi-channel speech. This paper proposes a three-stage modular system to enhance single-channel neural speaker diarization systems and recognition performance by utilizing spatial cues from multi-channel speech to provide more accurate initialization for each stage of neural speaker diarization (NSD) decoding: (1) Overlap detection and continuous speech separation (CSS) on multi-channel speech are used to obtain cleaner single speaker speech segments for clustering, followed by the first NSD decoding pass. (2) The results from the first pass initialize a complex Angular Central Gaussian Mixture Model (cACGMM) to estimate speaker-wise masks on multi-channel speech, and through Overlap-add and Mask-to-VAD, achieve initialization with lower speaker error (SpkErr), followed by the second NSD decoding pass. (3) The second decoding results are used for guided source separation (GSS), recognizing and filtering short segments containing less one word to obtain cleaner speech segments, followed by re-clustering and the final NSD decoding pass. We presented the progressively explored evaluation results from the CHiME-8 NOTSOFAR-1 (Natural Office Talkers in Settings Of Far-field Audio Recordings) challenge, demonstrating the effectiveness of our system and its contribution to improving recognition performance. Our final system achieved the first place in the challenge.

Speaker Overlap-aware Neural Diarization for Multi-party Meeting Analysis

Speaker Embedding-aware Neural Diarization: a Novel Framework for Overlapped Speech Diarization in the Meeting Scenario

TOLD: A Novel Two-Stage Overlap-Aware Framework for Speaker Diarization

Incorporating Spatial Cues in Modular Speaker Diarization for Multi-channel Multi-party Meetings

Speaker-turn aware diarization for speech-based cognitive assessments

Semi-supervised multi-channel speaker diarization with cross-channel attention

Online speaker diarization of meetings guided by speech separation

Sequence-to-Sequence Neural Diarization with Automatic Speaker Detection and Representation

ANSD-MA-MSE: Adaptive Neural Speaker Diarization Using Memory-Aware Multi-Speaker Embedding.

End-to-end speaker segmentation for overlap-aware resegmentation

Speech overlap detection and attribution using convolutive non-negative sparse coding

Community Detection Graph Convolutional Network for Overlap-Aware Speaker Diarization

Investigation of Spatial-Acoustic Features for Overlapping Speech Detection in Multiparty Meetings

Neural Speaker Diarization Using Memory-Aware Multi-Speaker Embedding with Sequence-to-Sequence Architecture

Unified Modeling of Multi-Talker Overlapped Speech Recognition and Diarization with a Sidecar Separator

Overlap-aware End-to-End Supervised Hierarchical Graph Clustering for Speaker Diarization

Multi-channel Conversational Speaker Separation via Neural Diarization

Towards Word-Level End-to-End Neural Speaker Diarization with Auxiliary Network

A Real-time Speaker Diarization System Based on Spatial Spectrum

Exploring Speaker-Related Information in Spoken Language Understanding for Better Speaker Diarization

Integrating Audio, Visual, and Semantic Information for Enhanced Multimodal Speaker Diarization