Abstract:Although fully end-to-end speaker diarization systems have made significant progress in recent years, modular systems often achieve superior results in real-world scenarios due to their greater adaptability and robustness. Historically, modular speaker diarization methods have seldom discussed how to leverage spatial cues from multi-channel speech. This paper proposes a three-stage modular system to enhance single-channel neural speaker diarization systems and recognition performance by utilizing spatial cues from multi-channel speech to provide more accurate initialization for each stage of neural speaker diarization (NSD) decoding: (1) Overlap detection and continuous speech separation (CSS) on multi-channel speech are used to obtain cleaner single speaker speech segments for clustering, followed by the first NSD decoding pass. (2) The results from the first pass initialize a complex Angular Central Gaussian Mixture Model (cACGMM) to estimate speaker-wise masks on multi-channel speech, and through Overlap-add and Mask-to-VAD, achieve initialization with lower speaker error (SpkErr), followed by the second NSD decoding pass. (3) The second decoding results are used for guided source separation (GSS), recognizing and filtering short segments containing less one word to obtain cleaner speech segments, followed by re-clustering and the final NSD decoding pass. We presented the progressively explored evaluation results from the CHiME-8 NOTSOFAR-1 (Natural Office Talkers in Settings Of Far-field Audio Recordings) challenge, demonstrating the effectiveness of our system and its contribution to improving recognition performance. Our final system achieved the first place in the challenge.

Spatial-aware Speaker Diarizaiton for Multi-channel Multi-party Meeting

The xmuspeech system for multi-channel multi-party meeting transcription challenge

Incorporating Spatial Cues in Modular Speaker Diarization for Multi-channel Multi-party Meetings

Enhancing Low-Latency Speaker Diarization with Spatial Dictionary Learning.

A Real-time Speaker Diarization System Based on Spatial Spectrum

Spatial-Temporal Activity-Informed Diarization and Separation

Spatial Diarization for Meeting Transcription with Ad-Hoc Acoustic Sensor Networks

A Spatial Long-Term Iterative Mask Estimation Approach for Multi-Channel Speaker Diarization and Speech Recognition.

Multisensory Fusion for Unsupervised Spatiotemporal Speaker Diarization

Speaker Embedding-aware Neural Diarization: a Novel Framework for Overlapped Speech Diarization in the Meeting Scenario

Speaker Overlap-aware Neural Diarization for Multi-party Meeting Analysis

ANSD-MA-MSE: Adaptive Neural Speaker Diarization Using Memory-Aware Multi-Speaker Embedding.

Investigation of Spatial-Acoustic Features for Overlapping Speech Detection in Multiparty Meetings

ASoBO: Attentive Beamformer Selection for Distant Speaker Diarization in Meetings

Multi-channel Conversational Speaker Separation via Neural Diarization

An Improved Speaker Diarization System for Multiple Distance Microphone Meetings

Speaker-turn aware diarization for speech-based cognitive assessments

Neural Speaker Diarization Using Memory-Aware Multi-Speaker Embedding with Sequence-to-Sequence Architecture

Online speaker diarization of meetings guided by speech separation

One model to rule them all ? Towards End-to-End Joint Speaker Diarization and Speech Recognition

Exploring Speaker-Related Information in Spoken Language Understanding for Better Speaker Diarization