Abstract:Although fully end-to-end speaker diarization systems have made significant progress in recent years, modular systems often achieve superior results in real-world scenarios due to their greater adaptability and robustness. Historically, modular speaker diarization methods have seldom discussed how to leverage spatial cues from multi-channel speech. This paper proposes a three-stage modular system to enhance single-channel neural speaker diarization systems and recognition performance by utilizing spatial cues from multi-channel speech to provide more accurate initialization for each stage of neural speaker diarization (NSD) decoding: (1) Overlap detection and continuous speech separation (CSS) on multi-channel speech are used to obtain cleaner single speaker speech segments for clustering, followed by the first NSD decoding pass. (2) The results from the first pass initialize a complex Angular Central Gaussian Mixture Model (cACGMM) to estimate speaker-wise masks on multi-channel speech, and through Overlap-add and Mask-to-VAD, achieve initialization with lower speaker error (SpkErr), followed by the second NSD decoding pass. (3) The second decoding results are used for guided source separation (GSS), recognizing and filtering short segments containing less one word to obtain cleaner speech segments, followed by re-clustering and the final NSD decoding pass. We presented the progressively explored evaluation results from the CHiME-8 NOTSOFAR-1 (Natural Office Talkers in Settings Of Far-field Audio Recordings) challenge, demonstrating the effectiveness of our system and its contribution to improving recognition performance. Our final system achieved the first place in the challenge.

Speaker Detection in the Wild: Lessons Learned from JSALT 2019

Enhancement And Analysis Of Conversational Speech: Jsalt 2017

Exploring Speaker-Related Information in Spoken Language Understanding for Better Speaker Diarization

A Real-time Speaker Diarization System Based on Spatial Spectrum

INTEGRATION OF SPEECH SEPARATION, DIARIZATION, AND RECOGNITION FOR MULTI-SPEAKER MEETINGS: SYSTEM DESCRIPTION, COMPARISON, AND ANALYSIS

Real-time Architecture for Audio-Visual Active Speaker Detection.

One model to rule them all ? Towards End-to-End Joint Speaker Diarization and Speech Recognition

Incorporating Spatial Cues in Modular Speaker Diarization for Multi-channel Multi-party Meetings

Self-Supervised Vision-Based Detection of the Active Speaker as Support for Socially-Aware Language Acquisition

A Real-Time Active Speaker Detection System Integrating an Audio-Visual Signal with a Spatial Querying Mechanism

A Spatial Long-Term Iterative Mask Estimation Approach for Multi-Channel Speaker Diarization and Speech Recognition.

Microsoft Speaker Diarization System for the VoxCeleb Speaker Recognition Challenge 2020

A Toolkit for Joint Speaker Diarization and Identification with Application to Speaker-Attributed ASR

A two-stage speaker extraction algorithm under adverse acoustic conditions using a single-microphone

ASoBO: Attentive Beamformer Selection for Distant Speaker Diarization in Meetings

Visually Supervised Speaker Detection and Localization via Microphone Array

Investigating Various Diarization Algorithms for Speaker in the Wild (SITW) Speaker Recognition Challenge

Spatial Diarization for Meeting Transcription with Ad-Hoc Acoustic Sensor Networks

Exploring Detection-based Method For Speaker Diarization @ Ego4D Audio-only Diarization Challenge 2022

Improving Speaker Assignment in Speaker-Attributed ASR for Real Meeting Applications

Speaker Diarization and Identification from Single-Channel Classroom Audio Recording Using Virtual Microphones