Abstract:In recent years, several supervised learning-based approaches have been proposed for estimating the direction of arrival (DOA) of a single talker in noisy and reverberant environments. In the absence of auxiliary information, such as a voice activity detector (VAD), the estimated DOA may be erroneous due to speech pauses or noise dominance. In this paper, we consider a speech-aware DOA estimation system for binaural hearing aids, which does not require a separate VAD. This system utilizes a combination of spatial features with an auditory-inspired periodicity feature called periodicity degree (PD) as input features of a convolutional neural network (CNN). Using speech and non-speech signals during the training, the CNN can capture the harmonic structure encoded in the PD features, thereby distinguishing speech from non-speech portions and simultaneously mapping spatial features to sound source DOA upon speech detection. To investigate the benefit of using PD features for speech-aware DOA estimation, we evaluated the performance of speech-aware systems that utilized either broadband or narrowband feature combinations compared to baseline systems. We propose to use a novel narrowband feature combination consisting of the narrowband cross-power spectrum (CPS) as the spatial feature and a new subband-averaged representation of PD features. The broadband feature combination consisted of the generalized cross-correlation with phase transform (GCC-PHAT) and the broadband PD features. The baseline systems considered in this work consisted of a CNN that exploits only a spatial feature, cascaded with a VAD. Evaluations in reverberant environments with different background noises for both static and dynamic single-talker scenarios demonstrate that incorporating the PD feature in conjunction with any type of spatial feature provides an advantage for binaural DOA estimation in terms of accuracy and angular error.

Speech overlap detection and attribution using convolutive non-negative sparse coding

Frame-Based Overlapping Speech Detection Using Convolutional Neural Networks

Speaker Overlap-aware Neural Diarization for Multi-party Meeting Analysis

Heterogeneous Convolutive Non-Negative Sparse Coding

Speech Enhancement with a GSC-like Structure Employing Sparse Coding

Audio-visual Recognition of Overlapped speech for the LRS2 dataset

Community Detection Graph Convolutional Network for Overlap-Aware Speaker Diarization

Overlapped speech recognition from a jointly learned multi-channel neural speech extraction and representation

Joint Speech Activity and Overlap Detection with Multi-Exit Architecture

BANC: Towards Efficient Binaural Audio Neural Codec for Overlapping Speech

Joint speech and overlap detection: a benchmark over multiple audio setup and speech domains

VADOI:Voice-Activity-Detection Overlapping Inference For End-to-end Long-form Speech Recognition

Online Pattern Learning for Non-Negative Convolutive Sparse Coding.

Adaptive Beamforming Based on Interference-Plus-Noise Covariance Matrix Reconstruction for Speech Separation

Parallel And Hierarchical Decision Making For Sparse Coding In Speech Recognition

Handling Overlaps in Spoken Term Detection

Learning-based Robust Speaker Counting and Separation with the Aid of Spatial Coherence

Overlap-aware End-to-End Supervised Hierarchical Graph Clustering for Speaker Diarization

Speech-Aware Binaural DOA Estimation Utilizing Periodicity and Spatial Features in Convolutional Neural Networks

Target Speaker Extraction for Overlapped Multi-Talker Speaker Verification

Causal Signal-Based DCCRN with Overlapped-Frame Prediction for Online Speech Enhancement