Abstract:In recent years, several supervised learning-based approaches have been proposed for estimating the direction of arrival (DOA) of a single talker in noisy and reverberant environments. In the absence of auxiliary information, such as a voice activity detector (VAD), the estimated DOA may be erroneous due to speech pauses or noise dominance. In this paper, we consider a speech-aware DOA estimation system for binaural hearing aids, which does not require a separate VAD. This system utilizes a combination of spatial features with an auditory-inspired periodicity feature called periodicity degree (PD) as input features of a convolutional neural network (CNN). Using speech and non-speech signals during the training, the CNN can capture the harmonic structure encoded in the PD features, thereby distinguishing speech from non-speech portions and simultaneously mapping spatial features to sound source DOA upon speech detection. To investigate the benefit of using PD features for speech-aware DOA estimation, we evaluated the performance of speech-aware systems that utilized either broadband or narrowband feature combinations compared to baseline systems. We propose to use a novel narrowband feature combination consisting of the narrowband cross-power spectrum (CPS) as the spatial feature and a new subband-averaged representation of PD features. The broadband feature combination consisted of the generalized cross-correlation with phase transform (GCC-PHAT) and the broadband PD features. The baseline systems considered in this work consisted of a CNN that exploits only a spatial feature, cascaded with a VAD. Evaluations in reverberant environments with different background noises for both static and dynamic single-talker scenarios demonstrate that incorporating the PD feature in conjunction with any type of spatial feature provides an advantage for binaural DOA estimation in terms of accuracy and angular error.

Speech-Aware Binaural DOA Estimation Utilizing Periodicity and Spatial Features in Convolutional Neural Networks

Spatial Sectorized Neural Network for 2-D DOA Estimation in the Full Azimuth

Neural-Network-Based Direction-of-Arrival Estimation for Reverberant Speech - The Importance of Energetic, Temporal, and Spatial Information

Time-Frequency Bins Selection for Direction of Arrival Estimation Based on Speech Presence Probability Learning

Comparison of Frequency-Fusion Mechanisms for Binaural Direction-of-Arrival Estimation for Multiple Speakers

Assisted RTF-Vector-Based Binaural Direction of Arrival Estimation Exploiting a Calibrated External Microphone Array

Neural Ambisonic Encoding For Multi-Speaker Scenarios Using A Circular Microphone Array

Cognitive-Driven Binaural Beamforming Using EEG-Based Auditory Attention Decoding

Estimating speaker direction on a humanoid robot with binaural acoustic signals

Exploiting an External Microphone for Binaural RTF-Vector-Based Direction of Arrival Estimation for Multiple Speakers

Study of speaker localization with binaural microphone array incorporating auditory filters and lateral angle estimation

Broadband DOA estimation using Convolutional neural networks trained with noise signals

Acoustic Source Localization in the Circular Harmonic Domain Using Deep Learning Architecture

Delay-and-Sum Beamforming Based Spatial Mapping for Multi-Source Sound Localization

Two microphone based direction of arrival estimation for multiple speech sources using spectral properties of speech

Frequency-invariant direction-of-arrival estimation of circular acoustic vector sensor array for wideband signals on the cylindrical baffle

Online Monaural Speech Enhancement Based on Periodicity Analysis and A Priori SNR Estimation

Learning Deep Direct-Path Relative Transfer Function for Binaural Sound Source Localization

Simultaneously Learning Speaker's Direction and Head Orientation from Binaural Recordings

Robust Sound Source Tracking Using SRP-PHAT and 3D Convolutional Neural Networks