Abstract:In recent years, several supervised learning-based approaches have been proposed for estimating the direction of arrival (DOA) of a single talker in noisy and reverberant environments. In the absence of auxiliary information, such as a voice activity detector (VAD), the estimated DOA may be erroneous due to speech pauses or noise dominance. In this paper, we consider a speech-aware DOA estimation system for binaural hearing aids, which does not require a separate VAD. This system utilizes a combination of spatial features with an auditory-inspired periodicity feature called periodicity degree (PD) as input features of a convolutional neural network (CNN). Using speech and non-speech signals during the training, the CNN can capture the harmonic structure encoded in the PD features, thereby distinguishing speech from non-speech portions and simultaneously mapping spatial features to sound source DOA upon speech detection. To investigate the benefit of using PD features for speech-aware DOA estimation, we evaluated the performance of speech-aware systems that utilized either broadband or narrowband feature combinations compared to baseline systems. We propose to use a novel narrowband feature combination consisting of the narrowband cross-power spectrum (CPS) as the spatial feature and a new subband-averaged representation of PD features. The broadband feature combination consisted of the generalized cross-correlation with phase transform (GCC-PHAT) and the broadband PD features. The baseline systems considered in this work consisted of a CNN that exploits only a spatial feature, cascaded with a VAD. Evaluations in reverberant environments with different background noises for both static and dynamic single-talker scenarios demonstrate that incorporating the PD feature in conjunction with any type of spatial feature provides an advantage for binaural DOA estimation in terms of accuracy and angular error.

Estimating speaker direction on a humanoid robot with binaural acoustic signals

Estimating speaker direction on a humanoid robot with binaural acoustic signals

Direction of Arrival Estimation Using Microphone Array Processing for Moving Humanoid Robots

Acoustic DOA Estimation Using Space Alternating Sparse Bayesian Learning

Vision-Guided Robot Hearing

Study of speaker localization with binaural microphone array incorporating auditory filters and lateral angle estimation

Sound source localization for robotics by binaural methods

Efficient and Robust Approaches for Three-Dimensional Sound Source Recognition and Localization Using Humanoid Robots Sensor Arrays

Sound source localization for auditory perception of a humanoid robot using deep neural networks

Enhanced Robot Speech Recognition Using Biomimetic Binaural Sound Source Localization

Speech-Aware Binaural DOA Estimation Utilizing Periodicity and Spatial Features in Convolutional Neural Networks

Audio-Visual Bimodal Combination-Based Speaker Tracking Method for Mobile Robot

Simultaneously Learning Speaker's Direction and Head Orientation from Binaural Recordings

Spatial Direction Estimation for Multiple Sound Sources in Reverberation Environment

Binaural multiple sources localization with extracting high local SNR frequencies based on cepstrum

Spatial Localization of Concurrent Multiple Sound Sources Using Phase Candidate Histogram

Exploiting an External Microphone for Binaural RTF-Vector-Based Direction of Arrival Estimation for Multiple Speakers

Towards Real-Time 3D Sound Sources Mapping with Linear Microphone Arrays.

Assisted RTF-Vector-Based Binaural Direction of Arrival Estimation Exploiting a Calibrated External Microphone Array

Neural-Network-Based Direction-of-Arrival Estimation for Reverberant Speech - The Importance of Energetic, Temporal, and Spatial Information

Improved direction of arrival estimations with a wearable microphone array for dynamic environments by reliability weighting