Abstract:Multi-source sound localization can find applications in many domains including auditory scene analysis, fault detection and diagnosis in manufacturing, augmented reality, etc. In far fields, 3D sound source localization is equivalent to finding the direction of arrival (DOA), namely, the azimuth and elevation angles of sound sources. Recent DOA estimation pipelines take multichannel audio inputs, extract spectral features from each channel and then feed them into a deep neural network. Unfortunately, the spectral features contain only the time-frequency information of the audio signals, while spatial information is only implicitly captured in the signals across different channels, which is highly dependent on the acoustic array geometry. To embed the spatial information of the sound source into the spectral feature representation, we propose a DSB-based spatial mapping method encode sound source location information. It can be combined with different feature extraction methods and machine learning models for DOA estimation. Furthermore, a redundancy removal procedure is proposed to accelerate DSB computation so that the pipeline can run in real-time on embedded GPUs, such as NVidia Jeston Nano. We conduct extensive experiments using two neural network models along with the DSB method on two datasets. The experiments demonstrate that the DOA errors can be effectively reduced using the DSB method. When combining DSB for feature extraction, the DOA errors are reduced by up to 19.24%. In addition, the feature extraction process is accelerated by up to 30.42% after the application of redundancy removal.

Direct source and early reflections localization using deep deconvolution network under reverbrate environment

Direct source and early reflections localization using deep deconvolution network under reverberant environment

A robust super-resolution approach with sparsity constraint for near-field wideband acoustic imaging

A High Resolution Bearing Estimation Method Based on Sparse Reconstruction

A Deep Learning Method for DOA Estimation with Covariance Matrices in Reverberant Environments

Joint DOA Estimation and Dereverberation Based on Multi-Channel Linear Prediction Filtering and Azimuth Sparsity

Sound Source Localization in Spherical Harmonics Domain Based on High-Order Ambisonics Signals Enhancement Neural Network

Acoustic Source Localization in a Reverberant Environment Based on Sound Field Morphological Component Analysis and Alternating Direction Method of Multipliers

Acoustic Source Localization in the Circular Harmonic Domain Using Deep Learning Architecture

Enhancing Direct‐path Relative Transfer Function Using Deep Neural Network for Robust Sound Source Localization

Spherical Convolutional Recurrent Neural Network for Real-Time Sound Source Tracking

Sparse DNN Model for Frequency Expanding of Higher Order Ambisonics Encoding Process

Neural-Network-Based Direction-of-Arrival Estimation for Reverberant Speech - The Importance of Energetic, Temporal, and Spatial Information

Closed-form multiple source direction-of-arrival estimator under reverberant environments

Reactive Near-Field to 3-Meter Far-Field Transformation Based on Deep Convolutional Neural Networks and Plane Wave Spectrum

Learning Deep Direct-Path Relative Transfer Function for Binaural Sound Source Localization

Delay-and-Sum Beamforming Based Spatial Mapping for Multi-Source Sound Localization

SRP-DNN: Learning Direct-Path Phase Difference for Multiple Moving Sound Source Localization

Deep learning approach for high-resolution DOA estimation

Direct-Path Signal Cross-Correlation Estimation for Sound Source Localization in Reverberation

Source Localization Using Distributed Microphones in Reverberant Environments Based on Deep Learning and Ray Space Transform