Abstract:In this paper, we propose a convolutional recurrent neural network for joint sound event localization and detection (SELD) of multiple overlapping sound events in three-dimensional (3-D) space. The proposed network takes a sequence of consecutive spectrogram time frames as input and maps it to two outputs in parallel. As the first output, the sound event detection (SED) is performed as a multi-label classification task on each time frame producing temporal activity for all the sound event classes. As the second output, localization is performed by estimating the 3-D Cartesian coordinates of the direction-of-arrival (DOA) for each sound event class using multi-output regression. The proposed method is able to associate multiple DOAs with respective sound event labels and further track this association with respect to time. The proposed method uses separately the phase and magnitude component of the spectrogram calculated on each audio channel as the feature, thereby avoiding any method- and array-specific feature extraction. The method is evaluated on five Ambisonic and two circular array format datasets with different overlapping sound events in anechoic, reverberant, and real-life scenarios. The proposed method is compared with two SED, three DOA estimation, and one SELD baselines. The results show that the proposed method is generic and applicable to any array structures, robust to unseen DOA values, reverberation, and low SNR scenarios. The proposed method achieved a consistently higher recall of the estimated number of DOAs across datasets in comparison to the best baseline. Additionally, this recall was observed to be significantly better than the best baseline method for a higher number of overlapping sound events.

A report on sound event detection with different binaural features

Sound Event Detection Using Spatial Features and Convolutional Recurrent Neural Network

Sound Event Detection in Multichannel Audio Using Spatial and Harmonic Features

Binaural Signal Representations for Joint Sound Event Detection and Acoustic Scene Classification

Sound Event Localization and Detection of Overlapping Sources Using Convolutional Recurrent Neural Networks

MTF-CRNN: Multiscale Time-Frequency Convolutional Recurrent Neural Network for Sound Event Detection.

Convolutional Recurrent Neural Networks for Polyphonic Sound Event Detection

Sound event localization and detection based on crnn using rectangular filters and channel rotation data augmentation

Robust Audio Sensing with Multi-Sound Classification.

A novel hybrid ensemble approach to enhance the acoustic event classification in environmental sound analysis

Improving Polyphonic Sound Event Detection on Multichannel Recordings with the Sørensen-Dice Coefficient Loss and Transfer Learning

Overview and Evaluation of Sound Event Localization and Detection in DCASE 2019

A two-step system for sound event localization and detection

A hybrid parametric-deep learning approach for sound event localization and detection

End-to-End Polyphonic Sound Event Detection Using Convolutional Recurrent Neural Networks with Learned Time-Frequency Representation Input

Binaural Rendering of Ambisonic Signals by Neural Networks

Sound Event Detection for Human Safety and Security in Noisy Environments

Audio Event Recognition Based on DBN Features from Multiple Filter-Bank Representations.

The impact of non-target events in synthetic soundscapes for sound event detection

Sound event detection using weakly labeled dataset with stacked convolutional and recurrent neural network

A research for sound event localization and detection based on local–global adaptive fusion and temporal importance network