Abstract:Sound event localization and detection (SELD) has seen substantial advancements through learning-based methods. These systems, typically trained from scratch on specific datasets, have shown considerable generalization capabilities. Recently, deep neural networks trained on large-scale datasets have achieved remarkable success in the sound event classification (SEC) field, prompting an open question of whether these advancements can be extended to develop general-purpose SELD models. In this paper, leveraging the power of pre-trained SEC models, we propose pre-trained SELD networks (PSELDNets) on large-scale synthetic datasets. These synthetic datasets, generated by convolving sound events with simulated spatial room impulse responses (SRIRs), contain 1,167 hours of audio clips with an ontology of 170 sound classes. These PSELDNets are transferred to downstream SELD tasks. When we adapt PSELDNets to specific scenarios, particularly in low-resource data cases, we introduce a data-efficient fine-tuning method, AdapterBit. PSELDNets are evaluated on a synthetic-test-set using collected SRIRs from TAU Spatial Room Impulse Response Database (TAU-SRIR DB) and achieve satisfactory performance. We also conduct our experiments to validate the transferability of PSELDNets to three publicly available datasets and our own collected audio recordings. Results demonstrate that PSELDNets surpass state-of-the-art systems across all publicly available datasets. Given the need for direction-of-arrival estimation, SELD generally relies on sufficient multi-channel audio clips. However, incorporating the AdapterBit, PSELDNets show more efficient adaptability to various tasks using minimal multi-channel or even just monophonic audio clips, outperforming the traditional fine-tuning approaches.

SECL-UMons Database for Sound Event Classification and Localization

A Dataset of Reverberant Spatial Sound Scenes with Moving Sources for Sound Event Localization and Detection

Sound Event Localization and Detection for Real Spatial Sound Scenes: Event-Independent Network and Data Augmentation Chains

Sound source detection, localization and classification using consecutive ensemble of CRNN models

A Benchmark of State-of-the-Art Sound Event Detection Systems Evaluated on Synthetic Soundscapes

Two vs. Four-Channel Sound Event Localization and Detection

Large-Scale Weakly Labeled Semi-Supervised Sound Event Detection in Domestic Environments

Overview and Evaluation of Sound Event Localization and Detection in DCASE 2019

DCASE 2024 Task 4: Sound Event Detection with Heterogeneous Data and Missing Labels

Sound Event Detection in Synthetic Audio: Analysis of the DCASE 2016 Task Results

FMSG-JLESS Submission for DCASE 2024 Task4 on Sound Event Detection with Heterogeneous Training Dataset and Potentially Missing Labels

Sound Event Detection in Synthetic Domestic Environments

CoLoC: Conditioned Localizer and Classifier for Sound Event Localization and Detection

Data Augmentation and Squeeze-and-Excitation Network on Multiple Dimension for Sound Event Localization and Detection in Real Scenes

Sound Event Localization and Detection Using Imbalanced Real and Synthetic Data via Multi-Generator

Training Sound Event Detection On A Heterogeneous Dataset

Dynamic Kernel Convolution Network with Scene-dedicate Training for Sound Event Localization and Detection

Improving Sound Event Localization and Detection with Class-Dependent Sound Separation for Real-World Scenarios

PSELDNets: Pre-trained Neural Networks on Large-scale Synthetic Datasets for Sound Event Localization and Detection

ICASSP 2022 L3DAS22 Challenge: Ensemble of Resnet-Conformers with Ambisonics Data Augmentation for Sound Event Localization and Detection