Abstract:The objective of sound event localization and detection (SELD) is to accurately identify the temporal occurrence and spatial coordinates of a specific sound category. The existing mainstream offline methods may unintentionally introduce unfavorable future feature information during the training process, thereby potentially hindering the system's performance. The utilization of online methods can lead to improved localization accuracy to a certain extent. Nevertheless, it may result in a diminished ability with the detection capability for sound events. In this paper, a hybrid offline-online method (HOOM) is proposed that involves extracting comprehensive audio information using offline network layers, while simultaneously filtering out irrelevant future information using online network layers. Based on this method, we designed two simple sub-network architectures. The first, convolution and causal convolution alternating network (CCAN), employs regular convolution along with causal convolutions to achieve the offline and online convolution features, respectively. The second, bidirectional and unidirectional alternating network (BUAN), combines bidirectional recurrent neural networks with unidirectional recurrent neural networks, capturing the offline and online contextual sequence information, respectively. Our proposed method demonstrates a 6% improvement in localization recall on the Sony-TAU Realistic Spatial Soundscapes 2023 (STARSS23) dataset. Furthermore, compared to offline or online methods, there is a 4% overall performance improvement. On the detection and classification of acoustic scenes and events 2022 (DCASE2022) synthetic dataset, the overall performance improvement is 5%. These results indicate a significant advantage and provide a novel and robust solution for the SELD task.

Sound Event Localization and Detection Based on Deep Learning

Joint Spatio-Temporal-Frequency Representation Learning for Improved Sound Event Localization and Detection

Sound Event Localization and Detection of Overlapping Sources Using Convolutional Recurrent Neural Networks

Sound Event Localization and Detection Based on Multiple DOA Beamforming and Multi-Task Learning

A Model Ensemble Approach for Sound Event Localization and Detection.

Attention mechanism combined with residual recurrent neural network for sound event detection and localization

Sound Event Localization and Detection Using Parallel Multi-attention Enhancement

Polyphonic Sound Event Detection and Localization using a Two-Stage Strategy

Dynamic Kernel Convolution Network with Scene-dedicate Training for Sound Event Localization and Detection

Improving Sound Event Localization and Detection with Class-Dependent Sound Separation for Real-World Scenarios

A Study of Improved Two-Stage Dual-Conv Coordinate Attention Model for Sound Event Detection and Localization

Sound Event Localization and Detection Using a Spatial Omni-Dimensional Dynamic Interactions Network

The NERCSLIP-USTC System for the L3DAS23 Challenge Task2: 3D Sound Event Localization and Detection (SELD)

Sound Event Localization and Detection Based on Iterative Separation in Embedding Space

Sound Event Detection and Localization with Distance Estimation

A hybrid offline-online method for sound event localization and detection

Polyphonic sound event localization and detection using channel-wise FusionNet

Decoupling Temporal Convolutional Networks Model in Sound Event Detection and Localization

DOA-Aware Audio-Visual Self-Supervised Learning for Sound Event Localization and Detection

PSELDNets: Pre-trained Neural Networks on Large-scale Synthetic Datasets for Sound Event Localization and Detection

A Polyphonic SELD Network Based on Attentive Feature Fusion and Multi-stage Training Strategy