Abstract:The objective of sound event localization and detection (SELD) is to accurately identify the temporal occurrence and spatial coordinates of a specific sound category. The existing mainstream offline methods may unintentionally introduce unfavorable future feature information during the training process, thereby potentially hindering the system's performance. The utilization of online methods can lead to improved localization accuracy to a certain extent. Nevertheless, it may result in a diminished ability with the detection capability for sound events. In this paper, a hybrid offline-online method (HOOM) is proposed that involves extracting comprehensive audio information using offline network layers, while simultaneously filtering out irrelevant future information using online network layers. Based on this method, we designed two simple sub-network architectures. The first, convolution and causal convolution alternating network (CCAN), employs regular convolution along with causal convolutions to achieve the offline and online convolution features, respectively. The second, bidirectional and unidirectional alternating network (BUAN), combines bidirectional recurrent neural networks with unidirectional recurrent neural networks, capturing the offline and online contextual sequence information, respectively. Our proposed method demonstrates a 6% improvement in localization recall on the Sony-TAU Realistic Spatial Soundscapes 2023 (STARSS23) dataset. Furthermore, compared to offline or online methods, there is a 4% overall performance improvement. On the detection and classification of acoustic scenes and events 2022 (DCASE2022) synthetic dataset, the overall performance improvement is 5%. These results indicate a significant advantage and provide a novel and robust solution for the SELD task.

Sound Event Localization and Detection Using a Spatial Omni-Dimensional Dynamic Interactions Network

Joint Spatio-Temporal-Frequency Representation Learning for Improved Sound Event Localization and Detection

Dynamic Kernel Convolution Network with Scene-dedicate Training for Sound Event Localization and Detection

A Study of Improved Two-Stage Dual-Conv Coordinate Attention Model for Sound Event Detection and Localization

A hybrid offline-online method for sound event localization and detection

Sound Event Localization and Detection of Overlapping Sources Using Convolutional Recurrent Neural Networks

Decoupling Temporal Convolutional Networks Model in Sound Event Detection and Localization

Improving Sound Event Localization and Detection with Class-Dependent Sound Separation for Real-World Scenarios

Polyphonic Sound Event Detection and Localization using a Two-Stage Strategy

A Track-Wise Ensemble Event Independent Network for Polyphonic Sound Event Localization and Detection

Sound Event Localization and Detection Based on Multiple DOA Beamforming and Multi-Task Learning

Polyphonic sound event localization and detection using channel-wise FusionNet

A Model Ensemble Approach for Sound Event Localization and Detection.

Improving Polyphonic Sound Event Detection on Multichannel Recordings with the Sørensen-Dice Coefficient Loss and Transfer Learning

Sound Event Localization and Detection for Real Spatial Sound Scenes: Event-Independent Network and Data Augmentation Chains

Fusion of Audio and Visual Embeddings for Sound Event Localization and Detection

Data Augmentation and Squeeze-and-Excitation Network on Multiple Dimension for Sound Event Localization and Detection in Real Scenes

An Experimental Study on Sound Event Localization and Detection under Realistic Testing Conditions

SELD-Mamba: Selective State-Space Model for Sound Event Localization and Detection with Source Distance Estimation

SALSA: Spatial Cue-Augmented Log-Spectrogram Features for Polyphonic Sound Event Localization and Detection

A Generalized Network Based on Multi-Scale Densely Connection and Residual Attention for Sound Source Localization and Detection.