Abstract:Sound event detection SED aims to detect when and recognize what sound events happen in an audio clip. Many supervised SED algorithms rely on strongly labelled data that contains the onset and offset annotations of sound events. However, many audio tagging datasets are weakly labelled, that is, only the presence of the sound events is known, without knowing their onset and offset annotations. In this paper, we propose a time–frequency T–F segmentation framework trained on weakly labelled data to tackle the sound event detection and separation problem. In training, a segmentation mapping is applied on a T–F representation, such as log mel spectrogram of an audio clip to obtain T–F segmentation masks of sound events. The T–F segmentation masks can be used for separating the sound events from the background scenes in the T–F domain. Then, a classification mapping is applied on the T–F segmentation masks to estimate the presence probabilities of the sound events. We model the segmentation mapping using a convolutional neural network and the classification mapping using a global weighted rank pooling. In SED, predicted onset and offset times can be obtained from the T–F segmentation masks. As a byproduct, separated waveforms of sound events can be obtained from the T–F segmentation masks. We remixed the DCASE 2018 Task 1 acoustic scene data with the DCASE 2018 Task 2 sound events data. When mixing under 0 dB, the proposed method achieved F1 scores of 0.534, 0.398, and 0.167 in audio tagging, frame-wise SED and event-wise SED, outperforming the fully connected deep neural network baseline of 0.331, 0.237, and 0.120, respectively. In T–F segmentation, we achieved an F1 score of 0.218, where previous methods were not able to do T–F segmentation.

COMBINED SOUND EVENT DETECTION AND SOUND EVENT SEPARATION NETWORKS FOR DCASE 2020 TASK 4 Technical Report

Guided Learning Convolution System for DCASE 2019 Task 4

Sound event detection based on auxiliary decoder and maximum probability aggregation for DCASE Challenge 2024 Task 4

Decoupling Temporal Convolutional Networks Model in Sound Event Detection and Localization

A Multi-Task Learning Framework for Sound Event Detection using High-level Acoustic Characteristics of Sounds

Auditory Neural Response Inspired Sound Event Detection Based on Spectro-temporal Receptive Field

Sound Event Localization and Detection for Real Spatial Sound Scenes: Event-Independent Network and Data Augmentation Chains

Sound Event Detection and Time–Frequency Segmentation from Weakly Labelled Data

Improving Sound Event Localization and Detection with Class-Dependent Sound Separation for Real-World Scenarios

Sound Event Detection and Time-Frequency Segmentation from Weakly Labelled Data

Semi-supervsied Learning-based Sound Event Detection using Freuqency Dynamic Convolution with Large Kernel Attention for DCASE Challenge 2023 Task 4

Polyphonic Sound Event Detection and Localization using a Two-Stage Strategy

Non-Negative Matrix Factorization-Convolutional Neural Network (NMF-CNN) For Sound Event Detection

Joint Analysis of Sound Events and Acoustic Scenes Using Multitask Learning

Exploring Text-Queried Sound Event Detection with Audio Source Separation

FMSG-JLESS Submission for DCASE 2024 Task4 on Sound Event Detection with Heterogeneous Training Dataset and Potentially Missing Labels

Multi-Scale Recurrent Neural Network for Sound Event Detection

Squeeze-and-Excite ResNet-Conformers for Sound Event Localization, Detection, and Distance Estimation for DCASE 2024 Challenge

Sound Event Detection by Consistency Training and Pseudo-Labeling with Feature-Pyramid Convolutional Recurrent Neural Networks

A scene-dependent sound event detection approach using multi-task learning

Cross-Referencing Self-Training Network for Sound Event Detection in Audio Mixtures