Abstract:The complexity of polyphonic sounds imposes numerous challenges on their classification. Especially in real life, polyphonic sound events have discontinuity and unstable time-frequency variations. Traditional single acoustic features cannot characterize the key feature information of the polyphonic sound event, and this deficiency results in poor model classification performance. In this paper, we propose a convolutional recurrent neural network model based on the temporal-frequency (TF) attention mechanism and feature space (FS) attention mechanism (TFFS-CRNN). The TFFS-CRNN model aggregates Log-Mel spectrograms and MFCCs feature as inputs, which contains the TF-attention module, the convolutional recurrent neural network (CRNN) module, the FS-attention module and the bidirectional gated recurrent unit (BGRU) module. In polyphonic sound events detection (SED), the TF-attention module can capture the critical temporal-frequency features more capably. The FS-attention module assigns different dynamically learnable weights to different dimensions of features. The TFFS-CRNN model improves the characterization of features for key feature information in polyphonic SED. By using two attention modules, the model can focus on semantically relevant time frames, key frequency bands, and important feature spaces. Finally, the BGRU module learns contextual information. The experiments were conducted on the DCASE 2016 Task3 dataset and the DCASE 2017 Task3 dataset. Experimental results show that the F1-score of the TFFS-CRNN model improved 12.4% and 25.2% compared with winning system models in DCASE challenge; the ER is reduced by 0.41 and 0.37 as well. The proposed TFFS-CRNN model algorithm has better classification performance and lower ER in polyphonic SED.

Convolutional Recurrent Neural Networks for Polyphonic Sound Event Detection

End-to-End Polyphonic Sound Event Detection Using Convolutional Recurrent Neural Networks with Learned Time-Frequency Representation Input

Multi-scale Convolutional Recurrent Neural Network and Data Augmentation for Polyphonic Sound Event Detection

MTF-CRNN: Multiscale Time-Frequency Convolutional Recurrent Neural Network for Sound Event Detection.

Convolutional Recurrent Neural Networks with Multi-Sized Convolution Filters for Sound-Event Recognition

Relational Recurrent Neural Networks for Polyphonic Sound Event Detection

Robust Sound Event Recognition Using Convolutional Neural Networks

Sound event localization and detection based on crnn using rectangular filters and channel rotation data augmentation

Sound Event Detection Using Spatial Features and Convolutional Recurrent Neural Network

U Recurrent Neural Network for Polyphonic Sound Event Detection and Localization

Convolutional bidirectional long short-term memory hidden Markov model hybrid system for polyphonic sound event detection

Multi-Scale Convolutional Recurrent Neural Network with Ensemble Method for Weakly Labeled Sound Event Detection

Fully Convolutional DenseNet Based Polyphonic Sound Event Detection

Polyphonic Sound Event Detection Using Temporal-Frequency Attention and Feature Space Attention

Robust Sound Event Classification With Local Time-Frequency Information And Convolutional Neural Networks

Sound Event Detection Based on Bidirectional Temporal Convolutional Network and Gated Recurrent Unit

Sound Event Detection Using Multi-Scale Dense Convolutional Recurrent Neural Network with Lightweight Attention

Sound Event Detection Via Conformer Recurrent Neural Networks

Early Detection of Continuous and Partial Audio Events Using CNN

NAS-DYMC: NAS-Based Dynamic Multi-Scale Convolutional Neural Network for Sound Event Detection

Multi-Scale Recurrent Neural Network for Sound Event Detection