Abstract:Environmental sound classiﬁcation (ESC) is a challenging problem due to the unstructured spatial-temporal relations that exist in the sound signals. Re-cently, many studies have focused on abstracting features from convolutional neural networks while the learning of semantically relevant frames of sound signals has been overlooked. To this end, we present an end-to-end framework, namely feature pyramid attention network (FPAM), focusing on abstracting the semantically relevant features for ESC. We ﬁrst extract the feature maps of the preprocessed spectrogram of the sound waveform by a backbone network. Then, to build multi-scale hierarchical features of sound spectrograms, we construct a feature pyramid representation of the sound spectrograms by aggregating the feature maps from multi-scale layers, where the temporal frames and spatial locations of semantically relevant frames are localized by FPAM. Speciﬁcally, the multiple features are ﬁrst processed by a dimension alignment module. Af-terward, the pyramid spatial attention module (PSA) is attached to localize the important frequency regions spatially with a spatial attention module (SAM). Last, the processed feature maps are reﬁned by a pyramid channel attention (PCA) to localize the important temporal frames. To justify the eﬀectiveness of the proposed FPAM, visualization of attention maps on the spectrograms has been presented. The visualization results show that FPAM can focus more on the semantic relevant regions while neglecting the noises. The eﬀectiveness of the proposed methods is validated on two widely used ESC datasets: the ESC-50 and ESC-10 datasets. The experimental results show that the FPAM yields comparable performance to state-of-the-art methods. A substantial performance increase has been achieved by FPAM compared with the baseline methods.

Multi-stream Network With Temporal Attention For Environmental Sound Classification

Automatic Respiratory Sound Classification Via Multi-Branch Temporal Convolutional Network

Environmental Sound Classification Based on Multi-temporal Resolution Convolutional Neural Network Combining with Multi-level Features

Attention based Convolutional Recurrent Neural Network for Environmental Sound Classification

MSARN: A Multi-scale Attention Residual Network for End-to-End Environmental Sound Classification

Environment Sound Classification using Multiple Feature Channels and Attention based Deep Convolutional Neural Network

Spatio-Temporal Attention Pooling for Audio Scene Classification

Robust Audio Sensing with Multi-Sound Classification.

Feature Pyramid Attention based Residual Neural Network for Environmental Sound Classification

Learning Frame Level Attention for Environmental Sound Classification

Environmental Sound Classification Via Time–Frequency Attention and Framewise Self-Attention-Based Deep Neural Networks

End-to-End Environmental Sound Classification using a 1D Convolutional Neural Network

A Multi-Spike Approach For Robust Sound Recognition

Multi-Modal Attention Mechanisms In Lstm And Its Application To Acoustic Scene Classification

Research on Environmental Sound Recognition Method based on Residual Network with Dual Attention Mechanism

Multi-Temporal Resolution Convolutional Neural Networks for Acoustic Scene Classification

Environmental Sound Recognition Based on Double-input Convolutional Neural Network Model

Deep Convolutional Neural Network with Mixup for Environmental Sound Classification

Environmental sound analysis with mixup based multitask learning and cross-task fusion

Look and Listen: A Multi-modality Late Fusion Approach to Scene Classification for Autonomous Machines

Rethinking environmental sound classification using convolutional neural networks: optimized parameter tuning of single feature extraction