Abstract:The capability for environmental sound recognition (ESR) can determine the fitness of individuals in a way to avoid dangers or pursue opportunities when critical sound events occur. It still remains mysterious about the fundamental principles of biological systems that result in such a remarkable ability. Additionally, the practical importance of ESR has attracted an increasing amount of research attention, but the chaotic and nonstationary difficulties continue to make it a challenging task. In this article, we propose a spike-based framework from a more brain-like perspective for the ESR task. Our framework is a unifying system with consistent integration of three major functional parts which are sparse encoding, efficient learning, and robust readout. We first introduce a simple sparse encoding, where key points are used for feature representation, and demonstrate its generalization to both spike- and nonspike-based systems. Then, we evaluate the learning properties of different learning rules in detail with our contributions being added for improvements. Our results highlight the advantages of multispike learning, providing a selection reference for various spike-based developments. Finally, we combine the multispike readout with the other parts to form a system for ESR. Experimental results show that our framework performs the best as compared to other baseline approaches. In addition, we show that our spike-based framework has several advantageous characteristics including early decision making, small dataset acquiring, and ongoing dynamic processing. Our framework is the first attempt to apply the multispike characteristic of nervous neurons to ESR. The outstanding performance of our approach would potentially contribute to draw more research efforts to push the boundaries of spike-based paradigm to a new horizon.

Temporal Coding of Local Spectrogram Features for Robust Sound Recognition

Spike-based Encoding and Learning of Spectrum Features for Robust Sound Recognition.

A Multi-Spike Approach For Robust Sound Recognition

Adaptive Temporal Encoding Leads to a Background-Insensitive Cortical Representation of Speech

A Spiking Neural Network Model for Sound Recognition.

Sparse Temporal Encoding of Visual Features for Robust Object Recognition by Spiking Neurons

A Spiking Neural Network System for Robust Sequence Recognition

Bipolar Population Threshold Encoding for Audio Recognition with Deep Spiking Neural Networks

Robust Environmental Sound Recognition with Sparse Key-point Encoding and Efficient Multi-spike Learning.

A Temporal Encoding Method Based on Expansion Representation

Robust Multipitch Estimation Of Piano Sounds Using Deep Spiking Neural Networks

Robust Audio Sensing with Multi-Sound Classification.

Attention-Based Deep Spiking Neural Networks for Temporal Credit Assignment Problems.

Sound source localization method based time-domain signal feature using deep learning

Robust sound event classification using deep neural networks

A Hybrid Learning Framework for Deep Spiking Neural Networks with One-Spike Temporal Coding

Joint Spatio-Temporal-Frequency Representation Learning for Improved Sound Event Localization and Detection

Robust Feature Learning on Long-Duration Sounds for Acoustic Scene Classification

Audio Recognition using Mel Spectrograms and Convolution Neural Networks

Spatiotemporal audio feature extraction with dynamic memristor-based time-surface neurons

Neural Population Coding for Effective Temporal Classification