Abstract:Speech enhancement is critical for improving speech intelligibility and quality in various audio devices. In recent years, deep learning-based methods have significantly improved speech enhancement performance, but they often come with a high computational cost, which is prohibitive for a large number of edge devices, such as headsets and hearing aids. This work proposes an ultra-low-power speech enhancement system based on the brain-inspired spiking neural network (SNN) called Spiking-FullSubNet. Spiking-FullSubNet follows a full-band and sub-band fusioned approach to effectively capture both global and local spectral information. To enhance the efficiency of computationally expensive sub-band modeling, we introduce a frequency partitioning method inspired by the sensitivity profile of the human peripheral auditory system. Furthermore, we introduce a novel spiking neuron model that can dynamically control the input information integration and forgetting, enhancing the multi-scale temporal processing capability of SNN, which is critical for speech denoising. Experiments conducted on the recent Intel Neuromorphic Deep Noise Suppression (N-DNS) Challenge dataset show that the Spiking-FullSubNet surpasses state-of-the-art methods by large margins in terms of both speech quality and energy efficiency metrics. Notably, our system won the championship of the Intel N-DNS Challenge (Algorithmic Track), opening up a myriad of opportunities for ultra-low-power speech enhancement at the edge. Our source code and model checkpoints are publicly available at <a class="link-external link-https" href="https://github.com/haoxiangsnr/spiking-fullsubnet" rel="external noopener nofollow">this https URL</a>.

Stochastic Pooling Maxout Networks for Low-Resource Speech Recognition

Convolutional maxout neural networks for low-resource speech recognition

Deep Maxout Neural Networks for Speech Recognition

Maxout Neurons for Deep Convolutional and LSTM Neural Networks in Speech Recognition

Convolutional Maxout Neural Networks for Speech Separation

MaxQ: Multi-Axis Query for N:M Sparsity Network

Max-Pooling Dropout for Regularization of Convolutional Neural Networks

Robust Audio Event Recognition with 1-Max Pooling Convolutional Neural Networks

An eight-layer convolutional neural network with stochastic pooling, batch normalization and dropout for fingerspelling recognition of Chinese sign language

Towards Dropout Training for Convolutional Neural Networks

Combining Max Pooling and ReLU Activation Function in Stochastic Computing

Money on the Table: Statistical information ignored by Softmax can improve classifier accuracy

Stochastic Subsampling With Average Pooling

Improving deep neural networks for LVCSR using dropout and shrinking structure

Advanced Recurrent Network-Based Hybrid Acoustic Models for Low Resource Speech Recognition

One-Class Neural Network With Directed Statistics Pooling for Spoofing Speech Detection

A Maximum Likelihood Approach to SNR-Progressive Learning Using Generalized Gaussian Distribution for LSTM-Based Speech Enhancement.

Variance regularization of RNNLM for speech recognition

Improved Regularization Techniques for End-to-End Speech Recognition

Towards Ultra-Low-Power Neuromorphic Speech Enhancement with Spiking-FullSubNet

A Maximum Likelihood Approach to Multi-Objective Learning Using Generalized Gaussian Distributions for Dnn-Based Speech Enhancement.