Abstract:Speech enhancement is an important method for improving speech quality and intelligibility in noisy environments. An effective speech enhancement model depends on precise modelling of the long-range dependencies of noisy speech. Several recent studies have examined ways to enhance speech by capturing the long-term contextual information. For speech enhancement, the time–frequency (T–F) distribution of speech spectral components is also important, but is usually ignored in these studies. The multi-stage learning method is an effective way to integrate various deep learning modules at the same time. The benefit of multi-stage training is that the optimization target can be iteratively updated stage by stage. In this paper, speech enhancement is investigated by multi-stage learning using a multi-stage structure in which time–frequency attention (TFA) blocks are followed by stacks of squeezed temporal convolutional networks (S-TCN) with exponentially increasing dilation rates. To reinject original information into later stages, a feature fusion block (FB) is inserted at the input of later stages to reduce the possibility of speech information being lost in the early stages. The S-TCN blocks are responsible for temporal sequence modelling tasks. The time–frequency attention (TFA) is a simple but effective network module that explicitly exploits position information to generate a 2D attention map to characterize the salient T–F distribution of speech by using two branches, time-frame attention and frequency attention in parallel. Extensive experiments have demonstrated that the proposed model consistently improves the performance over existing baselines across two widely used objective metrics such as PESQ and STOI. A significant improvement in system robustness to noise is also shown by our evaluation results using the TFA module.

Multi-Scale Temporal Frequency Convolutional Network With Axial Attention for Speech Enhancement

Densely Connected Multi-Stage Model with Channel Wise Subband Feature for Real-Time Speech Enhancement.

Supervised Attention Multi-Scale Temporal Convolutional Network for monaural speech enhancement

FB-MSTCN: A Full-Band Single-Channel Speech Enhancement Method Based on Multi-Scale Temporal Convolutional Network

Acoustic Echo Cancellation and Noise Suppression with a Full Time-Frequency Cascaded Neural Network

Stacked Multiscale Densely Connected Temporal Convolutional Attention Network for Multi-Objective Speech Enhancement in an Airborne Environment

Double Branches and Stages Neural Network for Joint Acoustic Echo and Noise Suppression

A Multi-Stage Acoustic Echo Cancellation Model Based on Adaptive Filters and Deep Neural Networks

A Robust and Cascaded Acoustic Echo Cancellation Based on Deep Learning

A Deep Hierarchical Fusion Network for Fullband Acoustic Echo Cancellation

Multi-Stage Progressive Speech Enhancement Network

Convolutional Recurrent Neural Network with Attention for 3D Speech Enhancement

Speech Enhancement Using Multi-Stage Self-Attentive Temporal Convolutional Networks

Icassp 2021 acoustic echo cancellation challenge: integrated adaptive echo cancellation with time alignment and deep learning-based residual echo plus noise suppression

Channel and Temporal-Frequency Attention UNet for Monaural Speech Enhancement.

Improving Acoustic Echo Cancellation by Exploring Speech and Echo Affinity with Multi-Head Attention.

Deep Multi-task Network for Delay Estimation and Echo Cancellation

Speech Enhancement Model for High Sampling Rate Speech Datasets Based on Multi-branch Time Convolutional Network

Multi-stage Progressive Learning-Based Speech Enhancement Using Time–Frequency Attentive Squeezed Temporal Convolutional Networks

End-to-End Complex-Valued Multidilated Convolutional Neural Network for Joint Acoustic Echo Cancellation and Noise Suppression

SpatialNet: Extensively Learning Spatial Information for Multichannel Joint Speech Separation, Denoising and Dereverberation