Abstract:Speech enhancement is an important method for improving speech quality and intelligibility in noisy environments. An effective speech enhancement model depends on precise modelling of the long-range dependencies of noisy speech. Several recent studies have examined ways to enhance speech by capturing the long-term contextual information. For speech enhancement, the time–frequency (T–F) distribution of speech spectral components is also important, but is usually ignored in these studies. The multi-stage learning method is an effective way to integrate various deep learning modules at the same time. The benefit of multi-stage training is that the optimization target can be iteratively updated stage by stage. In this paper, speech enhancement is investigated by multi-stage learning using a multi-stage structure in which time–frequency attention (TFA) blocks are followed by stacks of squeezed temporal convolutional networks (S-TCN) with exponentially increasing dilation rates. To reinject original information into later stages, a feature fusion block (FB) is inserted at the input of later stages to reduce the possibility of speech information being lost in the early stages. The S-TCN blocks are responsible for temporal sequence modelling tasks. The time–frequency attention (TFA) is a simple but effective network module that explicitly exploits position information to generate a 2D attention map to characterize the salient T–F distribution of speech by using two branches, time-frame attention and frequency attention in parallel. Extensive experiments have demonstrated that the proposed model consistently improves the performance over existing baselines across two widely used objective metrics such as PESQ and STOI. A significant improvement in system robustness to noise is also shown by our evaluation results using the TFA module.

TFCN: Temporal-Frequential Convolutional Network for Single-Channel Speech Enhancement

Single Channel Speech Enhancement Using Temporal Convolutional Recurrent Neural Networks.

Single Channel Speech Enhancement Based on Temporal Convolutional Network

Monaural Speech Enhancement Using a Multi-Branch Temporal Convolutional Network

Inter-channel Conv-TasNet for multichannel speech enhancement

Dense-TSNet: Dense Connected Two-Stage Structure for Ultra-Lightweight Speech Enhancement

Shortcut-Based Fully Convolutional Network for Speech Enhancement

FLGCNN: A Novel Fully Convolutional Neural Network for End-to-end Monaural Speech Enhancement with Utterance-Based Objective Functions

THLNet: two-stage heterogeneous lightweight network for monaural speech enhancement

Channel and Temporal-Frequency Attention UNet for Monaural Speech Enhancement.

PCNN: A Lightweight Parallel Conformer Neural Network for Efficient Monaural Speech Enhancement

Multi-stage Progressive Learning-Based Speech Enhancement Using Time–Frequency Attentive Squeezed Temporal Convolutional Networks

FB-MSTCN: A Full-Band Single-Channel Speech Enhancement Method Based on Multi-Scale Temporal Convolutional Network

Temporal Convolutional Network with Frequency Dimension Adaptive Attention for Speech Enhancement.

CFTNet: Complex-valued Frequency Transformation Network for Speech Enhancement

A Lightweight Fourier Convolutional Attention Encoder for Multi-Channel Speech Enhancement

An Improved Fully Convolutional Network Based on Post-Processing with Global Variance Equalization and Noise-Aware Training for Speech Enhancement.

Group Multi-Scale convolutional Network for Monaural Speech Enhancement in Time-domain

Tensor-to-Vector Regression for Multi-channel Speech Enhancement based on Tensor-Train Network

2D-to-2d Mask Estimation for Speech Enhancement Based on Fully Convolutional Neural Network

A Two-Stage Frequency-Time Dilated Dense Network for Speech Enhancement