Abstract:The recent development of speech enhancement methods has incorporated attention mechanisms for learning long-term speech signal dependencies. The utilization of deep convolution networks (DCN) equipped with the self-attention (SA) and transformers has showed promising results in speech enhancement (SE). While self-attention networks excel in extracting significant long-sequence contextual information in mining tasks, they may not effectively concentrate on subtle aspects within speech signals. These subtle details include temporal or spectral continuity, spectral structure, and timbre. To tackle this problem, in the current work, we propose a novel speech enhancement model based on adaptive attention. The proposed model incorporates both local and global attention modules in between a convolutional encoder and a convolutional decoder. The local attention module (LAM) integrates channel and spatial attentions, which can make the model pay more attention to the local details in the speech block, specifically the frame-level features. And the features at utterance-level are explored through a self-attention mechanism in global attention module (GAM). Different from existing transformers, the feed forward network of GAM is improved by introducing a 1D-Conv layer and Bi-directional long short-term memory (Bi-LSTM) for extracting global features, so that the network can more effectively model long sequence context. Moreover, a CNN module is also added to global attention module so that short-term noise can be reduced more effectively, based on the ability of CNN to extract local information. The proposed model stands apart from the current speech enhancement techniques that solely rely on self-attention networks. Instead, our approach models the speech signal using two different attention networks simultaneously, both local detail information and global contextual information of speech are considered, thus better extracting useful information from the speech signal. The effectiveness of the proposed model is assessed using both objective (PESQ and STOI) and subjective tests (signal distortion (CSIG), background distortion (CBAK) and overall quality (COVL)) on two distinct datasets: Voice Bank-Demand dataset and LibriSpeech dataset. The experimental findings demonstrate that our model outperformed the competing baselines on both the datasets.

Full Attention Bidirectional Deep Learning Structure for Single Channel Speech Enhancement

An Attention-Based Neural Network Approach For Single Channel Speech Enhancement

Densely Connected Multi-Stage Model with Channel Wise Subband Feature for Real-Time Speech Enhancement.

Monaural Speech Enhancement Using Deep Multi-Branch Residual Network with 1-D Causal Dilated Convolutions

FB-MSTCN: A Full-Band Single-Channel Speech Enhancement Method Based on Multi-Scale Temporal Convolutional Network

Combining Multi-Perspective Attention Mechanism With Convolutional Networks for Monaural Speech Enhancement

Single-channel speech enhancement using improved progressive deep neural network and masking-based harmonic regeneration

Adaptive attention mechanism for single channel speech enhancement

Dual-Branch Attention-In-Attention Transformer for Single-Channel Speech Enhancement

An Attention Based Speaker-Independent Audio-Visual Deep Learning Model for Speech Enhancement

A Feature Integration Network for Multi-Channel Speech Enhancement

Parallel Gated Neural Network With Attention Mechanism For Speech Enhancement

Efficient Monaural Speech Enhancement using Spectrum Attention Fusion

Single-channel Speech Enhancement Using Multi-Task Learning and Attention Mechanism

Encoder-decoder with focus-mechanism for sequence labelling based spoken language understanding

Monaural Speech Enhancement Using a Multi-Branch Temporal Convolutional Network

Multi-Scale Temporal Frequency Convolutional Network With Axial Attention for Speech Enhancement

A Time Domain Progressive Learning Approach with SNR Constriction for Single-Channel Speech Enhancement and Recognition

A Hybrid Deep-Learning Approach for Single Channel HF-SSB Speech Enhancement

Multichannel Speech Enhancement without Beamforming

Supervised Attention Multi-Scale Temporal Convolutional Network for monaural speech enhancement