Abstract:The recent development of speech enhancement methods has incorporated attention mechanisms for learning long-term speech signal dependencies. The utilization of deep convolution networks (DCN) equipped with the self-attention (SA) and transformers has showed promising results in speech enhancement (SE). While self-attention networks excel in extracting significant long-sequence contextual information in mining tasks, they may not effectively concentrate on subtle aspects within speech signals. These subtle details include temporal or spectral continuity, spectral structure, and timbre. To tackle this problem, in the current work, we propose a novel speech enhancement model based on adaptive attention. The proposed model incorporates both local and global attention modules in between a convolutional encoder and a convolutional decoder. The local attention module (LAM) integrates channel and spatial attentions, which can make the model pay more attention to the local details in the speech block, specifically the frame-level features. And the features at utterance-level are explored through a self-attention mechanism in global attention module (GAM). Different from existing transformers, the feed forward network of GAM is improved by introducing a 1D-Conv layer and Bi-directional long short-term memory (Bi-LSTM) for extracting global features, so that the network can more effectively model long sequence context. Moreover, a CNN module is also added to global attention module so that short-term noise can be reduced more effectively, based on the ability of CNN to extract local information. The proposed model stands apart from the current speech enhancement techniques that solely rely on self-attention networks. Instead, our approach models the speech signal using two different attention networks simultaneously, both local detail information and global contextual information of speech are considered, thus better extracting useful information from the speech signal. The effectiveness of the proposed model is assessed using both objective (PESQ and STOI) and subjective tests (signal distortion (CSIG), background distortion (CBAK) and overall quality (COVL)) on two distinct datasets: Voice Bank-Demand dataset and LibriSpeech dataset. The experimental findings demonstrate that our model outperformed the competing baselines on both the datasets.

Hybrid Attention Time-Frequency Analysis Network for Single-Channel Speech Enhancement.

Time-Frequency Attention for Monaural Speech Enhancement

TF(2)AN: A Temporal-Frequency Fusion Attention Network for Spectrum Energy Level Prediction

THLNet: two-stage heterogeneous lightweight network for monaural speech enhancement

Two-stage unet with channel and temporal-frequency attention for multi-channel speech enhancement

A Feature Integration Network for Multi-Channel Speech Enhancement

An Attention-Based Neural Network Approach For Single Channel Speech Enhancement

Temporal Convolutional Network with Frequency Dimension Adaptive Attention for Speech Enhancement.

Dual-Branch Attention-In-Attention Transformer for Single-Channel Speech Enhancement

FB-MSTCN: A Full-Band Single-Channel Speech Enhancement Method Based on Multi-Scale Temporal Convolutional Network

Stacked Multiscale Densely Connected Temporal Convolutional Attention Network for Multi-Objective Speech Enhancement in an Airborne Environment

Efficient Monaural Speech Enhancement using Spectrum Attention Fusion

HAM: Hybrid Attention Module in Deep Convolutional Neural Networks for Image Classification

Channel and Temporal-Frequency Attention UNet for Monaural Speech Enhancement.

Attention Mechanism Based ResNeXt Network for Automatic Modulation Classification.

Multi-Frequency Information Enhanced Channel Attention Module for Speaker Representation Learning

Time-Domain Neural Network Approach for Speech Bandwidth Extension.

Improving Visual Speech Enhancement Network by Learning Audio-visual Affinity with Multi-head Attention

Supervised Attention Multi-Scale Temporal Convolutional Network for monaural speech enhancement

Adaptive attention mechanism for single channel speech enhancement

A Multi-scale Subconvolutional U-Net with Time-Frequency Attention Mechanism for Single Channel Speech Enhancement