Abstract:The decoupling-style concept begins to ignite in the speech enhancement area, which decouples the original complex spectrum estimation task into multiple easier sub-tasks (i.e., the magnitude-only recovery and residual complex spectrum estimation), resulting in better performance and easier interpretability. In this paper, we propose a dual-branch federative magnitude and phase estimation framework, dubbed DBT-Net, for monaural speech enhancement, aiming at recovering the coarse- and fine-grained regions of the overall spectrum in parallel. From the complementary perspective, the magnitude estimation branch is designed to filter out dominant noise components in the magnitude domain, while the complex spectrum purification branch is elaborately designed to inpaint the missing spectral details and implicitly estimate the phase information in the complex-valued spectral domain. To facilitate the information flow between each branch, interaction modules are introduced to leverage features learned from one branch, so as to suppress the undesired parts and recover the missing components of the other branch. Instead of adopting the conventional RNNs and temporal convolutional networks for sequence modeling, we employ a novel attention-in-attention transformer-based network within each branch for better feature learning. More specially, it is composed of several adaptive spectro-temporal attention transformer-based modules and an adaptive hierarchical attention module, aiming to capture long-term time-frequency dependencies and further aggregate intermediate hierarchical contextual information. Comprehensive evaluations on the WSJ0-SI84 + DNS-Challenge and VoiceBank + DEMAND dataset demonstrate that the proposed approach consistently outperforms previous advanced systems and yields state-of-the-art performance in terms of speech quality and intelligibility.

DPT-FSNet: Dual-Path Transformer Based Full-Band and Sub-Band Fusion Network for Speech Enhancement

ForkNet: Simultaneous Time and Time-Frequency Domain Modeling for Speech Enhancement

FullSubNet: A Full-Band and Sub-Band Fusion Model for Real-Time Single-Channel Speech Enhancement

FSI-Net: A dual-stage full- and sub-band integration network for full-band speech enhancement

Lightweight Full-band and Sub-band Fusion Network for Real Time Speech Enhancement

PFRNet: Dual-Branch Progressive Fusion Rectification Network for Monaural Speech Enhancement.

FullSubNet+: Channel Attention FullSubNet with Complex Spectrograms for Speech Enhancement

DMF-Net: A decoupling-style multi-band fusion model for real-time full-band speech enhancement.

Speech Enhancement with Perceptually-motivated Optimization and Dual Transformations

DMF-Net: A decoupling-style multi-band fusion model for full-band speech enhancement

THLNet: two-stage heterogeneous lightweight network for monaural speech enhancement

Dual-path transformer-based network with equalization-generation components prediction for flexible vibrational sensor speech enhancement in the time domain

Streaming Dual-Path Transformer for Speech Enhancement

DBT-Net: Dual-branch Federative Magnitude and Phase Estimation with Attention-in-attention Transformer for Monaural Speech Enhancement

Multi-Scale Feature Fusion Transformer Network for End-to-End Single Channel Speech Separation

FB-MSTCN: A Full-Band Single-Channel Speech Enhancement Method Based on Multi-Scale Temporal Convolutional Network

Speech Enhancement with Fullband-Subband Cross-Attention Network.

A Two-Stage Frequency-Time Dilated Dense Network for Speech Enhancement

DCHT: Deep Complex Hybrid Transformer for Speech Enhancement

Dual-Path Transformer Network: Direct Context-Aware Modeling for End-to-End Monaural Speech Separation