Abstract:The decoupling-style concept begins to ignite in the speech enhancement area, which decouples the original complex spectrum estimation task into multiple easier sub-tasks (i.e., the magnitude-only recovery and residual complex spectrum estimation), resulting in better performance and easier interpretability. In this paper, we propose a dual-branch federative magnitude and phase estimation framework, dubbed DBT-Net, for monaural speech enhancement, aiming at recovering the coarse- and fine-grained regions of the overall spectrum in parallel. From the complementary perspective, the magnitude estimation branch is designed to filter out dominant noise components in the magnitude domain, while the complex spectrum purification branch is elaborately designed to inpaint the missing spectral details and implicitly estimate the phase information in the complex-valued spectral domain. To facilitate the information flow between each branch, interaction modules are introduced to leverage features learned from one branch, so as to suppress the undesired parts and recover the missing components of the other branch. Instead of adopting the conventional RNNs and temporal convolutional networks for sequence modeling, we employ a novel attention-in-attention transformer-based network within each branch for better feature learning. More specially, it is composed of several adaptive spectro-temporal attention transformer-based modules and an adaptive hierarchical attention module, aiming to capture long-term time-frequency dependencies and further aggregate intermediate hierarchical contextual information. Comprehensive evaluations on the WSJ0-SI84 + DNS-Challenge and VoiceBank + DEMAND dataset demonstrate that the proposed approach consistently outperforms previous advanced systems and yields state-of-the-art performance in terms of speech quality and intelligibility.

Dual-Branch Modeling Based on State-Space Model for Speech Enhancement

BSDB-Net: Band-Split Dual-Branch Network with Selective State Spaces Mechanism for Monaural Speech Enhancement

DBT-Net: Dual-branch Federative Magnitude and Phase Estimation with Attention-in-attention Transformer for Monaural Speech Enhancement

A Multi-dimensional Deep Structured State Space Approach to Speech Enhancement Using Small-footprint Models

Densely Connected Multi-Stage Model with Channel Wise Subband Feature for Real-Time Speech Enhancement.

DBNet: A Dual-branch Network Architecture Processing on Spectrum and Waveform for Single-channel Speech Enhancement

A Dual-Branch Neural Network for Phase-Aware Speech Bandwidth Extension

ForkNet: Simultaneous Time and Time-Frequency Domain Modeling for Speech Enhancement

Two Heads Are Better Than One: A Two-Stage Complex Spectral Mapping Approach for Monaural Speech Enhancement.

Dual-Branch Attention-In-Attention Transformer for Single-Channel Speech Enhancement

Spiking Structured State Space Model for Monaural Speech Enhancement

THLNet: two-stage heterogeneous lightweight network for monaural speech enhancement

Single Channel Speech Enhancement Algorithm Based on BLSTM-DNN Bidirectional Optimized Hybrid Model

Real-Time Speech Enhancement for Mobile Communication Based on Dual-Channel Complex Spectral Mapping

Single-Channel Speech Enhancement with Deep Complex U-Networks and Probabilistic Latent Space Models

Selective State Space Model for Monaural Speech Enhancement

D2Former: A Fully Complex Dual-Path Dual-Decoder Conformer Network using Joint Complex Masking and Complex Spectral Mapping for Monaural Speech Enhancement

A dual-region speech enhancement method based on voiceprint segmentation

Double Branches and Stages Neural Network for Joint Acoustic Echo and Noise Suppression

Dual Branch Deep Interactive UNet for Monaural Noisy-Reverberant Speech Enhancement

Neural Speech Enhancement with Very Low Algorithmic Latency and Complexity via Integrated Full- and Sub-Band Modeling