Abstract:The goal of speech enhancement (SE) is to eliminate the background interference from the noisy speech signal. Generative models such as diffusion models (DM) have been applied to the task of SE because of better generalization in unseen noisy scenes. Technical routes for the DM-based SE methods can be summarized into three types: task-adapted diffusion process formulation, generator-plus-conditioner (GPC) structures and the multi-stage frameworks. We focus on the first two approaches, which are constructed under the GPC architecture and use the task-adapted diffusion process to better deal with the real noise. However, the performance of these SE models is limited by the following issues: (a) Non-Gaussian noise estimation in the task-adapted diffusion process. (b) Conditional domain bias caused by the weak conditioner design in the GPC structure. (c) Large amount of residual noise caused by unreasonable interpolation operations during inference. To solve the above problems, we propose a noise-aware diffusion-based SE model (NADiffuSE) to boost the SE performance, where the noise representation is extracted from the noisy speech signal and introduced as a global conditional information for estimating the non-Gaussian components. Furthermore, the anchor-based inference algorithm is employed to achieve a compromise between the speech distortion and noise residual. In order to mitigate the performance degradation caused by the conditional domain bias in the GPC framework, we investigate three model variants, all of which can be viewed as multi-stage SE based on the preprocessing networks for Mel spectrograms. Experimental results show that NADiffuSE outperforms other DM-based SE models under the GPC infrastructure. Audio samples are available at: <a class="link-external link-https" href="https://square-of-w.github.io/NADiffuSE-demo/" rel="external noopener nofollow">this https URL</a>.

CRA-DIFFUSE: IMPROVED CROSS-DOMAIN SPEECH ENHANCEMENT BASED ON DIFFUSION MODEL WITH T-F DOMAIN PRE-DENOISING

NADiffuSE: Noise-aware Diffusion-based Model for Speech Enhancement

SRTNet: Time Domain Speech Enhancement Via Stochastic Refinement

A Study on Speech Enhancement Based on Diffusion Probabilistic Model

Noise-aware Speech Enhancement using Diffusion Probabilistic Model

Revisiting Denoising Diffusion Probabilistic Models for Speech Enhancement: Condition Collapse, Efficiency and Refinement

Extract and Diffuse: Latent Integration for Improved Diffusion-based Speech and Vocal Enhancement

Cross-domain Single-channel Speech Enhancement Model with Bi-projection Fusion Module for Noise-robust ASR

Pre-training Feature Guided Diffusion Model for Speech Enhancement

DDTSE: Discriminative Diffusion Model for Target Speech Extraction

Decoupling-style monaural speech enhancement with a triple-branch cross-domain fusion network

Investigating the Design Space of Diffusion Models for Speech Enhancement

SE-Bridge: Speech Enhancement with Consistent Brownian Bridge

Diffiner: A Versatile Diffusion-based Generative Refiner for Speech Enhancement

Densely Connected Multi-Stage Model with Channel Wise Subband Feature for Real-Time Speech Enhancement.

GALD-SE: Guided Anisotropic Lightweight Diffusion for Efficient Speech Enhancement

Diffusion-Based Speech Enhancement with Joint Generative and Predictive Decoders

Speech Enhancement with Perceptually-motivated Optimization and Dual Transformations

Diffusion-based Speech Enhancement with Schrödinger Bridge and Symmetric Noise Schedule

Variance-Preserving-Based Interpolation Diffusion Models for Speech Enhancement