Abstract:Speech enhancement models based on deep learning are typically trained in a supervised manner, requiring a substantial amount of paired noisy-to-clean speech data for training. However, synthetically generated training data can only capture a limited range of realistic environments, and it is often challenging or even impractical to gather real-world pairs of noisy and ground-truth clean speech. To overcome this limitation, we propose an unsupervised learning approach for speech enhancement that eliminates the need for paired noisy-to-clean training data. Specifically, our method utilizes the optimal transport criterion to train the speech enhancement model in an unsupervised manner. It employs a fidelity loss based on noisy speech and a distribution divergence loss to minimize the difference between the distribution of the model's output and that of unpaired clean speech. Further, we use the speech presence probability as an additional optimization objective and incorporate the short-time Fourier transform (STFT) domain loss as an extra term for the unsupervised learning loss. We also apply the multi-resolution STFT loss as the validation loss to enhance the stability of the training process and improve the algorithm's performance. Experimental results on the VCTK + DEMAND benchmark demonstrate that the proposed method achieves competitive performance compared to the supervised methods. Furthermore, the speech recognition results on the CHiME4 benchmark show the superiority of the proposed method over its supervised counterpart.

Iterative Noisy-Target Approach: Speech Enhancement Without Clean Speech

Noisy-target Training: A Training Strategy for DNN-based Speech Enhancement without Clean Speech

DENOISPEECH: DENOISING TEXT TO SPEECH WITH FRAME-LEVEL NOISE MODELING

Dynamic noise aware training for speech enhancement based on deep neural networks.

Noisy training for deep neural networks in speech recognition

A Novel Training Strategy Using Dynamic Data Generation for Deep Neural Network Based Speech Enhancement.

Deep Neural Network Based Noised Asian Speech Enhancement and Its Implementation on a Hearing Aid App.

Self-supervised Speech Denoising Using Only Noisy Audio Signals

Neural Speech Enhancement with Unsupervised Pre-Training and Mixture Training

Noise Modeling to Build Training Sets for Robust Speech Enhancement

Unsupervised Noise adaptation using Data Simulation

Deep Noise Tracking Network: A Hybrid Signal Processing/Deep Learning Approach to Speech Enhancement

A Progressive Learning Approach to Adaptive Noise and Speech Estimation for Speech Enhancement and Noisy Speech Recognition.

A regression approach to speech enhancement based on deep neural networks

Multiple Modules Speech Enhancement in Mixed Noise and Low SNR Environments

Joint Noise and Mask Aware Training for DNN-based Speech Enhancement with SUB-band Features

Improved Noisy Student Training for Automatic Speech Recognition

On Generating Mixing Noise Signals With Basis Functions For Simulating Noisy Speech And Learning Dnn-Based Speech Enhancement Models

A Joint Speech Enhancement and Self-Supervised Representation Learning Framework for Noise-Robust Speech Recognition

Unsupervised Speech Enhancement Using Optimal Transport and Speech Presence Probability