Abstract:Generative Adversarial Networks (GANs) have been used in the field of speech enhancement due to their huge potentials in reducing the noise mixed in the signals. Most of existing GAN-based speech enhancement approaches either operate on time domain or exploit the magnitude spectra in time-frequency domain, but lack consideration of direct optimization of the phase. In this paper, we propose a GAN architecture for speech enhancement based on gated linear units (GLUs) and Dual-Path Transformers (DPTs), which simultaneously deals with the amplitude and phase information on the time-frequency domain. The generator of the proposed GAN architecture is designed following an autoencoder structure fed by the real and imaginary parts of the time-frequency frames. The encoder of the generator is constructed by multiple cascaded convolutional GLUs (ConvGLUs), while the decoder consists of two groups of cascaded deconvolutional GLUs (DeconvGLUs), one for the real part of the spectrogram and the other for the imaginary part. The GLUs are adopted since they are potential in avoiding the gradient vanishing issue dwelling in deep architectures by providing a linear path for the gradients while retaining non-linear capabilities. Aiming at capturing the long-range dependent features in speech, we place DPTs between the encoder and the decoder of the generator, which contains multi-head attention modules and Bi-directional Gated Recurrent Units (BiGRUs). Moreover, the DPT structure is also merged with multiple one-dimensional convolutional layers in the discriminator of the GAN. Such a design not only improves the speech enhancement performance of GAN by focusing on multiple features of speech, but also reducing the volume of model parameters of GAN. Experimental results suggest that the proposed GAN architecture outperforms the existing benchmark GANs in terms of both objective speech intelligibility and quality with less computational complexity.

Exploring Speech Enhancement with Generative Adversarial Networks for Robust Speech Recognition

Investigating Generative Adversarial Networks based Speech Dereverberation for Robust Speech Recognition

Double Adversarial Network Based Monaural Speech Enhancement for Robust Speech Recognition.

Speech Enhancement Via Generative Adversarial Lstm Networks

Feature-Matching Speech Denoising GANs via Progressive Training.

GSC Based Speech Enhancement with Generative Adversarial Network

Boosting Noise Robustness of Acoustic Model via Deep Adversarial Training

Conditional Generative Adversarial Networks for Speech Enhancement and Noise-Robust Speaker Verification

Review of Speech Enhancement Methods using Generative Adversarial Networks

Towards Generalized Speech Enhancement with Generative Adversarial Networks

Generative Adversarial Networks Based Data Augmentation for Noise Robust Speech Recognition

Improving Speech Recognition Using GAN-Based Speech Synthesis and Contrastive Unspoken Text Selection

Data Augmentation using Conditional Generative Adversarial Networks for Robust Speech Recognition

Study of GANs for Noisy Speech Simulation from Clean Speech

Speech Enhancement Generative Adversarial Network Architecture with Gated Linear Units and Dual-Path Transformers

Dynamic Attention Based Generative Adversarial Network with Phase Post-Processing for Speech Enhancement

Noise Modeling to Build Training Sets for Robust Speech Enhancement

Channel-Aware Domain-Adaptive Generative Adversarial Network for Robust Speech Recognition

SEGAN: Speech Enhancement Generative Adversarial Network

Multi-scale Generative Adversarial Networks for Speech Enhancement

SE-MelGAN -- Speaker Agnostic Rapid Speech Enhancement