Abstract:Since the introduction of Generative Adversarial Networks (GANs) in speech synthesis, remarkable achievements have been attained. In a thorough exploration of vocoders, it has been discovered that audio waveforms can be generated at speeds exceeding real-time while maintaining high fidelity, achieved through the utilization of GAN-based models. Typically, the inputs to the vocoder consist of band-limited spectral information, which inevitably sacrifices high-frequency details. To address this, we adopt the full-band Mel spectrogram information as input, aiming to provide the vocoder with the most comprehensive information possible. However, previous studies have revealed that the use of full-band spectral information as input can result in the issue of over-smoothing, compromising the naturalness of the synthesized speech. To tackle this challenge, we propose VNet, a GAN-based neural vocoder network that incorporates full-band spectral information and introduces a Multi-Tier Discriminator (MTD) comprising multiple sub-discriminators to generate high-resolution signals. Additionally, we introduce an asymptotically constrained method that modifies the adversarial loss of the generator and discriminator, enhancing the stability of the training process. Through rigorous experiments, we demonstrate that the VNet model is capable of generating high-fidelity speech and significantly improving the performance of the vocoder.

GBNF-VAE: A Pathological Voice Enhancement Model Based on Gold Section for Bottleneck Feature With Variational Autoencoder

Audio-Visual Speech Enhancement with Deep Multi-modality Fusion

VSEGAN: Visual Speech Enhancement Generative Adversarial Network

LA-VocE: Low-SNR Audio-visual Speech Enhancement using Neural Vocoders

A variance modeling framework based on variational autoencoders for speech enhancement

Audio-Visual Speech Enhancement Based on Multiscale Features and Parallel Attention

Improving Deep Neural Network Based Speech Enhancement in Low SNR Environments

Enhancing Low-Quality Voice Recordings Using Disentangled Channel Factor and Neural Waveform Model

Statistical Speech Enhancement Based on Probabilistic Integration of Variational Autoencoder and Non-Negative Matrix Factorization

Electrolaryngeal Speech Enhancement Based on a Two Stage Framework with Bottleneck Feature Refinement and Voice Conversion

Pathological voice adaptation with autoencoder-based voice conversion

Speech Enhancement with Multi-granularity Vector Quantization.

Adversarial Data Augmentation Using VAE-GAN for Disordered Speech Recognition

A Speech Enhancement Method Based on Dual-Path Phase-Aware GAN Networks

Disentangling Content and Fine-Grained Prosody Information Via Hybrid ASR Bottleneck Features for Voice Conversion

Parallel Gated Neural Network With Attention Mechanism For Speech Enhancement

A Joint Framework of Denoising Autoencoder and Generative Vocoder for Monaural Speech Enhancement

VNet: A GAN-based Multi-Tier Discriminator Network for Speech Synthesis Vocoders

Improvement of Packet Loss Concealment for EVS Codec Based on Deep Learning

Generative Adversarial Networks for Unpaired Voice Transformation on Impaired Speech

An Improved Fully Convolutional Network Based on Post-Processing with Global Variance Equalization and Noise-Aware Training for Speech Enhancement.