Abstract:Voice conversion is to generate a new speech with the source content and a target voice style. In this paper, we focus on one general setting, i.e., non-parallel many-to-many voice conversion, which is close to the real-world scenario. As the name implies, non-parallel many-to-many voice conversion does not require the paired source and reference speeches and can be applied to arbitrary voice transfer. In recent years, Generative Adversarial Networks (GANs) and other techniques such as Conditional Variational Autoencoders (CVAEs) have made considerable progress in this field. However, due to the sophistication of voice conversion, the style similarity of the converted speech is still unsatisfactory. Inspired by the inherent structure of mel-spectrogram, we propose a new voice conversion framework, i.e., Subband-based Generative Adversarial Network for Voice Conversion (SGAN-VC). SGAN-VC converts each subband content of the source speech separately by explicitly utilizing the spatial characteristics between different subbands. SGAN-VC contains one style encoder, one content encoder, and one decoder. In particular, the style encoder network is designed to learn style codes for different subbands of the target speaker. The content encoder network can capture the content information on the source speech. Finally, the decoder generates particular subband content. In addition, we propose a pitch-shift module to fine-tune the pitch of the source speaker, making the converted tone more accurate and explainable. Extensive experiments demonstrate that the proposed approach achieves state-of-the-art performance on VCTK Corpus and AISHELL3 datasets both qualitatively and quantitatively, whether on seen or unseen data. Furthermore, the content intelligibility of SGAN-VC on unseen data even exceeds that of StarGANv2-VC with ASR network assistance.

GTDNN-Based Voice Conversion Using DAEs with Binary Distributed Hidden Units

Voice Conversion Using Deep Neural Networks with Layer-Wise Generative Training

Extracting Spectral Features Using Deep Autoencoders with Binary Distributed Hidden Units for Statistical Parametric Speech Synthesis.

Deep Neural Network Based Voice Conversion with A Large Synthesized Parallel Corpus

Voice Conversion Using Generative Trained Deep Neural Networks with Multiple Frame Spectral Envelopes

Denoising Recurrent Neural Network for Deep Bidirectional Lstm Based Voice Conversion

Spectral Conversion Using Deep Neural Networks Trained with Multi-Source Speakers

Efficient Non-Autoregressive GAN Voice Conversion using VQWav2vec Features and Dynamic Convolution

Voice Conversion with Denoising Diffusion Probabilistic GAN Models

NVCGAN: Leveraging Generative Adversarial Networks for Robust Voice Conversion

A Dynamic Gaussian Process for Voice Conversion.

NeuralVC: Any-to-Any Voice Conversion Using Neural Networks Decoder for Real-Time Voice Conversion

Conditional Deep Hierarchical Variational Autoencoder for Voice Conversion

Disentangling Content and Fine-Grained Prosody Information Via Hybrid ASR Bottleneck Features for Voice Conversion

Reconstructing Dual Learning for Neural Voice Conversion Using Relatively Few Samples.

Unsupervised Representation Disentanglement using Cross Domain Features and Adversarial Learning in Variational Autoencoder based Voice Conversion

Boosted deep neural networks and multi-resolution cochleagram features for voice activity detection.

Deep Vocoder: Low Bit Rate Compression of Speech with Deep Autoencoder

Subband-based Generative Adversarial Network for Non-parallel Many-to-many Voice Conversion

Denoising Deep Neural Networks Based Voice Activity Detection

Unsupervised End-to-End Learning of Discrete Linguistic Units for Voice Conversion