Abstract:Low resource of parallel data is the key challenge of accent conversion(AC) problem in which both the pronunciation units and prosody pattern need to be converted. We propose a two-stage generative framework "convert-and-speak" in which the conversion is only operated on the semantic token level and the speech is synthesized conditioned on the converted semantic token with a speech generative model in target accent domain. The decoupling design enables the "speaking" module to use massive amount of target accent speech and relieves the parallel data required for the "conversion" module. Conversion with the bridge of semantic token also relieves the requirement for the data with text transcriptions and unlocks the usage of language pre-training technology to further efficiently reduce the need of parallel accent speech data. To reduce the complexity and latency of "speaking", a single-stage AR generative model is designed to achieve good quality as well as lower computation cost. Experiments on Indian-English to general American-English conversion show that the proposed framework achieves state-of-the-art performance in accent similarity, speech quality, and speaker maintenance with only 15 minutes of weakly parallel data which is not constrained to the same speaker. Extensive experimentation with diverse accent types suggests that this framework possesses a high degree of adaptability, making it readily scalable to accommodate other accents with low-resource data. Audio samples are available at <a class="link-external link-https" href="https://www.microsoft.com/en-us/research/project/convert-and-speak-zero-shot-accent-conversion-with-minimumsupervision/" rel="external noopener nofollow">this https URL</a>.

Non-autoregressive real-time Accent Conversion model with voice cloning

APB2FaceV2: Real-Time Audio-Guided Multi-Face Reenactment

Accent conversion using discrete units with parallel data synthesized from controllable accented TTS

Low-latency Real-time Voice Conversion on CPU

Convert and Speak: Zero-shot Accent Conversion with Minimum Supervision

Transfer the linguistic representations from TTS to accent conversion with non-parallel data

Improving Accent Conversion with Reference Encoder and End-To-End Text-To-Speech

NeuralVC: Any-to-Any Voice Conversion Using Neural Networks Decoder for Real-Time Voice Conversion

Non-Parallel Voice Conversion with Autoregressive Conversion Model and Duration Adjustment

Voice-preserving Zero-shot Multiple Accent Conversion

Improving Accent Conversion with Reference Encoder and End-To-End Text-To-Speech

Evaluating Methods for Ground-Truth-Free Foreign Accent Conversion

End-To-End Accent Conversion Without Using Native Utterances

Cross-lingual Multi-speaker Text-to-speech Synthesis for Voice Cloning without Using Parallel Corpus for Unseen Speakers

TTS-Guided Training for Accent Conversion Without Parallel Data

Neural Fusion for Voice Cloning

Voice Conversion for Stuttered Speech, Instruments, Unseen Languages and Textually Described Voices

Accent Conversion with Articulatory Representations

Improving Model Stability and Training Efficiency in Fast, High Quality Expressive Voice Conversion System

Building Bilingual and Code-Switched Voice Conversion with Limited Training Data Using Embedding Consistency Loss

Sequence-to-Sequence Acoustic Modeling for Voice Conversion