Abstract:End-to-end speech synthesis can reach high quality and naturalness with low-resource adaptation data. However, the generalization of out-domain texts and the improving modeling accuracy of speaker representations are still challenging tasks. The limited adaptation data leads to unacceptable errors and low similarity of the synthetic speech. In this paper, both speaker representations modeling and acoustic model structure are improved for the speaker adaptation task. On the one hand, compared with the conventional methods that focused on using fixed global speaker representations, the attention gating is proposed to adjust speaker representations dynamically based on the attended context and prosody information, which can describe more pronunciation characteristics in phoneme level. On the other hand, to improve the robustness and avoid over-fitting, the decoder model is factored into average-net and adaptation-net, which are designed for learning speaker independent acoustic features and target speaker timbre imitation respectively. And the context discriminator is pre-trained by large ASR data to supervise the average-net generating proper speaker independent acoustic features for different phoneme. Experimental results on Mandarin dataset show that proposed methods lead to an improvement on intelligibility, naturalness and similarity.

Duration optimization of speaker adaptation in Mandarin TTS

An unvoiced/voiced duration adjustment algorithm based on context features in mandarin TTS

Phoneme Dependent Speaker Embedding And Model Factorization For Multi-Speaker Speech Synthesis And Adaptation

Label Transform Based Cross-Language Speaker Adaptation in Bilingual (Mandarin-English) TTS

Duration Prediction in Mandarin TTS System

HMM Based TTS for Mixed Language Text.

Duration Refinement by Jointly Optimizing State and Longer Unit Likelihood

Focusing on attention: prosody transfer and adaptative optimization strategy for multi-speaker end-to-end speech synthesis

Dynamic Speaker Representations Adjustment and Decoder Factorization for Speaker Adaptation in End-to-End Speech Synthesis

Improving Prosody with Linguistic and Bert Derived Features in Multi-Speaker Based Mandarin Chinese Neural TTS

Improving F0 prediction using bidirectional associative memories and syllable-level F0 features for HMM-based Mandarin speech synthesis

Duration Modeling of Neural TTS for Automatic Dubbing

Speaker Adaption with Intuitive Prosodic Features for Statistical Parametric Speech Synthesis

A Novel Prosody Adaptation Method for Mandarin Concatenation-Based Text-to-speech System

Mandarin-English Mixed TTS Based on HCSIPA

Spoken Content and Voice Factorization for Few-Shot Speaker Adaptation

Standard Speaker Selection in Speech Synthesis for Mandarin Tone Learning

Duration Model for post-processing in a Mandarin speech recognition system

Total-Duration-Aware Duration Modeling for Text-to-Speech Systems

Cross-Lingual Speaker Adaptation for HMM-Based Speech Synthesis

A Real-Time Tone Enhancement Method for Continuous Mandarin Speeches