Abstract:End-to-end speech synthesis can reach high quality and naturalness with low-resource adaptation data. However, the generalization of out-domain texts and the improving modeling accuracy of speaker representations are still challenging tasks. The limited adaptation data leads to unacceptable errors and low similarity of the synthetic speech. In this paper, both speaker representations modeling and acoustic model structure are improved for the speaker adaptation task. On the one hand, compared with the conventional methods that focused on using fixed global speaker representations, the attention gating is proposed to adjust speaker representations dynamically based on the attended context and prosody information, which can describe more pronunciation characteristics in phoneme level. On the other hand, to improve the robustness and avoid over-fitting, the decoder model is factored into average-net and adaptation-net, which are designed for learning speaker independent acoustic features and target speaker timbre imitation respectively. And the context discriminator is pre-trained by large ASR data to supervise the average-net generating proper speaker independent acoustic features for different phoneme. Experimental results on Mandarin dataset show that proposed methods lead to an improvement on intelligibility, naturalness and similarity.

A Decision Tree-Structured Algorithm Of Speaker Adaptation Based On Gaussian Similarity Analysis

of Speaker Adaptation Based on Gaussian

Interpolation adaptation algorithm based on gaussian similarity analysis

Agmma: A Novel Incremental Adaptation Method And Its Application To Speaker Recognition

Speaker Adaptation and Adaptive Training for Jointly Optimised Tandem Systems.

Research on the influence of decision tree structure on speaker adaptation

UBM Based Speaker Selection and Model Re-Estimation for Speaker Adaptation

Speaker Adaptation for Telephony Data Using Speaker Clustering

A Speaker Adaptation Algorithm Based on Matrix Linear Interpolation

Linguistic tree based maximum likelihood model interpolation

Speaker adaptation using maximum likelihood model interpolation

On-line Bayesian Speaker Adaptation Using Tree-Structured Transformation and Robust Priors

Dynamic Speaker Representations Adjustment and Decoder Factorization for Speaker Adaptation in End-to-End Speech Synthesis

SVM Based Speaker Selection Using GMM Supervector for Rapid Speaker Adaptation

Codebook-Based Speaker Adaptation

Unsupervised Speaker Adaptation Of Deep Neural Network Based On The Combination Of Speaker Codes And Singular Value Decomposition For Speech Recognition

Genetic Algorithms and Fuzzy Approach to Gaussian Mixture Model for Speaker Recognition

Dynamic Speaker Selected Training for Rapid Speaker Adaptation

Speaker Recognition Based on Adaptive Niche Hybrid Genetic Algorithms

A New Subspace Based Speaker Adaptation Method

Speaker adaptation based on combination of MAP estimation and weighted neighbor regression