Abstract:Recent advances of neural TTS have made "human parity" synthesized speech possible when a large amount of studio-quality training data from a voice talent is available. However, with only limited, casual recordings from an ordinary speaker, human-like TTS is still a big challenge, in addition to other artifacts like incomplete sentences, repetition of words, etc. Chinese, a language, of which the text is different from that of other roman-letter based languages like English, has no blank space between adjacent words, hence word segmentation errors can cause serious semantic confusions and unnatural prosody. In this study, with a multi-speaker TTS to accommodate the insufficient training data of a target speaker, we investigate linguistic features and Bert-derived information to improve the prosody of our Mandarin Chinese TTS. Three factors are studied: phone-related and prosody-related linguistic features; better predicted breaks with a refined Bert-CRF model; augmented phoneme sequence with character embedding derived from a Bert model. Subjective tests on in- and out-domain tasks of News, Chat and Audiobook, have shown that all factors are effective for improving prosody of our Mandarin TTS. The model with additional character embeddings from Bert is the best one, which outperforms the baseline by 0.17 MOS gain.

Improving Prosodic Boundaries Prediction For Mandarin Speech Synthesis By Using Enhanced Embedding Feature And Model Fusion Approach

Dual Audio Encoders Based Mandarin Prosodic Boundary Prediction by Using Multi-Granularity Prosodic Representations

Adversarial Multi-Task Learning for Mandarin Prosodic Boundary Prediction with Multi-Modal Embeddings

Blstm-Crf Based End-To-End Prosodic Boundary Prediction With Context Sensitive Embeddings In A Text-To-Speech Front-End

Parsing Hierarchical Prosodic Structure For Mandarin Speech Synthesis

Prosodic boundary prediction based on maximum entropy model with error-driven modification

Prosodic Word Boundaries Prediction for Mandarin Text-to-Speech

Investigating Effect of Rich Syntactic Features on Mandarin Prosodic Boundaries Prediction

Improving Mandarin Prosodic Structure Prediction with Multi-level Contextual Information

A Character-level Span-based Model for Mandarin Prosodic Structure Prediction

Mandarin prosodic word prediction using dependency relationships

A Two-stage Prosodic Structure Generation Strategy for Mandarin Text-to-speech Systems

A Novel Hybrid Approach for Mandarin Speech Synthesis

A Maximum Entropy Based Hierarchical Model for Automatic Prosodic Boundary Labeling in Mandarin

Automatic prosody prediction for Chinese speech synthesis using BLSTM-RNN and embedding features

Pitch Prediction for Mandarin TTS with Mutual Prosodic Constraint

Improving Prosody with Linguistic and Bert Derived Features in Multi-Speaker Based Mandarin Chinese Neural TTS

Phoneme Dependent Speaker Embedding And Model Factorization For Multi-Speaker Speech Synthesis And Adaptation

Chinese Prosodic Word Prediction Using the Conditional Random Fields.

Automatic Conversion from Lexical Words to Prosodic Words for Mandarin Text-to-speech System

Prosody Learning Mechanism for Speech Synthesis System Without Text Length Limit