Abstract:Recent researches have shown superior performance of applying end-to-end architecture in text-to-speech (TTS) synthesis. However, considering the complex linguistic structure of Chinese, using Chinese characters directly for Mandarin TTS may suffer from the poor linguistic encoding performance, resulting in improper word tokenization and pronunciation errors. To ensure the naturalness and intelligibility of synthetic speech, state-of-the-art Mandarin TTS systems employ a list of components, such as word tokenization, part-of-speech (POS) tagging and grapheme-to-phoneme (G2P) conversion, to produce knowledge-enhanced inputs to alleviate the problems caused by linguistic encoding. These components are based on linguistic expertise and well-designed, but trained individually, leading to errors compounding for the TTS system. In this paper, to reduce the complexity of Mandarin TTS system and bring further improvement, we proposed a knowledge-based linguistic encoder for the character-based end-to-end Mandarin TTS system. Developed with multi-task learning structure, the proposed encoder can learn from linguistic analysis subtasks, providing robust and discriminative linguistic encodings for the following speech generation decoder. Experimental results demonstrate the effectiveness of the proposed framework, with word tokenization error dropped from 12.81% to 1.58%, syllable pronunciation error dropped from 10.89% to 2.81% compared with state-of-the-art baseline approach, providing mean opinion score (MOS) improvement from 3.76 to 3.87.

A unified sequence-to-sequence front-end model for Mandarin text-to-speech synthesis

A unified front-end framework for English text-to-speech synthesis

Unified Mandarin TTS Front-end Based on Distilled BERT Model

End-to-end Code-switched TTS with Mix of Monolingual Recordings.

A Novel Chinese Dialect TTS Frontend with Non-Autoregressive Neural Machine Translation

An Unified and Automatic Approach of Mandarin HTS System.

Knowledge-based Linguistic Encoding for End-to-End Mandarin Text-to-Speech Synthesis

Mandarin Text-to-Speech Front-End with Lightweight Distilled Convolution Network

Improving Prosody with Linguistic and Bert Derived Features in Multi-Speaker Based Mandarin Chinese Neural TTS

A Novel Method for Mandarin Speech Synthesis by Inserting Prosodic Structure Prediction into Tacotron2.

A unified framework for text analysis in chinese TTS

An Enhanced Text Analysis Approach in Text-to-Speech Synthesis for Mandarin Chinese

A Two-stage Prosodic Structure Generation Strategy for Mandarin Text-to-speech Systems

A Novel Hybrid Mandarin Speech Synthesis System Using Different Base Units for Model Training and Concatenation

End-to-End Mongolian Text-to-Speech System

A Novel Prosody Adaptation Method for Mandarin Concatenation-Based Text-to-speech System

Text Enhancement for Paragraph Processing in End-to-End Code-switching TTS

Chinese Speech Synthesis System Based on End to End

Scalable Multilingual Frontend for TTS

A Preliminary Study on Deep Learning-based Chinese Text to Taiwanese Speech Synthesis System

Automatic Conversion from Lexical Words to Prosodic Words for Mandarin Text-to-speech System