Abstract:Recent researches have shown superior performance of applying end-to-end architecture in text-to-speech (TTS) synthesis. However, considering the complex linguistic structure of Chinese, using Chinese characters directly for Mandarin TTS may suffer from the poor linguistic encoding performance, resulting in improper word tokenization and pronunciation errors. To ensure the naturalness and intelligibility of synthetic speech, state-of-the-art Mandarin TTS systems employ a list of components, such as word tokenization, part-of-speech (POS) tagging and grapheme-to-phoneme (G2P) conversion, to produce knowledge-enhanced inputs to alleviate the problems caused by linguistic encoding. These components are based on linguistic expertise and well-designed, but trained individually, leading to errors compounding for the TTS system. In this paper, to reduce the complexity of Mandarin TTS system and bring further improvement, we proposed a knowledge-based linguistic encoder for the character-based end-to-end Mandarin TTS system. Developed with multi-task learning structure, the proposed encoder can learn from linguistic analysis subtasks, providing robust and discriminative linguistic encodings for the following speech generation decoder. Experimental results demonstrate the effectiveness of the proposed framework, with word tokenization error dropped from 12.81% to 1.58%, syllable pronunciation error dropped from 10.89% to 2.81% compared with state-of-the-art baseline approach, providing mean opinion score (MOS) improvement from 3.76 to 3.87.

Combining Active and Semi-Supervised Learning for Homograph Disambiguation in Mandarin Text-to-Speech Synthesis.

Tree-guided Transformation-Based Homograph Disambiguation in Mandarin TTS System

Polyphone Disambiguation in Mandarin Chinese with Semi-Supervised Learning

Heteronym Verification for Mandarin Speech Synthesis

Towards Language-universal Mandarin-English Speech Recognition with Unsupervised Label Synchronous Adaptation

External Knowledge Augmented Polyphone Disambiguation Using Large Language Model

End-to-end Code-switched TTS with Mix of Monolingual Recordings.

A Novel Hybrid Approach for Mandarin Speech Synthesis

A Novel Hybrid Mandarin Speech Synthesis System Using Different Base Units for Model Training and Concatenation

Speech Sense Disambiguation: Tackling Homophone Ambiguity in End-to-End Speech Translation

Multi-level Linguistic Knowledge Based Chinese Grapheme-to-Phoneme Conversion.

LEARNING CROSS-LINGUAL INFORMATION WITH MULTILINGUAL BLSTM FOR SPEECH SYNTHESIS OF LOW-RESOURCE LANGUAGES

Design of English text-to-speech conversion algorithm based on machine learning

Knowledge-based Linguistic Encoding for End-to-End Mandarin Text-to-Speech Synthesis

Mandarin-English bilingual phone modeling and combining MPE based Discriminative training for cross-language speech recognition

Label Transform Based Cross-Language Speaker Adaptation in Bilingual (Mandarin-English) TTS

Using heterogeneity in semi-supervised transcription hypotheses to improve code-switched speech recognition

Dict-TTS: Learning to Pronounce with Prior Dictionary Knowledge for Text-to-Speech

A Polyphone BERT for Polyphone Disambiguation in Mandarin Chinese

Data Augmentation for Code-Switch Language Modeling by Fusing Multiple Text Generation Methods.

HC$^2$L: Hybrid and Cooperative Contrastive Learning for Cross-lingual Spoken Language Understanding