Abstract:Recent researches have shown superior performance of applying end-to-end architecture in text-to-speech (TTS) synthesis. However, considering the complex linguistic structure of Chinese, using Chinese characters directly for Mandarin TTS may suffer from the poor linguistic encoding performance, resulting in improper word tokenization and pronunciation errors. To ensure the naturalness and intelligibility of synthetic speech, state-of-the-art Mandarin TTS systems employ a list of components, such as word tokenization, part-of-speech (POS) tagging and grapheme-to-phoneme (G2P) conversion, to produce knowledge-enhanced inputs to alleviate the problems caused by linguistic encoding. These components are based on linguistic expertise and well-designed, but trained individually, leading to errors compounding for the TTS system. In this paper, to reduce the complexity of Mandarin TTS system and bring further improvement, we proposed a knowledge-based linguistic encoder for the character-based end-to-end Mandarin TTS system. Developed with multi-task learning structure, the proposed encoder can learn from linguistic analysis subtasks, providing robust and discriminative linguistic encodings for the following speech generation decoder. Experimental results demonstrate the effectiveness of the proposed framework, with word tokenization error dropped from 12.81% to 1.58%, syllable pronunciation error dropped from 10.89% to 2.81% compared with state-of-the-art baseline approach, providing mean opinion score (MOS) improvement from 3.76 to 3.87.

Linguistic Pbolems in text-to-speech conversion Processing of the Chinese

Children's Use of Phonological Information in Ambiguity Resolution: a View from Mandarin Chinese.

A Chinese Text-to-Speech System

Grapheme-to-phoneme conversion in Chinese TTS system

Text Normalization in Chinese Text-to-Speech System

Design of English text-to-speech conversion algorithm based on machine learning

Multi-level Linguistic Knowledge Based Chinese Grapheme-to-Phoneme Conversion.

A New Chinese Text-to-speech System with High Naturalness

A phonetic-based approach to Chinese chat text normalization

Knowledge-based Linguistic Encoding for End-to-End Mandarin Text-to-Speech Synthesis

End-to-end Code-switched TTS with Mix of Monolingual Recordings.

A novel voice conversion system based on codebook mapping with phoneme-tied weighting.

Possible Problems and Measures Taken to Deal with Them in the Test of Standard Chinese Pronunciation on the Level of Techniques

Towards Fine-Grained Prosody Control for Voice Conversion

From Speech to Text in Chinese: A Computer-Aided Transcription System for the Legal Domain.

Research and Implementation of Text-to-speech System for Chinese

A Three-Stage Text Normalization Strategy for Mandarin Text-to-Speech Systems

Study about Chinese Speech Synthesis Algorithm and Acoustic Model Based on Wireless Communication Network

Text Enhancement for Paragraph Processing in End-to-End Code-switching TTS

Discussion on the Problems of Computer Assisted English Translation

Document Structure Analysis and Text Normalization for Chinese Putonghua and Cantonese Text-to-Speech Synthesis