Abstract:Recent researches have shown superior performance of applying end-to-end architecture in text-to-speech (TTS) synthesis. However, considering the complex linguistic structure of Chinese, using Chinese characters directly for Mandarin TTS may suffer from the poor linguistic encoding performance, resulting in improper word tokenization and pronunciation errors. To ensure the naturalness and intelligibility of synthetic speech, state-of-the-art Mandarin TTS systems employ a list of components, such as word tokenization, part-of-speech (POS) tagging and grapheme-to-phoneme (G2P) conversion, to produce knowledge-enhanced inputs to alleviate the problems caused by linguistic encoding. These components are based on linguistic expertise and well-designed, but trained individually, leading to errors compounding for the TTS system. In this paper, to reduce the complexity of Mandarin TTS system and bring further improvement, we proposed a knowledge-based linguistic encoder for the character-based end-to-end Mandarin TTS system. Developed with multi-task learning structure, the proposed encoder can learn from linguistic analysis subtasks, providing robust and discriminative linguistic encodings for the following speech generation decoder. Experimental results demonstrate the effectiveness of the proposed framework, with word tokenization error dropped from 12.81% to 1.58%, syllable pronunciation error dropped from 10.89% to 2.81% compared with state-of-the-art baseline approach, providing mean opinion score (MOS) improvement from 3.76 to 3.87.

G2P based on multi-sensing field and modified focus loss

ViSPer: A Multilingual TTS Approach Based on VITS Using Deep Feature Loss

A Polyphone BERT for Polyphone Disambiguation in Mandarin Chinese

Multi-level Linguistic Knowledge Based Chinese Grapheme-to-Phoneme Conversion.

g2pW: A Conditional Weighted Softmax BERT for Polyphone Disambiguation in Mandarin

LiteG2P: A fast, light and high accuracy model for grapheme-to-phoneme conversion

External Knowledge Augmented Polyphone Disambiguation Using Large Language Model

Polyphone Disambiguation for Mandarin Chinese Using Conditional Neural Network with Multi-level Embedding Features

Polyphone Disambiguation in Mandarin Chinese with Semi-Supervised Learning

Multilingual context-based pronunciation learning for Text-to-Speech

Knowledge-based Linguistic Encoding for End-to-End Mandarin Text-to-Speech Synthesis

Grapheme-to-phoneme conversion in Chinese TTS system

Dual-Decoder Transformer For end-to-end Mandarin Chinese Speech Recognition with Pinyin and Character

r-G2P: Evaluating and Enhancing Robustness of Grapheme to Phoneme Conversion by Controlled noise introducing and Contextual information incorporation

R-G2p: Evaluating and Enhancing Robustness of Grapheme to Phoneme Conversion by Controlled Noise Introducing and Contextual Information Incorporation

Combining Active and Semi-Supervised Learning for Homograph Disambiguation in Mandarin Text-to-Speech Synthesis.

CVTE-Poly: A New Benchmark for Chinese Polyphone Disambiguation

Improving Grapheme-to-Phoneme Conversion through In-Context Knowledge Retrieval with Large Language Models

Dict-TTS: Learning to Pronounce with Prior Dictionary Knowledge for Text-to-Speech

Good Neighbors Are All You Need for Chinese Grapheme-to-Phoneme Conversion

Boosting Character-based Mandarin ASR via Chinese Pinyin Representation