Abstract:Recent researches have shown superior performance of applying end-to-end architecture in text-to-speech (TTS) synthesis. However, considering the complex linguistic structure of Chinese, using Chinese characters directly for Mandarin TTS may suffer from the poor linguistic encoding performance, resulting in improper word tokenization and pronunciation errors. To ensure the naturalness and intelligibility of synthetic speech, state-of-the-art Mandarin TTS systems employ a list of components, such as word tokenization, part-of-speech (POS) tagging and grapheme-to-phoneme (G2P) conversion, to produce knowledge-enhanced inputs to alleviate the problems caused by linguistic encoding. These components are based on linguistic expertise and well-designed, but trained individually, leading to errors compounding for the TTS system. In this paper, to reduce the complexity of Mandarin TTS system and bring further improvement, we proposed a knowledge-based linguistic encoder for the character-based end-to-end Mandarin TTS system. Developed with multi-task learning structure, the proposed encoder can learn from linguistic analysis subtasks, providing robust and discriminative linguistic encodings for the following speech generation decoder. Experimental results demonstrate the effectiveness of the proposed framework, with word tokenization error dropped from 12.81% to 1.58%, syllable pronunciation error dropped from 10.89% to 2.81% compared with state-of-the-art baseline approach, providing mean opinion score (MOS) improvement from 3.76 to 3.87.

A Pair-Based Language Model for the Robust Lexical Analysis in Chinese Text-to-speech Synthesis.

A unified framework for text analysis in chinese TTS

Automatic Conversion from Lexical Words to Prosodic Words for Mandarin Text-to-speech System

HMM and CRF Based Hybrid Model for Chinese Lexical Analysis

ZhoBLiMP: a Systematic Assessment of Language Models with Linguistic Minimal Pairs in Chinese

An Enhanced Text Analysis Approach in Text-to-Speech Synthesis for Mandarin Chinese

Research on Chinese Lexical Analysis System by Fusing Multiple Knowledge Sources

Phonetic Enhanced Language Modeling for Text-to-Speech Synthesis

Joint n-gram Chinese language modeling with an application to Chinese word segmentation

Joint-Character-Poc N-Gram Language Modeling for Chinese Speech Recognition

HMM Based TTS for Mixed Language Text.

Linguistic Minimal Pairs Elicit Linguistic Similarity in Large Language Models

Parsing Chinese Based on Lexicalized Model

CLiMP: A Benchmark for Chinese Language Model Evaluation

An Adaptive Wordpiece Language Model For Learning Chinese Word Embeddings

Six-Writings multimodal processing with pictophonetic coding to enhance Chinese language models

Using Large Language Model for End-to-End Chinese ASR and NER

Knowledge-based Linguistic Encoding for End-to-End Mandarin Text-to-Speech Synthesis

A Word Language Model Based Contextual Language Processing On Chinese Character Recognition

Towards Joint Modeling of Dialogue Response and Speech Synthesis based on Large Language Model

Two Language Models Using Chinese Semantic Parsing