Abstract:State-of-the-art text-to-speech (TTS) synthesis models can produce monolingual speech with high intelligibility and naturalness. However, when the models are applied to synthesize code-switched (CS) speech, the performance declines seriously. Conventionally, developing a CS TTS system requires multilingual data to incorporate language-specific and cross-lingual knowledge. Recently, end-to-end (E2E) architecture has achieved satisfactory results in monolingual TTS. The architecture enables the training from one end of alphabetic text input to the other end of acoustic feature output. In this paper, we explore the use of E2E framework for CS TTS, using a combination of Mandarin and English monolingual speech corpus uttered by two female speakers. To handle alphabetic input from different languages, we explore two kinds of encoders: (1) shared multilingual encoder with explicit language embedding (LDE); (2) separated monolingual encoder (SPE) for each language. The two systems use identical decoder architecture, where a discriminative code is incorporated to enable the model to generate speech in one speaker's voice consistently. Experiments confirm the effectiveness of the proposed modifications on the E2E TTS framework in terms of quality and speaker similarity of the generated speech. Moreover, our proposed systems can generate controllable foreign-accented speech at character-level using only mixture of monolingual training data.

Implementation of Chinese-Uyghur Bilateral EBMT System

Multilingual Machine Translation system

Using RBMT Systems to Produce Bilingual Corpus for SMT

Uyghur-Chinese statistical machine translation by incorporating morphological information

TransEasy: A Chinese-English Machine Translation System Based on Hybrid Approach

Memory-augmented Chinese-Uyghur neural machine translation

Example-based Machine Translation Based on Tree–string Correspondence and Statistical Generation

End-to-end Code-switched TTS with Mix of Monolingual Recordings.

Performance Comparison of Neural Machine Translation Systems in Uyghur-Chinese Translation

NJUNLP’s Machine Translation System for CCMT-2020 Uighur $$\rightarrow $$ Chinese Translation Task

A Hybrid Approach For Hindi-English Machine Translation

Development of Translation Database based on Chinese-English parallel corpora

NJUNLP’s Machine Translation System for CCMT-2020 Uighur → Chinese Translation Task

Summer: WeChat Neural Machine Translation Systems for the WMT22 Biomedical Translation Task

SubCharacter Chinese-English Neural Machine Translation with Wubi encoding

WeChat Neural Machine Translation Systems for WMT20

BJTU-WeChat's Systems for the WMT22 Chat Translation Task

Uyghur, Chinese and English Multilingual Document Recognition

Neural System Combination For Machine Translation

Exploring the traditional NMT model and Large Language Model for chat translation

Benchmarking Machine Translation with Cultural Awareness