Abstract:State-of-the-art text-to-speech (TTS) synthesis models can produce monolingual speech with high intelligibility and naturalness. However, when the models are applied to synthesize code-switched (CS) speech, the performance declines seriously. Conventionally, developing a CS TTS system requires multilingual data to incorporate language-specific and cross-lingual knowledge. Recently, end-to-end (E2E) architecture has achieved satisfactory results in monolingual TTS. The architecture enables the training from one end of alphabetic text input to the other end of acoustic feature output. In this paper, we explore the use of E2E framework for CS TTS, using a combination of Mandarin and English monolingual speech corpus uttered by two female speakers. To handle alphabetic input from different languages, we explore two kinds of encoders: (1) shared multilingual encoder with explicit language embedding (LDE); (2) separated monolingual encoder (SPE) for each language. The two systems use identical decoder architecture, where a discriminative code is incorporated to enable the model to generate speech in one speaker's voice consistently. Experiments confirm the effectiveness of the proposed modifications on the E2E TTS framework in terms of quality and speaker similarity of the generated speech. Moreover, our proposed systems can generate controllable foreign-accented speech at character-level using only mixture of monolingual training data.

Overview of the IBM Mandarin Text-to-Speech System

The 2009 Ibm Gale Mandarin Broadcast Transcription System

A Chinese Text-to-Speech System

The IBM Mandarin Broadcast Speech Transcription System

Recent Advances in the Ibm Gale Mandarin Transcription System

A New Chinese Text-to-speech System with High Naturalness

IBM Voice Conversion Systems for 2007 TC-STAR Evaluation

A unified sequence-to-sequence front-end model for Mandarin text-to-speech synthesis

The IBM Submission to the 2008 Text-to-Speech Blizzard Challenge

The IBM Speech-to-speech Translation System for Smartphone: Improvements for Resource-Constrained Tasks.

High quality Chinese text-to-speech system - BEYOND

An Enhanced Text Analysis Approach in Text-to-Speech Synthesis for Mandarin Chinese

End-to-end Code-switched TTS with Mix of Monolingual Recordings.

IBM GALE Mandarin transcription system

Advances in Mandarin Broadcast Speech Transcription at IBM under the DARPA GALE Program.

Syllable HMM Based Mandarin TTS and Comparison with Concatenative TTS.

Total Quality Evaluation of Speech Synthesis Systems.

KD2000 Chinese Text-To-Speech System

Text-To-Visual Speech in Chinese Based on Data-Driven Approach

IBM Mastor: Multilingual Automatic Speech-To-Speech Translator

IBM MASTOR System