Abstract:State-of-the-art text-to-speech (TTS) synthesis models can produce monolingual speech with high intelligibility and naturalness. However, when the models are applied to synthesize code-switched (CS) speech, the performance declines seriously. Conventionally, developing a CS TTS system requires multilingual data to incorporate language-specific and cross-lingual knowledge. Recently, end-to-end (E2E) architecture has achieved satisfactory results in monolingual TTS. The architecture enables the training from one end of alphabetic text input to the other end of acoustic feature output. In this paper, we explore the use of E2E framework for CS TTS, using a combination of Mandarin and English monolingual speech corpus uttered by two female speakers. To handle alphabetic input from different languages, we explore two kinds of encoders: (1) shared multilingual encoder with explicit language embedding (LDE); (2) separated monolingual encoder (SPE) for each language. The two systems use identical decoder architecture, where a discriminative code is incorporated to enable the model to generate speech in one speaker's voice consistently. Experiments confirm the effectiveness of the proposed modifications on the E2E TTS framework in terms of quality and speaker similarity of the generated speech. Moreover, our proposed systems can generate controllable foreign-accented speech at character-level using only mixture of monolingual training data.

A Novel Hmm-Based Tts System Using Both Continuous Hmms And Discrete Hmms

A Novel HTS System Using both Continuous HMMs and Discrete HMMs

HMM-based Smoothing for Concatenative Speech Synthesis.

Improving voice quality of HMM-based speech synthesis using voice conversion method

Robust Speaker-Adaptive HMM-Based Text-to-Speech Synthesis

Realization of improved HMM-based speech synthesis system

Continuous F0 Modeling for HMM Based Statistical Parametric Speech Synthesis

An HMM-based Cantonese speech synthesis system

Joint Modelling of Voicing Label and Continuous F0 for HMM Based Speech Synthesis

Amplitude Spectrum Based Excitation Model For Hmm-Based Speech Synthesis

HMM based speech synthesis with Global Variance Training method

High Fidelity Text-to-Speech Via Discrete Tokens Using Token Transducer and Group Masked Language Model

End-to-end Code-switched TTS with Mix of Monolingual Recordings.

Improving HMM Based Speech Synthesis by Reducing Over-Smoothing Problems

Formant-Controlled HMM-Based Speech Synthesis.

An HMM Compensatioon Approach for Dynamic Features Using Unscented Transformation and Its Application to Noisy Speech Recognition

Study for English Synthesis with Improved HMM Synthesis Systems

CM-TTS: Enhancing Real Time Text-to-Speech Synthesis Efficiency through Weighted Samplers and Consistency Models

An Unified and Automatic Approach of Mandarin HTS System.

Probablistic Modelling of F0 in Unvoiced Regions in HMM Based Speech Synthesis

Statistical Modification Based Post-Filtering Technique for HMM-based Speech Synthesis