Abstract:Accented text-to-speech (TTS) synthesis seeks to generate speech with an accent (L2) as a variant of the standard version (L1), which is challenging as L2 is different from L1 in terms of phonetic rendering and prosody pattern (pitch, energy, and duration variance, etc.). Accented TTS has several significant real-world applications, such as language learning, preserving and documenting endangered languages and dialects, etc. that make it an important area of research and development. Moreover, changing the accent intensity of any conversational AI system has the potential to allow specific users to understand its produced speech better. However, there is no intuitive solution for the control of the accent intensity for an utterance at both fine and coarse-grained levels, that are phoneme and utterance levels respectively. In this work, we propose a neural TTS architecture that allows us to control the accent style and its intensity. This is achieved through two novel mechanisms: 1) the front-end and back-end accent knowledge injection mechanism to enhance the accent interpretability of TTS modeling; and 2) an automatic speech recognition (ASR) based accent intensity modeling strategy to quantify the accent intensity in both L2 phoneme and utterance levels. In the front-end, a new accent variation adaptor seeks to project the accent-aware pitch, energy and duration features at a phoneme level, with the help of the fine-grained accent intensity information; In the back-end, a consistency constraint module that ensures the synthesized L2 speech manifests the expected accent intensity, is injected in the front-end, precisely. Experiments show that the proposed system attains superior performance to the baseline models in terms of accent rendering and intensity control. To our knowledge, this is the first study of accented TTS with explicit intensity control at both fine and coarse-grained levels.

QI-TTS: Questioning Intonation Control for Emotional Speech Synthesis

Fine-grained Emotional Control of Text-To-Speech: Learning To Rank Inter- And Intra-Class Emotion Intensities

Toward Synthesizing Expressive Mandarin Speech

Interactive Text-to-Speech via Semi-supervised Style Transfer Learning

Hierarchical Control of Emotion Rendering in Speech Synthesis

Into-TTS : Intonation Template Based Prosody Control System

EE-TTS: Emphatic Expressive TTS with Linguistic Information

InstructTTS: Modelling Expressive TTS in Discrete Latent Space with Natural Language Style Prompt

Controllable Accented Text-to-Speech Synthesis with Fine and Coarse-Grained Intensity Rendering

Towards Realistic Emotional Voice Conversion using Controllable Emotional Intensity

iEmoTTS: Toward Robust Cross-Speaker Emotion Transfer and Control for Speech Synthesis Based on Disentanglement Between Prosody and Timbre

Analysis of Chinese Interrogative Intonation and Its Synthesis in HMM-Based Synthesis System

MM-TTS: A Unified Framework for Multimodal, Prompt-Induced Emotional Text-to-Speech Synthesis

Emotional Dimension Control in Language Model-Based Text-to-Speech: Spanning a Broad Spectrum of Human Emotions

Quantitative Intonation Modeling of Interrogative Sentences for Mandarin Speech Synthesis

EmoMix: Emotion Mixing via Diffusion Models for Emotional Speech Synthesis

Explicit Intensity Control for Accented Text-to-speech.

RSET: Remapping-based Sorting Method for Emotion Transfer Speech Synthesis

Laugh Now Cry Later: Controlling Time-Varying Emotional States of Flow-Matching-Based Zero-Shot Text-to-Speech

A Study of Modeling Rising Intonation in Cantonese Neural Speech Synthesis

Enhancing Emotional Text-to-Speech Controllability with Natural Language Guidance through Contrastive Learning and Diffusion Models