Abstract:This work focuses on the development of expressive text-to-speech synthesis techniques for a Chinese spoken dialog system, where the expressivity is driven by the message content. We adapt the three-dimensional pleasure-displeasure, arousal-nonarousal and dominance-submissiveness (PAD) model for describing expressivity in input text semantics. The context of our study is based on response messages generated by a spoken dialog system in the tourist information domain. We use the P (pleasure) and A (arousal) dimensions to describe expressivity at the prosodic word level based on lexical semantics . The D (dominance) dimension is used to describe expressivity at the utterance level based on dialog acts . We analyze contrastive (neutral versus expressive) speech recordings to develop a nonlinear perturbation model that incorporates the PAD values of a response message to transform neutral speech into expressive speech. Two levels of perturbations are implemented-local perturbation at the prosodic word level, as well as global perturbation at the utterance level. Perceptual experiments involving 14 subjects indicate that the proposed approach can significantly enhance expressivity in response generation for a spoken dialog system.

Mandarin dialog prosody model

Modeling Incompletion Phenomenon in Mandarin Dialog Prosody.

Prosody modeling for mandarin exclamatory speech

Modeling Prosody Patterns for Chinese Expressive Text-to-speech Synthesis

A Novel Prosody Adaptation Method for Mandarin Concatenation-Based Text-to-speech System

Prosody Model for Mandarin Text-to-Speech System

Pitch Prediction for Mandarin TTS with Mutual Prosodic Constraint

Pitch Models of Mandarin Text-to-speech

An Optimized Neural Network Based Prosody Model of Chinese Speech Synthesis System

Prosodic Modeling with Rich Syntactic Context in HMM-based Mandarin Speech Synthesis

Prosody Modification on Mixed-language Speech Synthesis

Learning Prosodic Patterns for Mandarin Speech Synthesis

Modeling prosody pattern of Chinese expressive speech and its application in personalized speech conversion

Prosody Learning Mechanism for Speech Synthesis System Without Text Length Limit

Prosody Analysis And Modeling For Emotional Speech Synthesis

The Pause Duration Prediction for Mandarin Text-to-speech System

An Innovative Prosody Modeling Method for Chinese Speech Recognition

HIERARCHICAL PROSODY MODELING FOR NON-AUTOREGRESSIVE SPEECH SYNTHESIS

Modeling the Expressivity of Input Text Semantics for Chinese Text-to-Speech Synthesis in a Spoken Dialog System

Study of Prosody Model on Chinese Speech Synthesis Based on the Classification of Syllabic Prosody Features

Modeling the Acoustic Correlates of Dialog Act for Expressive Chinese TTS Synthesis