Abstract:In this paper, an automatic and unsupervised method using context-dependent hidden Markov models (CD-HMMs) is proposed for the prosodic labeling of speech synthesis databases. This method consists of three main steps, i.e., initialization, model training and prosodic labeling. The initial prosodic labels are obtained by unsupervised clustering using the acoustic features designed according to the characteristics of the prosodic descriptor to be labeled. Then, CD-HMMs of the spectral parameters, Rh and phone durations are estimated by a means similar to the HMM-based parametric speech synthesis using the initial prosodic labels. These labels are further updated by Viterbi decoding under the maximum likelihood criterion given the acoustic feature sequences and the trained CD-HMMs. The model training and prosodic labeling procedures are conducted iteratively until convergence. The performance of the proposed method is evaluated on Mandarin speech synthesis databases and two prosodic descriptors are investigated, i.e., the prosodic phrase boundary and the emphasis expression. In our implementation, the prosodic phrase boundary labels are initialized by clustering the durations of the pauses between every two consecutive prosodic words, and the emphasis expression labels are initialized by examining the differences between the original and the synthetic F0 trajectories. Experimental results show that the proposed method is able to label the prosodic phrase boundary positions much more accurately than the text-analysis-based method without requiring any manually labeled training data. The unit selection speech synthesis system constructed using the prosodic phrase boundary labels generated by our proposed method achieves similar performance to that using the manual labels. Furthermore, the unit selection speech synthesis system constructed using the emphasis expression labels generated by our proposed method can convey the emphasis information effectively while maintaining the naturalness of synthetic speech.

Prosodic Chinese Sign Language Synthesis Driven by Speech

Text-driven Visual Prosody Generation for Embodied Conversational Agents

Head Movement Synthesis Based on Semantic and Prosodic Features for a Chinese Expressive Avatar

CSLDS: Chinese Sign Language Dialog System

A Method to Synthesize Chinese Sign Language Based on Virtual Human Technologies

Human-Computer Chinese Sign Language Interaction System

Learning Prosodic Patterns for Mandarin Speech Synthesis

A Speech-Driven 3-D Lip Synthesis with Realistic Dynamics in Mandarin Chinese

Toward Synthesizing Expressive Mandarin Speech

CSLNSpeech: solving the extended speech separation problem with the help of Chinese Sign Language

Modeling Prosody Patterns for Chinese Expressive Text-to-speech Synthesis

CSLNSpeech: solving extended speech separation problem with the help of Chinese sign language

Prosodic Modeling with Rich Syntactic Context in HMM-based Mandarin Speech Synthesis

HandTalker II: a Chinese sign language recognition and synthesis system

Prosody Analysis And Modeling For Emotional Speech Synthesis

Chinese sign language recognition with adaptive HMM

Modeling the Expressivity of Input Text Semantics for Chinese Text-to-Speech Synthesis in a Spoken Dialog System

Mandarin Singing Voice Synthesis Based on Harmonic Plus Noise Model and Singing Expression Analysis

An Emotional Text-Driven 3D Visual Pronunciation System for Mandarin Chinese

Unsupervised Prosodic Labeling Of Speech Synthesis Databases Using Context-Dependent Hmms

Generating data for signer adaptation