Abstract:Prosody is an essential aspect of speech, as it carries both lexical and non-lexical information. A conventional approach for studying speech prosody is to collect and analyze F0 data based on certain hypotheses and then develop a theory based on the observation as the final conclusion of the study. This process is however far from complete, as the developed theory has not been actually tested for its ability to predict actual acoustic data. This paper presents PENTATrainer2, a prosody modeling tool based on the parallel encoding and target approximation framework. PENTATrainer2 can facilitate prosody studies in testing hypotheses and theories against speech data by using an automatic analysis-by-synthesis and stochastic learning algorithm. Users can flexibly design the annotation scheme based on their own hypotheses and determine whether the hypothesized categories can lead to accurate synthetic F0 contours. PENTATrainer2 consists of three main components: multi-layer annotation, target approximation, and stochastic optimization. First, acoustic data are annotated in parallel layers, each of which corresponds to a functional category that may affect F0 contours. These layers are then compiled into unique functional combinations. The combinations represent underlying invariant representations of communicative functions and their interaction with each other. Target approximation parameters of each combination are then learned through analysis-by-synthesis and stochastic optimization. Pilot tests of PENTATrainer 2 have been conducted on Thai, Mandarin and English. The results demonstrate not only high accuracy of the synthesized F0 contours but also distinctive contrasts in the distribution of pitch target parameters. This indicates the effectiveness of PENTATrainer2 in modeling speech prosody.

A Syllable-Based Prosody Modeling for L1 and L2 English Speeches

Modeling Prosody Patterns for Chinese Expressive Text-to-speech Synthesis

Prosodic Modeling with Rich Syntactic Context in HMM-based Mandarin Speech Synthesis

HIERARCHICAL PROSODY MODELING FOR NON-AUTOREGRESSIVE SPEECH SYNTHESIS

A Superposed Prosodic Model for Chinese Text-To-Speech Synthesis

An Innovative Prosody Modeling Method for Chinese Speech Recognition

Learning Prosodic Patterns for Mandarin Speech Synthesis

A New Prosodic Strength Calculation Method for Prosody Reduction Modeling

Prosody Learning Mechanism for Speech Synthesis System Without Text Length Limit

Modeling Incompletion Phenomenon in Mandarin Dialog Prosody.

Modeling prosody pattern of Chinese expressive speech and its application in personalized speech conversion

Study of Prosody Model on Chinese Speech Synthesis Based on the Classification of Syllabic Prosody Features

PENTATrainer2: A Hypothesis-Driven Prosody Modeling Tool

Prosody Modification on Mixed-language Speech Synthesis

Improving Prosody for Unseen Texts in Speech Synthesis by Utilizing Linguistic Information and Noisy Data

Modeling the cross-linguistic variations of tonal systems

Modeling Duration and Intonation in Mandarin Chinese Synthesis with a Neural Network

MODELLING THE GLOBAL ACOUSTIC CORRELATES OF EXPRESSIVITY FOR CHINESE TEXT-TO-SPEECH SYNTHESIS

Applying Syntax$\unicode{x2013}$Prosody Mapping Hypothesis and Prosodic Well-Formedness Constraints to Neural Sequence-to-Sequence Speech Synthesis

Modeling the Acoustic Correlates of Expre Expressive Text-to-Spe

Evaluating Prosody of Mandarin Spe