Abstract:The modeling of fundamental frequency, or F0, in HMM-based speech synthesis is a critical factor in delivering speech which is both natural and accurately conveys all of the many nuances of the message. However, F0 modeling is difficult because F0 values are normally considered to depend on a binary voicing decision such that they are continuous in voiced regions and undefined in unvoiced regions. F0 is therefore a discontinuous function of time. Multi-space probability distribution HMM (MSDHMM) is a widely used solution to this problem. The MSDHMM essentially uses a joint distribution of discrete voicing labels and the discontinuous F0 observations. However, due to the discontinuity assumption, the MSDHMM provides a rather weak F0 trajectory model. In this paper, F0 is viewed as being a continuous function of time and this is achieved by assuming that F0 can be observed within unvoiced regions as well as voiced regions. This provides a continuous F0 data stream which can be modeled by standard HMMs. Voicing labels are modeled either implicitly or explicitly in order to perform voicing classification and a globally tied distribution (GTD) technique is used to achieve robust F0 estimation. Both objective measures and subjective listening tests demonstrate that continuous F0 modeling yields better synthesized F0 trajectories and significant improvements to the naturalness of synthesized speech compared to using the MSDHMM model.

Improving F0 prediction using bidirectional associative memories and syllable-level F0 features for HMM-based Mandarin speech synthesis

F0 Transformation for Emotional Speech Synthesis Using Target Approximation Features and Bidirectional Associative Memories

Tone Generation by Maximizing Joint Likelihood of Syllabic HMMs for Mandarin Speech Synthesis

Asynchronous F0 and Spectrum Modeling for HMM-based Speech Synthesis

Statistical modeling of syllable-level F0 features for HMM-based unit selection speech synthesis

Multi-Layer F0 Modeling for HMM-Based Speech Synthesis

Improved Mandarin Speech Recognition by Lattice Rescoring with Enhanced Tone Models

A Hierarchical F0 Modeling Method for HMM-based Speech Synthesis

Continuous F0 Modeling for HMM Based Statistical Parametric Speech Synthesis

Auditive Learning Based Chinese F0 Prediction

A Novel Hybrid Approach for Mandarin Speech Synthesis

Robust F0 Modeling for Mandarin Speech Recognition in Noise.

Modeling F0 Trajectories in Hierarchically Structured Deep Neural Networks.

Joint Modelling of Voicing Label and Continuous F0 for HMM Based Speech Synthesis

Voice Conversion Using Improved Spectral and F0 Transformation Methods.

Probablistic Modelling of F0 in Unvoiced Regions in HMM Based Speech Synthesis

Towards the Automatic Extraction of Fujisaki Model Parameters for Mandarin

Experiment on pitch target approximation model for generating Mandarin F0 contour

Generation of Fundamental Frequency Contours for Mandarin Speech Synthesis Based on Tone Nucleus Model

Direct F0 Incorporation for Acoustic Modeling in Mandarin Speech Recognition

Effective Acoustic Modeling for Pronunciation Quality Scoring of Strongly Accented Mandarin Speech