Abstract:Comparing with traditional text-to-speech (TTS) systems, conversational TTS systems are required to synthesize speeches with proper speaking style confirming to the conversational context. However, state-of-the-art context modeling methods in conversational TTS only model the textual information in context with a recurrent neural network (RNN). Such methods have limited ability in modeling the inter-speaker influence in conversations, and also neglect the speaking styles and the intra-speaker inertia inside each speaker. Inspired by DialogueGCN and its superiority in modeling such conversational influences than RNN based approaches, we propose a graph-based multi-modal context modeling method and adopt it to conversational TTS to enhance the speaking styles of synthesized speeches. Both the textual and speaking style information in the context are extracted and processed by DialogueGCN to model the inter- and intra-speaker influence in conversations. The outputs of DialogueGCN are then summarized by attention mechanism, and converted to the enhanced speaking style for current utterance. An English conversation corpus is collected and annotated for our research and released to public. Experiment results on this corpus demonstrate the effectiveness of our proposed approach, which outperforms the state-of-the-art context modeling method in conversational TTS in both MOS and ABX preference rate.

CARD Based Context Specified Weights Training Algorism for Unit Selection in Speech Synthesis

Perceptual Evaluation Weight Training for Text-to-Speech Synthesis

Perceptual Clustering Based Unit Selection Optimization for Concatenative Text-to-speech Synthesis

A Decision Tree Based Approach for Construction of Speech Database and Unit Selection

Optimization Method for Unit Selection Speech Synthesis Based on Synthesis Quality Predictions

Context features based pre-selection and weight prediction in concatenation speech synthesis system

Statistical Acoustic Model Based Unit Selection Algorithm for Speech Synthesis

DECISION TREE BASED UNIT PRE-SELECTION IN MANDARIN CHINESE SYNTHESIS

A novel unit selection method for concatenation speech system using similarity measure

Script Design Based on Decision Tree with Context Vector and Acoustic Distance for Mandarin TTS.

Trainable Unit Selection Speech Synthesis under Statistical Framework

Restricted Boltzmann Machine Based Spectrum Modeling and Unit Selection Speech Synthesis Method

Corpus Analysis Based on Decision Tree

A Novel Re-weighted CTC Loss for Data Imbalance in Speech Keyword Spotting

Improved unit selection speech synthesis method utilizing subjective evaluation results on synthetic speech

An initial attempt to improve spoken term detection by learning optimal weights for different indexing features

Unit Selection Speech Synthesis Integrating Automatic Error Detection

Automatic Segmentation for TTS Units

Enhancing Speaking Styles in Conversational Text-to-Speech Synthesis with Graph-based Multi-modal Context Modeling

HMM-based Unit Selection Speech Synthesis Using Log Likelihood Ratios Derived from Perceptual Data

The USTC and iFlytek Speech Synthesis Systems for Blizzard Challenge 2007