Abstract:This paper proposes to select frame-sized speech segments for waveform concatenation speech synthesis using neural network based acoustic models. First, a deep neural network (DNN) based frame selection method is presented. In this method, three DNNs are adopted to calculate target costs and concatenation costs respectively for selecting candidate frames of 5ms length. One DNN is built in the same way as the DNN-based statistical parametric speech synthesis, which predicts target acoustic features given linguistic context inputs. The distance between the acoustic features of a candidate unit and the predicted ones for a target unit is calculated as the target cost. The other two DNNs are constructed to predict the acoustic features at current frame using its context features and the acoustic features of preceding frames. At synthesis time, these two DNNs are employed to calculate the concatenation cost for each candidate frame given its preceding ones. Furthermore, recurrent neural networks (RNNs) with long short-term memory (LSTM) cells are adopted to replace DNNs for acoustic modeling in order to make better use of the sequential information. A strategy of using multi-frame instead of single frame as the basic unit for selection is also presented to reduce the concatenation points within synthetic speech. Experimental results show that our proposed method can achieve better naturalness than the hidden Markov model (HMM)-based frame selection method and the HMM-based parametric speech synthesis method.

A data driven method for target and concatenation cost calculation with KL-Divergence in Mandarin hybrid speech synthesis

A Novel Hybrid Approach for Mandarin Speech Synthesis

Deep Metric Learning For The Target Cost In Unit-Selection Speech Synthesizer

HMM-Based Hierarchical Unit Selection Combining Kullback-Leibler Divergence with Likelihood Criterion

A Novel Hybrid Mandarin Speech Synthesis System Using Different Base Units for Model Training and Concatenation

Context features based pre-selection and weight prediction in concatenation speech synthesis system

Korean Text-to-speech and Concatenation Cost Function

DNN-based unit selection using frame-sized speech segments

A Hierarchical Viterbi Algorithm For Mandarin Hybrid Speech Synthesis System

HMM-BASED HIERARCHICALUNITSELECTIONCOMBINING KULLBACK-LEIBLER DIVERGENCE WITH LIKELIHOODCRITERION

Perceptual Clustering Based Unit Selection Optimization for Concatenative Text-to-speech Synthesis

HMM-based Unit Selection Speech Synthesis Using Log Likelihood Ratios Derived from Perceptual Data

HMM-based Unit Selection Using F

Minimum Kullback–Leibler Divergence Parameter Generation for HMM-Based Speech Synthesis

A novel unit selection method for concatenation speech system using similarity measure

Statistical modeling of syllable-level F0 features for HMM-based unit selection speech synthesis

Unit Selection Speech Synthesis Using Frame-Sized Speech Segments and Neural Network Based Acoustic Models

Hybrid Model Method for Automatic Segmentation of Mandarin TTS Corpus

Extracting Unit Embeddings Using Sequence-To-Sequence Acoustic Models for Unit Selection Speech Synthesis

Statistical Acoustic Model Based Unit Selection Algorithm for Speech Synthesis

Progressive Neural Networks Based Features Prediction for the Target Cost in Unit-Selection Speech Synthesizer