Abstract:End-to-end text-to-speech (TTS) synthesis is a method that directly converts input text to output acoustic features using a single network. A recent advance of end-to-end TTS is due to a key technique called attention mechanisms, and all successful methods proposed so far have been based on soft attention mechanisms. However, although network structures are becoming increasingly complex, end-to-end TTS systems with soft attention mechanisms may still fail to learn and to predict accurate alignment between the input and output. This may be because the soft attention mechanisms are too flexible. Therefore, we propose an approach that has more explicit but natural constraints suitable for speech signals to make alignment learning and prediction of end-to-end TTS systems more robust. The proposed system, with the constrained alignment scheme borrowed from segment-to-segment neural transduction (SSNT), directly calculates the joint probability of acoustic features and alignment given an input text. The alignment is designed to be hard and monotonically increase by considering the speech nature, and it is treated as a latent variable and marginalized during training. During prediction, both the alignment and acoustic features can be generated from the probabilistic distributions. The advantages of our approach are that we can simplify many modules for the soft attention and that we can train the end-to-end TTS model using a single likelihood function. As far as we know, our approach is the first end-to-end TTS without a soft attention mechanism.

Letter-to-sound Conversion Using Coupled Hidden Markov Models for Lexicon Compression

Letter-to-Sound Pronunciation Prediction Using Conditional Random Fields

Improving Letter-to-sound Conversion Performance with Automatically Generated New Words

End-to-end Code-switched TTS with Mix of Monolingual Recordings.

Phone-aware LSTM-RNN for Voice Conversion

Initial investigation of an encoder-decoder end-to-end TTS framework using marginalization of monotonic hard latent alignments

A Novel Hybrid Approach for Mandarin Speech Synthesis

High-Accuracy and Low-Latency Speech Recognition with Two-Head Contextual Layer Trajectory LSTM Model

Spectro-Temporal Modelling with Time-Frequency Lstm and Structured Output Layer for Voice Conversion

Label Transform Based Cross-Language Speaker Adaptation in Bilingual (Mandarin-English) TTS

Automatic Conversion from Lexical Words to Prosodic Words for Mandarin Text-to-speech System

A Novel Hybrid Mandarin Speech Synthesis System Using Different Base Units for Model Training and Concatenation

Enhancing Local Dependencies for Transformer-Based Text-to-Speech via Hybrid Lightweight Convolution

Syllable HMM Based Mandarin TTS and Comparison with Concatenative TTS.

LSTM Acoustic Models Learn to Align and Pronounce with Graphemes

MacST: Multi-Accent Speech Synthesis via Text Transliteration for Accent Conversion

HAM-TTS: Hierarchical Acoustic Modeling for Token-Based Zero-Shot Text-to-Speech with Model and Data Scaling

A Complementary Joint Training Approach Using Unpaired Speech and Text for Low-Resource Automatic Speech Recognition.

Improving the Performance of HMM-based Voice Conversion Using Context Clustering Decision Tree and Appropriate Regression Matrix Format.

A Cross-Language State Mapping Approach to Bilingual (Mandarin-English) TTS

Building Multi lingual TTS using Cross Lingual Voice Conversion