Abstract:The application of Tacotron model in Tibetan end-to-end speech synthesis has achieved good results, however, this model which is based Recurrent Neural Network (RNN) suffers from low training and prediction efficiency and long-range information loss. To further improve the effect of Tibetan speech synthesis, an end-to-end speech synthesis model based on Transformer is proposed to realize the speech synthesis of multiple dialects of Tibetan. In the model, the hidden state in the Encoder and Decoder is constructed in parallel by using the multi-head attention mechanism, which effectively solves the problem of modeling long-distance information correlation and can take the advantage of multi-GPU parallel training. In this work, three different granular units (Tibetan characters, Latin letters, and Tibetan components) are selected as the input to the acoustic model for the purpose of selecting the best granular unit to improve synthesis result. The Transformer TTS network transforms the text sequences into Mel spectrograms, and the WaveNet vocoder converts the Mel spectrograms into final speech waveforms. This study conducts a series of comparative experiments. Firstly, the performances of three synthesis granular units based on our proposed method are compared, and then the effects of single-GPU training and multi-GPU parallel training are compared. In addition, a comparative experiment of Tacotron and Transformer applied to Tibetan Lhasa dialect speech synthesis is conducted. The experimental results show that the end-to-end speech synthesis model based on Transformer has better performance than Tacotron in Tibetan Lhasa dialect speech synthesis. The speech obtained by using Latin letters as the synthesis unit and parallel training with multi-GPU has better clarity and naturalness.

Building Mongolian Tts Front-End With Encoder-Decoder Model By Using Bridge Method And Multi-View Features

ViSPer: A Multilingual TTS Approach Based on VITS Using Deep Feature Loss

End-to-end Code-switched TTS with Mix of Monolingual Recordings.

Unified Mandarin TTS Front-end Based on Distilled BERT Model

Building Multi lingual TTS using Cross Lingual Voice Conversion

A unified sequence-to-sequence front-end model for Mandarin text-to-speech synthesis

Bidirectional Decoding Tacotron for Attention Based Neural Speech Synthesis

Knowledge-based Linguistic Encoding for End-to-End Mandarin Text-to-Speech Synthesis

Bridging the Gap Between Pre-Training and Fine-Tuning for End-to-End Speech Translation

Mandarin Text-to-Speech Front-End with Lightweight Distilled Convolution Network

A unified front-end framework for English text-to-speech synthesis

Scalable Multilingual Frontend for TTS

A Novel Chinese Dialect TTS Frontend with Non-Autoregressive Neural Machine Translation

Prior-agnostic Multi-scale Contrastive Text-Audio Pre-training for Parallelized TTS Frontend Modeling

Forward-Backward Decoding for Regularizing End-to-End TTS

EM-TTS: Efficiently Trained Low-Resource Mongolian Lightweight Text-to-Speech

Transformer Speech Synthesis for Tibetan Lhasa Dialect Based on Multilevel Granular Unit

Improving Uyghur ASR systems with decoders using morpheme-based language models

Improve Bilingual TTS Using Dynamic Language and Phonology Embedding

ParaTTS: Learning Linguistic and Prosodic Cross-sentence Information in Paragraph-based TTS

Multi-channel Encoder for Neural Machine Translation