Abstract:Text-to-speech synthesis plays an essential role in facilitating human-computer interaction. Currently, the predominant approach in Text-to-speech acoustic models selects only the Mel spectrum as an intermediate feature for converting text to speech. However, the Mel spectrograms obtained may exhibit ambiguity in some aspects owing to the limited capability of the Fourier transform to capture mutation signals during the acquisition of the Mel spectrograms. With the aim of improving the clarity of synthesized speech, this study proposes a multi-task learning optimization method and conducts experiments on the Tacotron2 speech synthesis system to demonstrate the effectiveness of the proposed method. The method in the study introduces an additional task: wavelet spectrograms. The continuous wavelet transform has gained significant popularity in various applications, including speech enhancement and speech recognition, which is primarily attributed to its capability to adaptively vary the time-frequency resolution and its excellent performance in capturing non-stationary signals. This study highlights that the clarity of Tacotron2 synthesized speech can be improved by introducing Wavelet-spectrogram as an auxiliary task through theoretical and experimental analysis: a feature extraction network is added, and Wavelet-spectrogram features are extracted from the Mel spectrum output generated by the decoder. Experimental findings indicate that the Mean Opinion Score achieved for the speech synthesized by the model using multi-task learning is 0.17 higher compared to the baseline model. Furthermore, by analyzing the factors contributing to the success of the continuous wavelet transform-based multi-task learning method in the Tacotron2 model, as well as the effectiveness of multi-task learning, the study conjectures that the proposed method has the potential to enhance the performance of other acoustic models.

Improving Multi-Speaker Tacotron with Speaker Gating Mechanisms

Bidirectional Decoding Tacotron for Attention Based Neural Speech Synthesis

Deep Voice 2: Multi-Speaker Neural Text-to-Speech

Towards Transfer Learning for End-to-End Speech Synthesis from Deep Pre-Trained Language Models.

Monaural Multi-Talker Speech Recognition With Attention Mechanism And Gated Convolutional Networks

Multi-speaker Chinese news broadcasting system based on improved Tacotron2

Msdtron: a high-capability multi-speaker speech synthesis system for diverse data using characteristic information

DOP-Tacotron: a Fast Chinese TTS System with Local-based Attention

Optimizing feature fusion for improved zero-shot adaptation in text-to-speech synthesis

IMPROVING NATURALNESS AND CONTROLLABILITY OF SEQUENCE-TO-SEQUENCE SPEECH SYNTHESIS BY LEARNING LOCAL PROSODY REPRESENTATIONS

A multi-task learning speech synthesis optimization method based on CWT: a case study of Tacotron2

Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis

Improving End-to-End Single-Channel Multi-Talker Speech Recognition.

Neural Speech Synthesis with Transformer Network.

Modeling Prosodic Phrasing with Multi-Task Learning in Tacotron-based TTS

Teacher-Student Training For Robust Tacotron-Based TTS

TMS: Temporal multi-scale in time-delay neural network for speaker verification

A New GAN-based End-to-End TTS Training Algorithm

Close to Human Quality TTS with Transformer.

Multi-Task Adversarial Training Algorithm for Multi-Speaker Neural Text-to-Speech

Towards High-Quality Neural TTS for Low-Resource Languages by Learning Compact Speech Representations