Abstract:This paper describes the Microsoft end-to-end neural text to speech (TTS) system: DelightfulTTS for Blizzard Challenge 2021. The goal of this challenge is to synthesize natural and high-quality speech from text, and we approach this goal in two perspectives: The first is to directly model and generate waveform in 48 kHz sampling rate, which brings higher perception quality than previous systems with 16 kHz or 24 kHz sampling rate; The second is to model the variation information in speech through a systematic design, which improves the prosody and naturalness. Specifically, for 48 kHz modeling, we predict 16 kHz mel-spectrogram in acoustic model, and propose a vocoder called HiFiNet to directly generate 48 kHz waveform from predicted 16 kHz mel-spectrogram, which can better trade off training efficiency, modelling stability and voice quality. We model variation information systematically from both explicit (speaker ID, language ID, pitch and duration) and implicit (utterance-level and phoneme-level prosody) perspectives: 1) For speaker and language ID, we use lookup embedding in training and inference; 2) For pitch and duration, we extract the values from paired text-speech data in training and use two predictors to predict the values in inference; 3) For utterance-level and phoneme-level prosody, we use two reference encoders to extract the values in training, and use two separate predictors to predict the values in inference. Additionally, we introduce an improved Conformer block to better model the local and global dependency in acoustic model. For task SH1, DelightfulTTS achieves 4.17 mean score in MOS test and 4.35 in SMOS test, which indicates the effectiveness of our proposed system

Mongolian Text-to-Speech Challenge Under Low-Resource Scenario for NCMMSC2022

The NTU-AISG Text-to-speech System for Blizzard Challenge 2020

Low-Resource Mongolian Speech Synthesis Based on Automatic Prosody Annotation

MnTTS: An Open-Source Mongolian Text-to-Speech Synthesis Dataset and Accompanied Baseline

MnTTS2: An Open-Source Multi-Speaker Mongolian Text-to-Speech Synthesis Dataset

EM-TTS: Efficiently Trained Low-Resource Mongolian Lightweight Text-to-Speech

Towards High-Quality Neural TTS for Low-Resource Languages by Learning Compact Speech Representations

MuLanTTS: The Microsoft Speech Synthesis System for Blizzard Challenge 2023

The huya multi-speaker and multi-style speech synthesis system for m2voc challenge 2020

MSV Challenge 2022: NPU-HC Speaker Verification System for Low-resource Indian Languages

End-to-end Code-switched TTS with Mix of Monolingual Recordings.

Multi-Speaker Multi-Lingual VQTTS System for LIMMITS 2023 Challenge

The VoiceMOS Challenge 2022

ZMM-TTS: Zero-shot Multilingual and Multispeaker Speech Synthesis Conditioned on Self-supervised Discrete Speech Representations

Code-Mixed Text to Speech Synthesis under Low-Resource Constraints

DelightfulTTS: the Microsoft Speech Synthesis System for Blizzard Challenge 2021

Rethinking MUSHRA: Addressing Modern Challenges in Text-to-Speech Evaluation

Exploring TTS without T Using Biologically/Psychologically Motivated Neural Network Modules (ZeroSpeech 2020)

MM-TTS: A Unified Framework for Multimodal, Prompt-Induced Emotional Text-to-Speech Synthesis

The VoiceMOS Challenge 2023: Zero-shot Subjective Speech Quality Prediction for Multiple Domains

An Initial Investigation of Language Adaptation for TTS Systems under Low-resource Scenarios