Abstract:Non-autoregressive text to speech (NAR-TTS) models have attracted much attention from both academia and industry due to their fast generation speed. One limitation of NARTTS models is that they ignore the correlation in time and frequency domains while generating speech mel-spectrograms, and thus cause blurry and over-smoothed results. In this work, we revisit this over-smoothing problem from a novel perspective: the degree of over-smoothness is determined by the gap between the complexity of data distributions and the capability of modeling methods. Both simplifying data distributions and improving modeling methods can alleviate the problem. Accordingly, we first study methods reducing the complexity of data distributions. Then we conduct a comprehensive study on NARTTS models that use some advanced modeling methods. Based on these studies, we find that 1) methods that provide additional condition inputs reduce the complexity of data distributions to model, thus alleviating the oversmoothing problem and achieving better voice quality. 2) Among advanced modeling methods, Laplacian mixture loss performs well at modeling multimodal distributions and enjoys its simplicity, while GAN and Glow achieve the best voice quality while suffering from increased training or model complexity. 3) The two categories of methods can be combined to further alleviate the over-smoothness and improve the voice quality. 4) Our experiments on the multi-speaker dataset lead to similar conclusions as above and providing more variance information can reduce the difficulty of modeling the target data distribution and alleviate the requirements for model capacity.

Improved katz smoothing for language modeling in speech recogniton

Improved Katz smoothing algorithms with POS information

A similarity-based smoothing algorithm for Chinese language modeling and its application in pinyin-to-character conversion

An Empirical Study of Non-Stationary Ngram Model and its Smoothing Techniques

Effective Acoustic Modeling for Pronunciation Quality Scoring of Strongly Accented Mandarin Speech

Modeling and Simulation of English Speech Rationality Optimization Recognition Based on Improved Particle Filter Algorithm

Improving HMM Based Speech Synthesis by Reducing Over-Smoothing Problems

Comparison of Several Smoothing Methods in Statistical Language Model

Improvement of Centric Clipping for Chinese Sound Tone Recognitionwith Adaptive Speaking Speed

Experimental Investigation of Mandarin Pronunciation Quality Assessment System

A Pitch Smoothing Method for Mandarin Tone Recognition

Revisiting Over-Smoothness in Text to Speech

A Word Language Model Based Contextual Language Processing On Chinese Character Recognition

An Approach of Fundamental Frequencies Smoothing for Chinese Tone Recognition

Acoustic Model Refining Algorithm for Pronunciation Quality Evaluation

Smoothing Algorithm of the Task Adaptation Chinese N-gram Model

A Modified Speech Enhancement Algorithm Using a Universal Speaker Model

An Innovative Prosody Modeling Method for Chinese Speech Recognition

Improvement in Text-Dependent Mispronunciation Detection for English Learners

Statistical Modification Based Post-Filtering Technique for HMM-based Speech Synthesis

Optimizing Data Usage for Low-Resource Speech Recognition