Abstract:Non-autoregressive text to speech (NAR-TTS) models have attracted much attention from both academia and industry due to their fast generation speed. One limitation of NARTTS models is that they ignore the correlation in time and frequency domains while generating speech mel-spectrograms, and thus cause blurry and over-smoothed results. In this work, we revisit this over-smoothing problem from a novel perspective: the degree of over-smoothness is determined by the gap between the complexity of data distributions and the capability of modeling methods. Both simplifying data distributions and improving modeling methods can alleviate the problem. Accordingly, we first study methods reducing the complexity of data distributions. Then we conduct a comprehensive study on NARTTS models that use some advanced modeling methods. Based on these studies, we find that 1) methods that provide additional condition inputs reduce the complexity of data distributions to model, thus alleviating the oversmoothing problem and achieving better voice quality. 2) Among advanced modeling methods, Laplacian mixture loss performs well at modeling multimodal distributions and enjoys its simplicity, while GAN and Glow achieve the best voice quality while suffering from increased training or model complexity. 3) The two categories of methods can be combined to further alleviate the over-smoothness and improve the voice quality. 4) Our experiments on the multi-speaker dataset lead to similar conclusions as above and providing more variance information can reduce the difficulty of modeling the target data distribution and alleviate the requirements for model capacity.

Comparison of Several Smoothing Methods in Statistical Language Model

Data Noising as Smoothing in Neural Network Language Models

A Similarity-Based Approach to Data Sparseness Problem of Chinese Language Modeling

Linear Interpolated Methods in Statistical Natural Language Processing

Comparative Study on Algorithms of Limited Corpus Language Model

An Empirical Study of Non-Stationary Ngram Model and its Smoothing Techniques

Improvement Comparison of Different Lattice-based Discriminative Training Methods in Chinese-monolingual and Chinese-English-bilingual Speech Recognition

A Comparative Study of Four Primary Statistical Models in Chinese Parsing

Improving Language Model Size Reduction Using Better Pruning Criteria

Revisiting Over-Smoothness in Text to Speech

Toward a Unified Approach to Statistical Language Modeling for Chinese

Performance Comparison of Several Speech Recognition Methods

ML-LMCL: Mutual Learning and Large-Margin Contrastive Learning for Improving ASR Robustness in Spoken Language Understanding

Comparative Analysis of Language Models for Linguistic Examination of Ancient Chinese Classics: A Case Study of Zuozhuan Corpus.

A Word Language Model Based Contextual Language Processing On Chinese Character Recognition

Smoothing Algorithm of the Task Adaptation Chinese N-gram Model

A similarity-based smoothing algorithm for Chinese language modeling and its application in pinyin-to-character conversion

Comparing Discrete and Continuous Space LLMs for Speech Recognition

User- Oriented Chinese Language Model and Its Machine Learning

An Efficient Approach of Language Model Applying in ASR Systems

Comparison of Modified Kneser-Ney and Witten-Bell Smoothing Techniques in Statistical Language Model of Bahasa Indonesia