Abstract:It is well known that LLM-based systems are data-hungry. Recent LLM-based TTS works typically employ complex data processing pipelines to obtain high-quality training data. These sophisticated pipelines require excellent models at each stage (e.g., speech denoising, speech enhancement, speaker diarization, and punctuation models), which themselves demand high-quality training data and are rarely open-sourced. Even with state-of-the-art models, issues persist, such as incomplete background noise removal and misalignment between punctuation and actual speech pauses. Moreover, the stringent filtering strategies often retain only 10-30\% of the original data, significantly impeding data scaling efforts. In this work, we leverage a noise-robust audio tokenizer (S3Tokenizer) to design a simplified yet effective TTS data processing pipeline that maintains data quality while substantially reducing data acquisition costs, achieving a data retention rate of over 50\%. Beyond data scaling challenges, LLM-based TTS systems also incur higher deployment costs compared to conventional approaches. Current systems typically use LLMs solely for text-to-token generation, while requiring separate models (e.g., flow matching models) for token-to-waveform generation, which cannot be directly executed by LLM inference engines, further complicating deployment. To address these challenges, we eliminate redundant modules in both LLM and flow components, replacing the flow model backbone with an LLM architecture. Building upon this simplified flow backbone, we propose a unified architecture for both streaming and non-streaming inference, significantly reducing deployment costs. Finally, we explore the feasibility of unifying TTS and ASR tasks using the same data for training, thanks to the simplified pipeline and the S3Tokenizer that reduces the quality requirements for TTS training data.

A Miniature Chinese TTS System Based on Tailored Corpus

A Chinese Text-to-Speech System

WenetSpeech4TTS: A 12,800-hour Mandarin TTS Corpus for Large Speech Generation Model Benchmark

An HMM-based Cantonese speech synthesis system

End-to-end Code-switched TTS with Mix of Monolingual Recordings.

Design of Speech Corpus for Mandarin Text to Speech

TH-CoSS,a Mandarin Speech Corpus for TTS

DOP-Tacotron: a Fast Chinese TTS System with Local-based Attention

AISHELL-3: A Multi-speaker Mandarin TTS Corpus and the Baselines

Mandarin Text-to-Speech Front-End with Lightweight Distilled Convolution Network

A Preliminary Study on Deep Learning-based Chinese Text to Taiwanese Speech Synthesis System

A Decision Tree Based Approach for Construction of Speech Database and Unit Selection

A New Chinese Text-to-speech System with High Naturalness

Initial-Final Based Embedded Mandarin TTS System

SR-TTS: a rhyme-based end-to-end speech synthesis system

Script Design Based on Decision Tree with Context Vector and Acoustic Distance for Mandarin TTS.

Text-To-Visual Speech in Chinese Based on Data-Driven Approach

Bailing-TTS: Chinese Dialectal Speech Synthesis Towards Human-like Spontaneous Representation

TouchTTS: An Embarrassingly Simple TTS Framework that Everyone Can Touch

Seed-TTS: A Family of High-Quality Versatile Speech Generation Models

Improving Audible Discontinuities in Tts over Wireless Networks System