Research and Implementation of Text-to-speech System for Chinese

Lianhong Cai,Huawu Wei

DOI: https://doi.org/10.1109/sipnn.1994.344844

1994-01-01

Abstract:The paper mainly introduces research based on a phonetic word-cutting algorithm and prosodic rules. An unlimited vocabulary text-to-speech system is introduced, which can output phoneme from texts stored in computer

What problem does this paper attempt to address?

A Chinese Text-to-Speech System

Weijun Chen,Jianmin Li,Fuzong Lin,Bo Zhang

DOI: https://doi.org/10.3321/j.issn:1002-8331.2000.09.001

2000-01-01

Abstract:This paper discusses the implementation of a typical Chinese Text-to-Speech op) system. Firstly, It presents the frame structure of the system and discusses all of the modules in detail, then analyzes its characteristics and problems existing. Lastly This paper does some experiments in order to improve the quality of the system: It puts forward a simple and effective algorithm to calculate the base period Of speech waveforms and verifis the importance of context in improving the naturalness of synthesis speech.
Design and implementation of a speaker recognition system

韩雁,陈利华

DOI: https://doi.org/10.3785/j.issn.1008-973X.2001.02.011

2001-01-01

Abstract:After a various of comparison among possible speech processingapproaches, we present and developed a speaker recognition system(SRS). The six Chinese single vowels are selected as the test elements of our models. The computer speech analysis techniques adopted in this system include short-time power and short-time average zero-cross rate test, cepstrum analysis and linear prediction coding, which are used to detect the start-end points of speech and to get the characteristic parameters of the speaker. This system is implemented under MATLAB and Windows environment and has perfect recognition results.
Design and Implementation of Word and Speech Libraries in Chinese Text-to-speech Conversion

潘春华,武港山

DOI: https://doi.org/10.3969/j.issn.1001-3695.2004.03.060

2004-01-01

Abstract:This paper describes the design and implementation of Chinese word library and speech library in Chinese text-to-speech conversion.
High quality Chinese text-to-speech system - BEYOND

Weijun Chen,Fuzong Lin,Jianmin Li,Jin Fang Yang

2001-01-01

Abstract:This paper presents a new Chinese text-to-speech system - BEYOND that produces natural synthetic Mandarin speech based on two relatively independent modules: prosodic structure analysis and speech synthesis, and using the CPML markup language as their interface. The distinguishing features and key technology for the system include: (1) A new markup language is proposed for labelling Chinese prosodic structure. (2) The required prosodic structure is derived from a two-stage procedure. The input sentence is first transformed into a word sequence with part-of-speech tags, and the prosodic structure is then obtained using a statistical model and some heuristic rules. (3) The speech synthesis module is based on the concatenation of recorded units and the units are selected according to their prosodic contexts.
A Preliminary Study on Deep Learning-based Chinese Text to Taiwanese Speech Synthesis System

Y. Liao,Chen-Ming Pan,Wen-Han Hsu,Wern-Jun Wang,Cheng-Jung Tseng

Abstract:This paper focuses on the development and implementation of a Chinese Text-to-Taiwanese speech synthesis system. The proposed system combines three deep neural network-based modules including (1) a sequence-to-sequence-based Chinese characters to Taiwan Minnanyu Luomazi Pinyin (shortened to as Tâi-lô) machine translation (called C2T from now on), (2) a Tacotron2-based Tâi-lô pinyin to spectrogram and (3) a WaveGlow-based spectrogram to speech waveform synthesis subsystems. Among them, the C2T module was trained using a Chinese-Taiwanese parallel corpus (iCorpus) and 9 dictionaries released by Academia Sinica and collected from internet, respectively. The Tacotron2 and Waveglow was tuned using a Taiwanese speech synthesis corpus (a female speaker, about 10 hours speech) recorded by Chunghwa Telecom Laboratories. At the same time, a demonstration Chinese Text-to-Taiwanese speech synthesis web page has also been implemented. From the experimental results, it was found that (1) the best syllable error rate (SER) of 6.53% was achieved by the C2T module, (2) and the average MOS score of the whole speech synthesis system evaluated by 20 listeners gains 4.30. These results confirm that the effectiveness of integration of C2T, Tacrtron2 and WaveGlow models. In addition, the real-time factor of the whole system achieved 1/3.5.

Computer Science,Linguistics
End-to-end Code-switched TTS with Mix of Monolingual Recordings.

Yuewen Cao,Xixin Wu,Songxiang Liu,Jianwei Yu,Xu Li,Zhiyong Wu,Xunying Liu,Helen Meng

DOI: https://doi.org/10.1109/icassp.2019.8682927

2019-01-01

Abstract:State-of-the-art text-to-speech (TTS) synthesis models can produce monolingual speech with high intelligibility and naturalness. However, when the models are applied to synthesize code-switched (CS) speech, the performance declines seriously. Conventionally, developing a CS TTS system requires multilingual data to incorporate language-specific and cross-lingual knowledge. Recently, end-to-end (E2E) architecture has achieved satisfactory results in monolingual TTS. The architecture enables the training from one end of alphabetic text input to the other end of acoustic feature output. In this paper, we explore the use of E2E framework for CS TTS, using a combination of Mandarin and English monolingual speech corpus uttered by two female speakers. To handle alphabetic input from different languages, we explore two kinds of encoders: (1) shared multilingual encoder with explicit language embedding (LDE); (2) separated monolingual encoder (SPE) for each language. The two systems use identical decoder architecture, where a discriminative code is incorporated to enable the model to generate speech in one speaker's voice consistently. Experiments confirm the effectiveness of the proposed modifications on the E2E TTS framework in terms of quality and speaker similarity of the generated speech. Moreover, our proposed systems can generate controllable foreign-accented speech at character-level using only mixture of monolingual training data.
A New Chinese Text-to-speech System with High Naturalness

RH Wang,QF Liu,DF Tang

DOI: https://doi.org/10.1109/icslp.1996.607886

1996-01-01

Abstract:Introduces a new Chinese text-to-speech system that produces far more natural and intelligible synthesized speech than existing systems. There are two distinguishing features in this system. One is the perfect prosodic rules that are made up from the linguistic knowledge and statistical results from a standard Chinese database. These rules are successfully used to modify the elemental synthesis units to get high naturalness while concatenating them into a sentence. The other feature is that the log-magnitude approximate (LMA) filter is used as the synthesis filter in the system. With the LMA filter, the prosody of the synthesized speech can be modified in a wide range while maintaining high intelligence and naturalness. In this paper, the formulated prosodic rules are presented and the LMA filter-based speech synthesis is described in detail
A Study on KD-863 Chinese Text-To-Speech

王仁华,倪晋富,刘庆峰

DOI: https://doi.org/10.3321/j.issn:1002-0470.1998.08.003

1998-01-01

Abstract:This paper presents the researched progress on the project of KD-863 Chinese text-to-speech. The main results include a quantitative F0 contour model, which has been proposed for analysis and synthesis of Chinese intonation, and a novel speech synthesizer based on the LMA filter. Furthermore, the idea of hearing quantization on the sound units has been proposed, and a Chinese text-to-speech system called KD-863 has been developed on the basis of combining it with a simplified prosodic model. The naturalness of the output of the system achieves the extent accepted by most of customers, which makes it possible to get into market from laboratory.
An HMM-based Cantonese speech synthesis system

Wang Xin,Wu Zhiyong

DOI: https://doi.org/10.1109/GHTCE.2012.6490141

2012-01-01

Abstract:This paper describes a Cantonese HMM-based speech synthesis system (HTS) using the general architecture of Crystal - a multilingual text-to-speech (TTS) framework developed in Tsinghua University. The generated synthesis engine of HTS has advantage of small footprint, the size of which is less than 7M bytes, and can be easily ported to embedded electronic devices such as smart-phones, set-top boxes, etc. Furthermore, the quality of the synthetic speech can be easily characterized by modifying the synthetic acoustic parameters of the proposed system. The result shows noticeable improvement in naturalness and smoother transition than the corpus-based unit-selection concatenative speech synthesis approach.
Linguistic Pbolems in text-to-speech conversion Processing of the Chinese

Gong Qunhu

1999-01-01

Abstract:Theoretically, phonetic data and phonologtical rules for use in text-to-speech conversion by computer are based on how character strings record speech strings. In combination with computer implementation strategy,this paper discusses the complexity and main linguistic issues encountered in the course of convening Chinese text intostandard Chinese speech.
A Computer System for Chinese Character Speech Input

Huang Xuedong,Cai Lianhong,Fang Ditang,Chi Bianjin,Zhou Li,Jiang Li

DOI: https://doi.org/10.1007/bf02943310

IF: 1.871

1986-01-01

Journal of Computer Science and Technology

Abstract:In this paper, we introduce a speaker-dependent isolated word recognizer which is dedicated for Chinese character input. The method presented here offers an effective solution to the large-vocabulary recognition problem by carrying out recognition hierachically. The vocabulary consists of 800 to 1000 words. The average recognition rate is 90% when monosyllable words takes up one third of the vocabulary. Recognition rate can reach 95% by selecting from the top 20 candidates.
An Unified and Automatic Approach of Mandarin HTS System.

Yong Guan,Jilei Tian,Yi-Jian Wu,Junichi Yamagishi,Jani Nurminen

2010-01-01

Abstract:Most studies on Mandarin HTS (HMM-based text-to-speech system) have taken the initial/final as the basic acoustic units. It is, however, challenging to develop a multilingual HTS in a uniformed and consistent way since most of other languages use the phoneme as the basic phonetic unit. It becomes hard to apply cross-lingual adaptation which need map phonemes from each other, particularly in the case of unified ASR and HTS system due to the phoneme nature of most of the ASR systems. In this paper, we propose a phoneme based Mandarin HTS system, which has been systematically evaluated by comparing it with the initial/final system. The experimental results show that the use of phoneme as the acoustic unit for Mandarin HTS is a promising unified approach, thus enabling better and more uniform development with other languages while significantly reducing the number of acoustic units. The flat-start training scheme is also evaluated to show that the phoneme segmentation problem is solved without any performance degradation for phoneme based Mandarin HTS system. This performs an automatic approach without dependency with particular ASR system.
The Study of the Trainable Prosodic Model for Chinese Text to Speech System

陶建华,蔡莲红,赵世霞,吴志勇

DOI: https://doi.org/10.3321/j.issn:0371-0025.2001.01.012

2001-01-01

Abstract:Putonghua prosody is characterized by its hierarchical structure when influenced by linguistic environments. Based on this, a neural network, with specially weighted factors and optimizing outputs, is described and applied to construct the Putonghua prosodic model in Text-to-Speech (TTS) system. Extensive tests show that the structure of the neural network characterizes the Putonghua prosody more exactly than traditional models. Learning rate is speeded up and computational precision is improved, which makes the whole prosodic model more efficient. Furthermore, the paper also stylizes the Putonghua syllable pitch contours with SPiS parameters (Syllable Pitch Stylized Parameters), and analyzes them in adjusting the syllable pitch. It shows that the SPiS parameters effectively characterize the Putonghua syllable pitch contours, and facilitate the establishment of the network model and the prosodic controlling.
Voice Style Cloning for Chinese Speech

Haiyun Wang,Ziqi Chen,Luoyi Yang

Abstract:In this project, we focus on training and evaluating an end-to-end text-to-speech synthesis system named Tacotron. Different from traditional text-to-speech model which requires engineers to train three modules including a frontend text analyzer, an acoustic module

Computer Science,Linguistics
Grapheme-to-phoneme conversion in Chinese TTS system

Honghui Dong,Jianhua Tao,Bo Xu

DOI: https://doi.org/10.1109/CHINSL.2004.1409612

2004-01-01

Abstract:Phonetization is an important component in Chinese TTS systems. However, the polyphonic characters make this problem more complex. The paper reports a study on the relation between Chinese characters and their pronunciation, proposes the solution to the disambiguation of polyphonic characters, a dictionary-based method, and a rules-based method. In the rules-based method, we use the statistical decision list method. The phonetization plan has been proved effective experimentally. Improvements in the accuracy of polyphone phonetization are mostly over 10%.
Text-To-Visual Speech in Chinese Based on Data-Driven Approach

WANG Zhi-Ming,CAI Lian-Hong,AI Hai-zhou

DOI: https://doi.org/10.1360/jos161054

2005-01-01

Journal of Software

Abstract:Text-To-Visual speech (TTVS) synthesis by computer can increase the speech intelligibility and make the human-computer interaction interfaces more friendly. This paper describes a Chinese text-to-visual speech synthesis system based on data-driven (sample based) approach, which is realized by short video segments concatenation. An effective method to construct two visual confusion trees for Chinese initials and finals is developed. A co-articulation model based on visual distance and hardness factor is proposed, which can be used in the recording corpus sentence selection in analysis phase and the unit selection in synthesis phase. The obvious difference between boundary images of the concatenation video segments is smoothed by image morphing technique. By combining with the acoustic Text-To-Speech (TTS) synthesis, a Chinese text-to-visual speech synthesis system is realized.
Multi-speaker Chinese news broadcasting system based on improved Tacotron2

Wei Zhao,Yue Lian,Jianping Chai,Zhongwen Tu

DOI: https://doi.org/10.1007/s11042-023-15279-z

IF: 2.577

2023-05-04

Multimedia Tools and Applications

Abstract:In recent years, the demand for news broadcasting has increased with the explosion of information. The automatic news broadcasting system based on deep learning text-to-speech technology can solve the problems of working time limitation and errors caused by manual broadcasting. Most of the existing speech synthesis technologies cannot switch speakers in real-time and cannot solve a series of additional news broadcasting scenarios problems in Chinese. In this paper, we propose a multi-speaker Chinese news broadcasting system with switchable timbres based on our established Chinese news corpus CNews dataset for training. This system uses the CPM module to convert Chinese into pinyin phonemes more accurately. Then a timbre encoder is used to construct multi-speaker timbre feature embeddings. As for the problem of having long texts in news, the acoustic model of this system is improved based on Tacotron2 and uses Discrete Grave attention as the attention mechanism so that the model reduces the demand for audio data in the training phase and better extracts the information from the context. The HiFi-GAN vocoder is also used to generate time domain waveforms instead of the original WaveNet, reducing the synthesis time and improving the voice quality of the synthesized speech. Experiments show that the system can change the target timbre flexibly according to the reference speech compared with Tacotron2. Moreover, it is able to synthesize speech with the prosody and style of the target presenter under the training of limited data and with better naturalness as well as faster inference speed, which can be used for real-time news broadcasting.

computer science, information systems, theory & methods,engineering, electrical & electronic, software engineering
Text Enhancement for Paragraph Processing in End-to-End Code-switching TTS

Chunyu Qiang,Jianhua Tao,Ruibo Fu,Zhengqi Wen,Jiangyan Yi,Tao Wang,Shiming Wang

DOI: https://doi.org/10.48550/arXiv.2210.11429

2022-10-21

Abstract:Current end-to-end code-switching Text-to-Speech (TTS) can already generate high quality two languages speech in the same utterance with single speaker bilingual corpora. When the speakers of the bilingual corpora are different, the naturalness and consistency of the code-switching TTS will be poor. The cross-lingual embedding layers structure we proposed makes similar syllables in different languages relevant, thus improving the naturalness and consistency of generated speech. In the end-to-end code-switching TTS, there exists problem of prosody instability when synthesizing paragraph text. The text enhancement method we proposed makes the input contain prosodic information and sentence-level context information, thus improving the prosody stability of paragraph text. Experimental results demonstrate the effectiveness of the proposed methods in the naturalness, consistency, and prosody stability. In addition to Mandarin and English, we also apply these methods to Shanghaiese and Cantonese corpora, proving that the methods we proposed can be extended to other languages to build end-to-end code-switching TTS system.

Sound,Audio and Speech Processing
A Transformer-based Chinese Non-autoregressive Speech Synthesis Scheme

Yueqing Cai,Wenbi Rao

DOI: https://doi.org/10.1145/3459212.3459222

2021-03-19

Abstract:At present, the main research hotspot in the field of speech synthesis is still English speech synthesis, and there are few non-autoregressive Chinese speech synthesis models. During the Chinese migration process of FastSpeech2, we found that the naturalness of the synthesized audio was not good enough and there were some abnormal interruptions and incorrect pronunciation. Inspired by the training method of generative adversarial network, we use FastSpeech2 as the generator, and add a discriminator to force FastSpeech2 to generate audio more similar to the real audio. In order to realize a complete text to Mel spectrogram speech synthesis scheme, we design a text-to-phoneme converter based on corpus and rule constraints. And we conduct experiments on Baker dataset. The results show that our model achieves a better Mel Cepstral Distance than that of FastSpeech2. And our model can achieve a mean opinion score of 3.94, which is slightly better than the original model.
Design and Implementation of Chinese TTS Engine:SmartTalk

潘春华,武港山

DOI: https://doi.org/10.3969/j.issn.1001-3695.2004.04.050

2004-01-01

Abstract:With the popularity of Chinese and the development of speech technology, it is becoming important to let the computer speak Chinese. This paper concentrates on the design and implementation of a Chinese Text-to-Speech ( TTS) engine named SmallTalk. It deals mainly with the design of Chinese speech library and Chinese word library. We implement Chinese speech segmentation and speech synthesis based on word library and speech library.

Research and Implementation of Text-to-speech System for Chinese

A Chinese Text-to-Speech System

Design and implementation of a speaker recognition system

Design and Implementation of Word and Speech Libraries in Chinese Text-to-speech Conversion

High quality Chinese text-to-speech system - BEYOND

A Preliminary Study on Deep Learning-based Chinese Text to Taiwanese Speech Synthesis System

End-to-end Code-switched TTS with Mix of Monolingual Recordings.

A New Chinese Text-to-speech System with High Naturalness

A Study on KD-863 Chinese Text-To-Speech

An HMM-based Cantonese speech synthesis system

Linguistic Pbolems in text-to-speech conversion Processing of the Chinese

A Computer System for Chinese Character Speech Input

An Unified and Automatic Approach of Mandarin HTS System.

The Study of the Trainable Prosodic Model for Chinese Text to Speech System

Voice Style Cloning for Chinese Speech

Grapheme-to-phoneme conversion in Chinese TTS system

Text-To-Visual Speech in Chinese Based on Data-Driven Approach

Multi-speaker Chinese news broadcasting system based on improved Tacotron2

Text Enhancement for Paragraph Processing in End-to-End Code-switching TTS

A Transformer-based Chinese Non-autoregressive Speech Synthesis Scheme

Design and Implementation of Chinese TTS Engine:SmartTalk