Abstract:Mandarin is a tonal language, and it is important to preserve lexical tone information in synthesized speech. With natural speech, Chinese cochlear implant (CI) users have difficulty perceiving voice pitch cues important for lexical tone perception; it is unclear whether this difficulty persists in Mandarin synthesized speech. In this study, intelligibility of naturally produced and synthesized Mandarin speech was measured in Chinese CI listeners; intelligibility was also measured in a control group of normal-hearing (NH) listeners. Five synthesized voices were selected to represent different talker genders (male, female, child), speaking rates (normal, slow), and speaking styles (emotional, accent). The data showed that while modern Mandarin text-to-speech (TTS) systems can provide perfect speech intelligibility for NH listeners, overall intelligibility was much poorer for CI than for NH listeners. CI performance was significantly poorer with synthesized speech than with natural speech (p < 0.001). CI listeners were highly sensitive to the “extra-atypical” synthesized emotional and accented speech. Performance with each of the synthesized speech types was significantly correlated with performance with natural speech in CI users (p < 0.01 in all cases). While modern TTS systems offer educational and communication benefits to CI users and hearing-impaired individuals, the selection of synthesized voices should be carefully considered in education applications of TTS for hearing-impaired individuals, especially CI children, since poor intelligibility performance may affect language learning.

Mandarin Speech Synthesis Based on Pitch Synchronous Time-Frequency Interpolation

Intonation and Prosody Conversion for Expressive Mandarin Speech Synthesis

Learning Prosodic Patterns for Mandarin Speech Synthesis

Analysis and Synthesis of Continuous Voice Pitch Contour for Improving Chinese Synthetic Speech Naturalness

Improving F0 prediction using bidirectional associative memories and syllable-level F0 features for HMM-based Mandarin speech synthesis

An Optimized Neural Network Based Prosody Model of Chinese Speech Synthesis System

Pitch Prediction for Mandarin TTS with Mutual Prosodic Constraint

Duration optimization of speaker adaptation in Mandarin TTS

Pitch Models of Mandarin Text-to-speech

Mandarin-English Mixed TTS Based on HCSIPA

A Novel Method for Mandarin Speech Synthesis by Inserting Prosodic Structure Prediction into Tacotron2.

Standard Speaker Selection in Speech Synthesis for Mandarin Tone Learning

Chinese prosody stepwise synthesis based on FD & TD PSOLA

Intelligibility of Naturally Produced and Synthesized Mandarin Speech by Cochlear Implant Listeners

IMPROVING NATURALNESS AND CONTROLLABILITY OF SEQUENCE-TO-SEQUENCE SPEECH SYNTHESIS BY LEARNING LOCAL PROSODY REPRESENTATIONS

On Fundamental Frequency Contour Synthesis And Control Method For Chinese Speech Synthesis

A Novel Prosody Adaptation Method for Mandarin Concatenation-Based Text-to-speech System

Toward Synthesizing Expressive Mandarin Speech

Study of pitch shifting technology and the sound quality evaluating

Improving Prosody with Linguistic and Bert Derived Features in Multi-Speaker Based Mandarin Chinese Neural TTS

A New Chinese Text-to-speech System with High Naturalness