Abstract:Generating speech across different accents while preserving speaker identity is crucial for various real-world applications. However, accurately and independently modeling both speaker and accent characteristics in text-to-speech (TTS) systems is challenging due to the complex variations of accents and the inherent entanglement between speaker and accent identities. In this paper, we propose a novel approach for multi-speaker multi-accent TTS synthesis that aims to synthesize speech for multiple speakers, each with various accents. Our approach employs a multi-scale accent modeling strategy to address accent variations on different levels. Specifically, we introduce both global (utterance level) and local (phoneme level) accent modeling to capture overall accent characteristics within an utterance and fine-grained accent variations across phonemes, respectively. To enable independent control of speakers and accents, we use the speaker embedding to represent speaker identity and achieve speaker-independent accent control through speaker disentanglement within the multi-scale accent modeling. Additionally, we present a local accent prediction model that enables our system to generate accented speech directly from phoneme inputs. We conduct extensive experiments on an English accented speech corpus. Experimental results demonstrate that our proposed system outperforms baseline systems in terms of speech quality and accent rendering for generating multi-speaker multi-accent speech. Ablation studies further validate the effectiveness of different components in our proposed system.

Semi-supervised Accent Detection and Modeling

Modeling Syllable-Based Pronunciation Variation for Accented Mandarin Speech Recognition

Study of Semi-supervised Approaches to Improving English-Mandarin Code-Switching Speech Recognition

Acoustic Model Refining Algorithm for Pronunciation Quality Evaluation

Self-supervised Learning Representation Based Accent Recognition with Persistent Accent Memory

Accent Recognition with Hybrid Phonetic Features

Asymmetric Acoustic Model for Accented Speech Recognition

END-TO-END MULTI-ACCENT SPEECH RECOGNITION WITH UNSUPERVISED ACCENT MODELLING

A English Pronunciation Error Detection and Quality Evaluation Model Based on English Text

Reliable Accent-Specific Unit Generation With Discriminative Dynamic Gaussian Mixture Selection for Multi-Accent Chinese Speech Recognition

Using Cepstral and Prosodic Features for Chinese Accent Identification

Multi-Scale Accent Modeling and Disentangling for Multi-Speaker Multi-Accent Text-to-Speech Synthesis

An Mandarin Pronunciation Quality Assessment System Using Two Kinds of Acoustic Models

Towards Robust Mispronunciation Detection and Diagnosis for L2 English Learners with Accent-Modulating Methods

Joint Modeling of Accents and Acoustics for Multi-Accent Speech Recognition

A Bi-directional Attention Based End to End Mispronunciation Detection and Diagnosis of Mandarin

Effective Acoustic Modeling for Pronunciation Quality Scoring of Strongly Accented Mandarin Speech

Refining Context-Dependent Tonal Acoustic Modeling in Mandarin LVCSR

Improving Accent Conversion with Reference Encoder and End-To-End Text-To-Speech

Unsupervised Accent Adaptation Through Masked Language Model Correction Of Discrete Self-Supervised Speech Units

Reliable accent specific unit generation with dynamic Gaussian mixture selection for multi-accent speech recognition