Improving Accented Speech Recognition using Data Augmentation based on Unsupervised Text-to-Speech Synthesis

Cong-Thanh Do,Shuhei Imai,Rama Doddipatla,Thomas Hain

2024-07-05

Abstract:This paper investigates the use of unsupervised text-to-speech synthesis (TTS) as a data augmentation method to improve accented speech recognition. TTS systems are trained with a small amount of accented speech training data and their pseudo-labels rather than manual transcriptions, and hence unsupervised. This approach enables the use of accented speech data without manual transcriptions to perform data augmentation for accented speech recognition. Synthetic accented speech data, generated from text prompts by using the TTS systems, are then combined with available non-accented speech data to train automatic speech recognition (ASR) systems. ASR experiments are performed in a self-supervised learning framework using a Wav2vec2.0 model which was pre-trained on large amount of unsupervised accented speech data. The accented speech data for training the unsupervised TTS are read speech, selected from L2-ARCTIC and British Isles corpora, while spontaneous conversational speech from the Edinburgh international accents of English corpus are used as the evaluation data. Experimental results show that Wav2vec2.0 models which are fine-tuned to downstream ASR task with synthetic accented speech data, generated by the unsupervised TTS, yield up to 6.1% relative word error rate reductions compared to a Wav2vec2.0 baseline which is fine-tuned with the non-accented speech data from Librispeech corpus.

Computation and Language,Sound,Audio and Speech Processing

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to use unsupervised text - to - speech synthesis (TTS) technology to enhance the performance of accent - aware speech recognition (ASR). Specifically, the researchers explored how to train a TTS system using a small amount of accented speech data (without manual transcription) and generate synthetic accented speech data to expand the training data set, thereby improving the performance of the ASR system. This method can effectively use accented speech data without manual transcription for data augmentation, and then improve the recognition effect of accented speech. The paper mentioned that the synthetic accented speech data generated by this method was combined with the existing non - accented speech data to train the Wav2vec2.0 model based on the self - supervised learning framework. The experimental results showed that the Wav2vec2.0 model fine - tuned with these synthetic accented speech data could achieve a relative word error rate (WER) reduction of up to 6.1% when processing accented spoken dialogues compared to the baseline model trained only with non - accented speech data. This indicates that even when only a small amount of accented speech data is available, the synthetic data generated by unsupervised TTS can significantly improve the performance of the ASR system.

Improving Accented Speech Recognition using Data Augmentation based on Unsupervised Text-to-Speech Synthesis

You Do Not Need More Data: Improving End-To-End Speech Recognition by Text-To-Speech Data Augmentation

Improving Speech Recognition with Augmented Synthesized Data and Conditional Model Training

Enhancing Pre-trained ASR System Fine-tuning for Dysarthric Speech Recognition using Adversarial Data Augmentation

Improving Speech Recognition Using GAN-Based Speech Synthesis and Contrastive Unspoken Text Selection

Generating Synthetic Audio Data for Attention-Based Speech Recognition Systems

Training Data Augmentation for Dysarthric Automatic Speech Recognition by Text-to-Dysarthric-Speech Synthesis

Pre-training Techniques for Improving Text-to-Speech Synthesis by Automatic Speech Recognition Based Data Enhancement

Effective Data Augmentation Methods for Neural Text-to-Speech Systems

Making More of Little Data: Improving Low-Resource Automatic Speech Recognition Using Data Augmentation

TTS-by-TTS: TTS-Driven Data Augmentation for Fast and High-Quality Speech Synthesis

Synthetic Cross-accent Data Augmentation for Automatic Speech Recognition

Almost Unsupervised Text to Speech and Automatic Speech Recognition

Zero Shot Text to Speech Augmentation for Automatic Speech Recognition on Low-Resource Accented Speech Corpora

Auditory-Based Data Augmentation for End-to-End Automatic Speech Recognition

Accented Text-to-Speech Synthesis with Limited Data

Towards Transfer Learning for End-to-End Speech Synthesis from Deep Pre-Trained Language Models.

Enhancing Low-Resource ASR through Versatile TTS: Bridging the Data Gap

Text Generation with Speech Synthesis for ASR Data Augmentation