Abstract:Despite the rapid progress of automatic speech recognition (ASR) technologies targeting normal speech, accurate recognition of dysarthric and elderly speech remains highly challenging tasks to date. It is difficult to collect large quantities of such data for ASR system development due to the mobility issues often found among these users. To this end, data augmentation techniques play a vital role. In contrast to existing data augmentation techniques only modifying the speaking rate or overall shape of spectral contour, fine-grained spectro-temporal differences between dysarthric, elderly and normal speech are modelled using a novel set of speaker dependent (SD) generative adversarial networks (GAN) based data augmentation approaches in this paper. These flexibly allow both: a) temporal or speed perturbed normal speech spectra to be modified and closer to those of an impaired speaker when parallel speech data is available; and b) for non-parallel data, the SVD decomposed normal speech spectral basis features to be transformed into those of a target elderly speaker before being re-composed with the temporal bases to produce the augmented data for state-of-the-art TDNN and Conformer ASR system training. Experiments are conducted on four tasks: the English UASpeech and TORGO dysarthric speech corpora; the English DementiaBank Pitt and Cantonese JCCOCC MoCA elderly speech datasets. The proposed GAN based data augmentation approaches consistently outperform the baseline speed perturbation method by up to 0.91% and 3.0% absolute (9.61% and 6.4% relative) WER reduction on the TORGO and DementiaBank data respectively. Consistent performance improvements are retained after applying LHUC based speaker adaptation.

Towards Automatic Data Augmentation for Disordered Speech Recognition

Adversarial Data Augmentation Using VAE-GAN for Disordered Speech Recognition

Personalized Adversarial Data Augmentation for Dysarthric and Elderly Speech Recognition

Enhancing Pre-trained ASR System Fine-tuning for Dysarthric Speech Recognition using Adversarial Data Augmentation

Training Data Augmentation for Dysarthric Automatic Speech Recognition by Text-to-Dysarthric-Speech Synthesis

Adaptive data augmentation for mandarin automatic speech recognition

Improving Code-Switching and Named Entity Recognition in ASR with Speech Editing based Data Augmentation

Auditory-Based Data Augmentation for End-to-End Automatic Speech Recognition

Towards a Single ASR Model That Generalizes to Disordered Speech

Data Augmentation for End-to-end Code-switching Speech Recognition

Data Augmentation with Locally-time Reversed Speech for Automatic Speech Recognition

Generative Adversarial Networks Based Data Augmentation for Noise Robust Speech Recognition

Semantic Data Augmentation for End-to-End Mandarin Speech Recognition

Inclusive ASR for Disfluent Speech: Cascaded Large-Scale Self-Supervised Learning with Targeted Fine-Tuning and Data Augmentation

ROAR: Reinforcing Original to Augmented Data Ratio Dynamics for Wav2Vec2.0 Based ASR

SCADA: Stochastic, Consistent and Adversarial Data Augmentation to Improve ASR

CyclicAugment: Speech Data Random Augmentation with Cosine Annealing Scheduler for Automatic Speech Recognition

Data Augmentation using Conditional Generative Adversarial Networks for Robust Speech Recognition

Data Augmentation for Deep Learning Based a Utomaticmo Dulation Recognition

SpecSwap: A Simple Data Augmentation Method for End-to-End Speech Recognition

Automatically Learning Data Augmentation Policies for Dialogue Tasks