Abstract:Despite the rapid progress of automatic speech recognition (ASR) technologies targeting normal speech, accurate recognition of dysarthric and elderly speech remains highly challenging tasks to date. It is difficult to collect large quantities of such data for ASR system development due to the mobility issues often found among these users. To this end, data augmentation techniques play a vital role. In contrast to existing data augmentation techniques only modifying the speaking rate or overall shape of spectral contour, fine-grained spectro-temporal differences between dysarthric, elderly and normal speech are modelled using a novel set of speaker dependent (SD) generative adversarial networks (GAN) based data augmentation approaches in this paper. These flexibly allow both: a) temporal or speed perturbed normal speech spectra to be modified and closer to those of an impaired speaker when parallel speech data is available; and b) for non-parallel data, the SVD decomposed normal speech spectral basis features to be transformed into those of a target elderly speaker before being re-composed with the temporal bases to produce the augmented data for state-of-the-art TDNN and Conformer ASR system training. Experiments are conducted on four tasks: the English UASpeech and TORGO dysarthric speech corpora; the English DementiaBank Pitt and Cantonese JCCOCC MoCA elderly speech datasets. The proposed GAN based data augmentation approaches consistently outperform the baseline speed perturbation method by up to 0.91% and 3.0% absolute (9.61% and 6.4% relative) WER reduction on the TORGO and DementiaBank data respectively. Consistent performance improvements are retained after applying LHUC based speaker adaptation.

Data augmentation using generative adversarial networks for robust speech recognition.

Generative Adversarial Networks Based Data Augmentation for Noise Robust Speech Recognition

Data Augmentation using Conditional Generative Adversarial Networks for Robust Speech Recognition

Personalized Adversarial Data Augmentation for Dysarthric and Elderly Speech Recognition

Adversarial Training with Gated Convolutional Neural Networks for Robust Speech Recognition

GANs for Children: A Generative Data Augmentation Strategy for Children Speech Recognition

Data Augmentation for Signal Modulation Classification using Generative Adverse Network

Improving Speech Emotion Recognition With Adversarial Data Augmentation Network

A robust speaker recognition based on Data augmentation

Improving Speech Recognition Using GAN-Based Speech Synthesis and Contrastive Unspoken Text Selection

Study of GANs for Noisy Speech Simulation from Clean Speech

Data Augmentation of Room Classifiers using Generative Adversarial Networks

Adversarial Data Augmentation Using VAE-GAN for Disordered Speech Recognition

Exploring Speech Enhancement with Generative Adversarial Networks for Robust Speech Recognition

Data Augmentation Using Deep Generative Models for Embedding Based Speaker Recognition

Channel-Aware Domain-Adaptive Generative Adversarial Network for Robust Speech Recognition

Enhancing Pre-trained ASR System Fine-tuning for Dysarthric Speech Recognition using Adversarial Data Augmentation

Double Adversarial Network Based Monaural Speech Enhancement for Robust Speech Recognition.

DLT-GAN: Dual-Layer Transfer Generative Adversarial Network-Based Time Series Data Augmentation Method

Noise Modeling to Build Training Sets for Robust Speech Enhancement

Generative Adversarial Networks based X-vector Augmentation for Robust Probabilistic Linear Discriminant Analysis in Speaker Verification