Abstract:Automatic speech recognition has made huge progress recently. However, the current modeling strategy still suffers a large performance degradation when facing the low-resource languages with limited training data. In this paper, we propose a series of methods to optimize the data usage for low-resource speech recognition. Multilingual speech recognition helps a lot in low-resource scenarios. The correlation and similarity between languages are further exploited for multilingual pretraining in our work. We utilize the posterior of the target language extracted from a language classifier to perform data weighing on training samples, which assists the model in being more biased towards the target language during pretraining. Furthermore, dynamic curriculum learning for data allocation and length perturbation for data augmentation are also designed. All these three methods form the new strategy on optimized data usage for low-resource languages. We evaluate the proposed method using rich resource languages for pretraining (PT) and finetuning (FT) the model on the target language with limited data. Experimental results show that the proposed data usage method obtains a 15 to 25% relative word error rate reduction for different target languages compared with the commonly adopted multilingual PT+FT method on CommonVoice dataset. The same improvement and conclusion are also observed on Babel dataset with conversational telephone speech, and $\sim$ 40% relative character error rate reduction can be obtained for the target low-resource language.

MLP-HMM Two-Stage Unsupervised Training for Low-Resource Languages on Conversational Telephone Speech Recognition

Articulatory Feature Based Multilingual MLPs for Low-Resource Speech Recognition.

Cross-Lingual and Ensemble MLPs Strategies for Low-Resource Speech Recognition

Strategies for using MLP based features with limited target-language training data.

Multi-Stream Posterior Features and Combining Subspace Gmms for Low Resource Lvcsr

A Parameter Transfer Method for HMM-DNN Heterogeneous Model with the Scarce Mongolian Data Set

Language-Universal Phonetic Representation in Multilingual Speech Pretraining for Low-Resource Speech Recognition

LEARNING CROSS-LINGUAL INFORMATION WITH MULTILINGUAL BLSTM FOR SPEECH SYNTHESIS OF LOW-RESOURCE LANGUAGES

Improve low-resource non-native mispronunciation detection with native speech by articulatory-based tandem feature

Low Resource Malay Dialect Automatic Speech Recognition Modeling Using Transfer Learning from a Standard Malay Model

A Semi-Supervised Complementary Joint Training Approach for Low-Resource Speech Recognition

Mandarin-English bilingual phone modeling and combining MPE based Discriminative training for cross-language speech recognition

Mismatched Training Data Enhancement for Automatic Recognition of Children's Speech Using DNN-HMM

Low-Resourced Speech Recognition for Iu Mien Language via Weakly-Supervised Phoneme-based Multilingual Pre-training

Optimizing Data Usage for Low-Resource Speech Recognition

Cross-Lingual Self-training to Learn Multilingual Representation for Low-Resource Speech Recognition

Phone modeling and combining discriminative training for Mandarin-English bilingual speech recognition

Fast and Efficient Multilingual Self-Supervised Pre-training for Low-Resource Speech Recognition

Convolutional maxout neural networks for low-resource speech recognition

Towards Unsupervised Speech Recognition Without Pronunciation Models

Learning Phone Recognition from Unpaired Audio and Phone Sequences Based on Generative Adversarial Network