Abstract:Automatic speech recognition has made huge progress recently. However, the current modeling strategy still suffers a large performance degradation when facing the low-resource languages with limited training data. In this paper, we propose a series of methods to optimize the data usage for low-resource speech recognition. Multilingual speech recognition helps a lot in low-resource scenarios. The correlation and similarity between languages are further exploited for multilingual pretraining in our work. We utilize the posterior of the target language extracted from a language classifier to perform data weighing on training samples, which assists the model in being more biased towards the target language during pretraining. Furthermore, dynamic curriculum learning for data allocation and length perturbation for data augmentation are also designed. All these three methods form the new strategy on optimized data usage for low-resource languages. We evaluate the proposed method using rich resource languages for pretraining (PT) and finetuning (FT) the model on the target language with limited data. Experimental results show that the proposed data usage method obtains a 15 to 25% relative word error rate reduction for different target languages compared with the commonly adopted multilingual PT+FT method on CommonVoice dataset. The same improvement and conclusion are also observed on Babel dataset with conversational telephone speech, and $\sim$ 40% relative character error rate reduction can be obtained for the target low-resource language.

Reducing pronunciation lexicon confusion and using more data without phonetic transcription for pronunciation modeling

A High Accuracy Approach for Word-Phoneme Translation Using Neural Networks

A METHOD TO CONSTRUCT AN ADAPTIVE MONGOLIAN SPEECH ACOUSTIC MODEL

MANDARIN PRONUNCIATION VARIATION MODELING 1

A Transcription Prompt-based Efficient Audio Large Language Model for Robust Speech Recognition

Mandarin Pronunciation Modeling Based on CASS Corpus.

Acquiring Pronunciation Knowledge from Transcribed Speech Audio via Multi-task Learning

Modeling Pronunciation Variation Using Context-Dependent Weighting and B/s Refined Acoustic Modeling.

End-to-End Mispronunciation Detection and Diagnosis Using Transfer Learning

Improving Non-native Word-level Pronunciation Scoring with Phone-level Mixup Data Augmentation and Multi-source Information

An Approach to Mispronunciation Detection and Diagnosis with Acoustic, Phonetic and Linguistic (APL) Embeddings

Improving Mispronunciation Detection with Wav2vec2-based Momentum Pseudo-Labeling for Accentedness and Intelligibility Assessment

No Need for a Lexicon? Evaluating the Value of the Pronunciation Lexica in End-to-End Models

Optimizing Data Usage for Low-Resource Speech Recognition

Text-Aware End-to-end Mispronunciation Detection and Diagnosis

A two-stage mispronunciation detection approach for computer-assisted pronunciation training

Improve low-resource non-native mispronunciation detection with native speech by articulatory-based tandem feature

Applying Multitask Learning To Acoustic-Phonemic Model For Mispronunciation Detection And Diagnosis In L2 English Speech

Modeling Syllable-Based Pronunciation Variation for Accented Mandarin Speech Recognition

Using Phoneme Representations to Build Predictive Models Robust to ASR Errors

Merging of British and American accents for embedded pronunciation scoring applications