Abstract:Conventional automatic speech recognition (ASR) and emerging end-to-end (E2E) speech recognition have achieved promising results after being provided with sufficient resources. However, for low-resource language, the current ASR is still challenging. The Lhasa dialect is the most widespread Tibetan dialect and has a wealth of speakers and transcriptions. Hence, it is meaningful to apply the ASR technique to the Lhasa dialect for historical heritage protection and cultural exchange. Previous work on Tibetan speech recognition focused on selecting phone-level acoustic modeling units and incorporating tonal information but underestimated the influence of limited data. The purpose of this paper is to improve the speech recognition performance of the low-resource Lhasa dialect by adopting multilingual speech recognition technology on the E2E structure based on the transfer learning framework. Using transfer learning, we first establish a monolingual E2E ASR system for the Lhasa dialect with different source languages to initialize the ASR model to compare the positive effects of source languages on the Tibetan ASR model. We further propose a multilingual E2E ASR system by utilizing initialization strategies with different source languages and multilevel units, which is proposed for the first time. Our experiments show that the performance of the proposed method-based ASR system exceeds that of the E2E baseline ASR system. Our proposed method effectively models the low-resource Lhasa dialect and achieves a relative 14.2% performance improvement in character error rate (CER) compared to DNN-HMM systems. Moreover, from the best monolingual E2E model to the best multilingual E2E model of the Lhasa dialect, the system’s performance increased by 8.4% in CER.

Tibetan-Mandarin Bilingual Speech Recognition Based on End-to-end Framework

End-to-End-Based Tibetan Multitask Speech Recognition.

A hybrid CTC+Attention model based on end-to-end framework for multilingual speech recognition

Towards Language-Universal Mandarin-English Speech Recognition

Cross-Language Transfer Learning-based Lhasa-Tibetan Speech Recognition

Improving Low-Resource Tibetan End-to-end ASR by Multilingual and Multilevel Unit Modeling

End-to-end Code-switched TTS with Mix of Monolingual Recordings.

End-to-End Mandarin Speech Recognition Combining CNN and BLSTM

Tibetan Multi-dialect Speech and Dialect Identity Recognition

End-to-End Speech Synthesis for Tibetan Multidialect

Investigation on Acoustic Modeling with Different Phoneme Set for Continuous Lhasa Tibetan Recognition Based on Dnn Method

Effective Fine-tuning Method for Tibetan Low-resource Dialect Speech Recognition

Speech Recognition Based on Deep Neural Networks on Tibetan Corpus

End-to-End Tibetan-Chinese Speech Translation Based on Multi-task and Multi-level Pre-training

A comparable study of modeling units for end-to-end Mandarin speech recognition

Investigation of Modeling Units for Mandarin Speech Recognition Using Dfsmn-ctc-smbr

Multi-Level Modeling Units for End-to-End Mandarin Speech Recognition

Lhasa Dialect Recognition of Different Phonemes Based on TDNN Method.

Syllable-Based Acoustic Modeling with CTC for Multi-Scenarios Mandarin Speech Recognition

Investigation of Transformer Based Spelling Correction Model for CTC-based End-to-End Mandarin Speech Recognition

End-to-end keywords spotting based on connectionist temporal classification for Mandarin