Abstract:The choice of acoustic modeling units is critical to acoustic modeling in large vocabulary continuous speech recognition (LVCSR) tasks. The recent connectionist temporal classification (CTC) based acoustic models have more options for the choice of modeling units. In this work, we propose a DFSMN-CTC-sMBR acoustic model and investigate various modeling units for Mandarin speech recognition. In addition to the commonly used context-independent Initial/Finals (CI-IF), context-dependent Initial/Finals (CD-IF) and Syllable, we also propose a hybrid Character-Syllable modeling units by mixing high frequency Chinese characters and syllables. Experimental results show that DFSMN-CTC-sMBR models with all these types of modeling units can significantly outperform the well-trained conventional hybrid models. Moreover, we find that the proposed hybrid Character-Syllable modeling units is the best choice for CTC based acoustic modeling for Mandarin speech recognition in our work since it can dramatically reduce substitution errors in recognition results. In a 20,000 hours Mandarin speech recognition task, the DFSMN-CTC-sMBR system with hybrid Character-Syllable achieves a character error rate (CER) of 7.45% while performance of the well-trained DFSMN-CE-sMBR system is 9.49%.

Asymmetric Acoustic Model for Accented Speech Recognition

Acoustic Model Reconstruction for Multi-Accent Chinese Speech Recognition

Accent Recognition with Hybrid Phonetic Features

Reliable accent specific unit generation with dynamic Gaussian mixture selection for multi-accent speech recognition

Reliable Accent-Specific Unit Generation With Discriminative Dynamic Gaussian Mixture Selection for Multi-Accent Chinese Speech Recognition

Joint Modeling of Accents and Acoustics for Multi-Accent Speech Recognition

Effective Acoustic Modeling for Pronunciation Quality Scoring of Strongly Accented Mandarin Speech

Detection-based accented speech recognition using articulatory features.

Improving Accented Mandarin Speech Recognition by Using Recurrent Neural Network Based Language Model Adaptation

A Highly Adaptive Acoustic Model for Accurate Multi-Dialect Speech Recognition

Recurrent Neural Network Based Language Model Adaptation for Accent Mandarin Speech.

Pronunciation Variation Modeling For Mandarin With Accent

Synthetic Cross-accent Data Augmentation for Automatic Speech Recognition

Investigation of Modeling Units for Mandarin Speech Recognition Using Dfsmn-ctc-smbr

Improved Accent Classification Combining Phonetic Vowels with Acoustic Features

Acoustic Model Fusion for End-to-end Speech Recognition

CTC Regularized Model Adaptation for Improving LSTM RNN Based Multi-Accent Mandarin Speech Recognition

Merging of British and American accents for embedded pronunciation scoring applications

Discriminative Dynamic Gaussian Mixture Selection with Enhanced Robustness and Performance for Multi-Accent Speech Recognition

Accent conversion using discrete units with parallel data synthesized from controllable accented TTS

Bridging the Gap Between Monaural Speech Enhancement and Recognition With Distortion-Independent Acoustic Modeling