Abstract:Despite the rapid progress of end-to-end (E2E) automatic speech recognition (ASR), it has been shown that incorporating external language models (LMs) into the decoding can further improve the recognition performance of E2E ASR systems. To align with the modeling units adopted in E2E ASR systems, subword-level (e.g., characters, BPE) LMs are usually used to cooperate with current E2E ASR systems. However, the use of subword-level LMs will ignore the word-level information, which may limit the strength of the external LMs in E2E ASR. Although several methods have been proposed to incorporate word-level external LMs in E2E ASR, these methods are mainly designed for languages with clear word boundaries such as English and cannot be directly applied to languages like Mandarin, in which each character sequence can have multiple corresponding word sequences. To this end, we propose a novel decoding algorithm where a word-level lattice is constructed on-the-fly to consider all possible word sequences for each partial hypothesis. Then, the LM score of the hypothesis is obtained by intersecting the generated lattice with an external word N-gram LM. The proposed method is examined on both Attention-based Encoder-Decoder (AED) and Neural Transducer (NT) frameworks. Experiments suggest that our method consistently outperforms subword-level LMs, including N-gram LM and neural network LM. We achieve state-of-the-art results on both Aishell-1 (CER 4.18%) and Aishell-2 (CER 5.06%) datasets and reduce CER by 14.8% relatively on a 21K-hour Mandarin dataset.

Candidate Expansion Algorithm Based on Weighted Syllable Confusion Matrix for Mandarin LVCSR

Visual Information Assisted Mandarin Large Vocabulary Continuous Speech Recognition

Integrating Multi-Level Linguistic Knowledge with a Unified Framework for Mandarin Speech Recognition

Improving Mandarin End-to-End Speech Recognition with Word N-gram Language Model

Investigation of Transformer Based Spelling Correction Model for CTC-based End-to-End Mandarin Speech Recognition

Investigation of Modeling Units for Mandarin Speech Recognition Using Dfsmn-ctc-smbr

Generation of syllable-lattice based on active state in Mandarin continuous speech recognition

Automatic Spelling Correction with Transformer for CTC-based End-to-End Speech Recognition

Improving the Syllable-Synchronous Network Search Algorithm for Word Decoding in Continuous Chinese Speech Recognition

Mandarin Isolated Words Recognition Method Based on Pitch Contour

Language-specific Acoustic Boundary Learning for Mandarin-English Code-switching Speech Recognition

Effective Acoustic Modeling for Pronunciation Quality Scoring of Strongly Accented Mandarin Speech

A Fast Fuzzy Keyword Spotting Algorithm Based on Syllable Confusion Network.

Linguistic Search Optimization for Deep Learning Based LVCSR

A C/v Segmentation Method for Mandarin Speech Based on Multiscale Fractal Dimension.

CTC Regularized Model Adaptation for Improving LSTM RNN Based Multi-Accent Mandarin Speech Recognition

A Hierarchical Viterbi Algorithm For Mandarin Hybrid Speech Synthesis System

Feature Selection In Mandarin Large Vocabulary Continuous Speech Recognition

Fast Fuzzy Keyword Spotting Using Syllable Confusion Network Indexing

Multi-Level Modeling Units for End-to-End Mandarin Speech Recognition

Mfcc And Svm Based Recognition Of Chinese Vowels