Abstract:Despite the rapid progress of end-to-end (E2E) automatic speech recognition (ASR), it has been shown that incorporating external language models (LMs) into the decoding can further improve the recognition performance of E2E ASR systems. To align with the modeling units adopted in E2E ASR systems, subword-level (e.g., characters, BPE) LMs are usually used to cooperate with current E2E ASR systems. However, the use of subword-level LMs will ignore the word-level information, which may limit the strength of the external LMs in E2E ASR. Although several methods have been proposed to incorporate word-level external LMs in E2E ASR, these methods are mainly designed for languages with clear word boundaries such as English and cannot be directly applied to languages like Mandarin, in which each character sequence can have multiple corresponding word sequences. To this end, we propose a novel decoding algorithm where a word-level lattice is constructed on-the-fly to consider all possible word sequences for each partial hypothesis. Then, the LM score of the hypothesis is obtained by intersecting the generated lattice with an external word N-gram LM. The proposed method is examined on both Attention-based Encoder-Decoder (AED) and Neural Transducer (NT) frameworks. Experiments suggest that our method consistently outperforms subword-level LMs, including N-gram LM and neural network LM. We achieve state-of-the-art results on both Aishell-1 (CER 4.18%) and Aishell-2 (CER 5.06%) datasets and reduce CER by 14.8% relatively on a 21K-hour Mandarin dataset.

Neural Network Language Model for Chinese Pinyin Input Method Engine.

Open Vocabulary Learning for Neural Chinese Pinyin IME

Chinese Pinyin Aided IME, Input What You Have Not Keystroked Yet

Moon Ime: Neural-Based Chinese Pinyin Aided Input Method With Customizable Association

Exploiting Word Positional Information in Ngram Model for Chinese Text Input Method

Neural or Statistical: An Empirical Study on Language Models for Chinese Input Recommendation on Mobile

Towards Energy-Preserving Natural Language Understanding with Spiking Neural Networks

A Joint Graph Model For Pinyin-To-Chinese Conversion With Typo Correction

Recurrent Neural Network Based Language Model Adaptation for Accent Mandarin Speech.

Exploiting Pinyin Constraints in Pinyin-to-Character Conversion Task: a Class-Based Maximum Entropy Markov Model Approach

Recurrent Neural Network Language Model with Part-of-speech for Mandarin Speech Recognition.

PERT: A New Solution to Pinyin to Character Conversion Task

A High Accuracy Approach for Word-Phoneme Translation Using Neural Networks

A Smart Sliding Chinese Pinyin Input Method Editor for Touchscreen Devices

An Online Incremental Language Model Adaptation Method

Integrating Pinyin to Improve Spelling Errors Detection for Chinese Language

Improving Accented Mandarin Speech Recognition by Using Recurrent Neural Network Based Language Model Adaptation

Empirical Study on Character Level Neural Network Classifier for Chinese Text.

Improving Mandarin End-to-End Speech Recognition with Word N-gram Language Model

Empirically Combining Unnormalized NNLM and Back-off N -Gram for Fast N -Best Rescoring in Speech Recognition

An Investigation On Statistical Machine Translation With Neural Language Models