Abstract:Despite the rapid progress of end-to-end (E2E) automatic speech recognition (ASR), it has been shown that incorporating external language models (LMs) into the decoding can further improve the recognition performance of E2E ASR systems. To align with the modeling units adopted in E2E ASR systems, subword-level (e.g., characters, BPE) LMs are usually used to cooperate with current E2E ASR systems. However, the use of subword-level LMs will ignore the word-level information, which may limit the strength of the external LMs in E2E ASR. Although several methods have been proposed to incorporate word-level external LMs in E2E ASR, these methods are mainly designed for languages with clear word boundaries such as English and cannot be directly applied to languages like Mandarin, in which each character sequence can have multiple corresponding word sequences. To this end, we propose a novel decoding algorithm where a word-level lattice is constructed on-the-fly to consider all possible word sequences for each partial hypothesis. Then, the LM score of the hypothesis is obtained by intersecting the generated lattice with an external word N-gram LM. The proposed method is examined on both Attention-based Encoder-Decoder (AED) and Neural Transducer (NT) frameworks. Experiments suggest that our method consistently outperforms subword-level LMs, including N-gram LM and neural network LM. We achieve state-of-the-art results on both Aishell-1 (CER 4.18%) and Aishell-2 (CER 5.06%) datasets and reduce CER by 14.8% relatively on a 21K-hour Mandarin dataset.

A Mandarin Speech Recognition Result Evaluation Algorithm on Mixed Words

Experimental Investigation into Alignment-based Acoustic Confidence Measures in Keyword Verification for Mandarin Speech

Algorithm for Mandarin Continuous Speech Recognition Based on Context-Dependent Unit Between Syllables

Mandarin Isolated Words Recognition Method Based on Pitch Contour

A Hierarchical Viterbi Algorithm For Mandarin Hybrid Speech Synthesis System

Experimental Investigation of Mandarin Pronunciation Quality Assessment System

A Fast Error-tolerant Algorithm in Decoding Module of Speech Recognition

Automatic Mandarin Pronunciation Scoring for Native Learners with Dialect Accent.

RESEARCH AND REALISATION OF TANDEM IN MANDARIN SPEECH ERROR DETECTION SYSTEM

High Performance Digit Mandarin Speech Recognition

Research on Inter-Syllable Context-Dependent Acoustic Unit for Mandarin Continuous Speech Recognition.

Improved syllable loop based method for confidence measure

Mandarin keyword spotting using syllable based confidence features and SVM

A Comparative Study on Confidence Measure in Mandarin Command Word Recognition

Effective Acoustic Modeling for Pronunciation Quality Scoring of Strongly Accented Mandarin Speech

Improving Mandarin End-to-End Speech Recognition with Word N-gram Language Model

Performance Comparison of Several Speech Recognition Methods

An Innovative Prosody Modeling Method for Chinese Speech Recognition

Mandarin-English bilingual phone modeling and combining MPE based Discriminative training for cross-language speech recognition

Grammar-Supervised End-to-End Speech Recognition with Part-of-Speech Tagging and Dependency Parsing

Feature Selection In Mandarin Large Vocabulary Continuous Speech Recognition