Abstract:Despite the rapid progress of end-to-end (E2E) automatic speech recognition (ASR), it has been shown that incorporating external language models (LMs) into the decoding can further improve the recognition performance of E2E ASR systems. To align with the modeling units adopted in E2E ASR systems, subword-level (e.g., characters, BPE) LMs are usually used to cooperate with current E2E ASR systems. However, the use of subword-level LMs will ignore the word-level information, which may limit the strength of the external LMs in E2E ASR. Although several methods have been proposed to incorporate word-level external LMs in E2E ASR, these methods are mainly designed for languages with clear word boundaries such as English and cannot be directly applied to languages like Mandarin, in which each character sequence can have multiple corresponding word sequences. To this end, we propose a novel decoding algorithm where a word-level lattice is constructed on-the-fly to consider all possible word sequences for each partial hypothesis. Then, the LM score of the hypothesis is obtained by intersecting the generated lattice with an external word N-gram LM. The proposed method is examined on both Attention-based Encoder-Decoder (AED) and Neural Transducer (NT) frameworks. Experiments suggest that our method consistently outperforms subword-level LMs, including N-gram LM and neural network LM. We achieve state-of-the-art results on both Aishell-1 (CER 4.18%) and Aishell-2 (CER 5.06%) datasets and reduce CER by 14.8% relatively on a 21K-hour Mandarin dataset.

Application of On-Line Filler Modeling in Specific Domain Speech Recognition System

Modeling and Simulation of English Speech Rationality Optimization Recognition Based on Improved Particle Filter Algorithm

Using Word Confusion Networks for Slot Filling in Spoken Language Understanding.

A METHOD TO CONSTRUCT AN ADAPTIVE MONGOLIAN SPEECH ACOUSTIC MODEL

Modeling and Simulation of English Speech Optimization Teaching Recognition Based on Intelligent Edge Detection Algorithm

A New Hybrid Hmm/Ann Model For Speech Recognition

STUDY ON FRAMEWORK FOR CHINESE PRONUNCIATION VARIATION MODELING

Improved syllable loop based method for confidence measure

Single Figure Syllable Modeling Based on Neural Network for Continuous Speech Recognition

Modeling F0 Trajectories in Hierarchically Structured Deep Neural Networks.

An Efficient Approach of Language Model Applying in ASR Systems

Modeling Pronunciation Variation Using Context-Dependent Weighting and B/s Refined Acoustic Modeling.

Model Compensation Using Robust Features for Robust Speech Recognition

MANDARIN PRONUNCIATION VARIATION MODELING 1

Speech-based Slot Filling using Large Language Models

A Bi-model based RNN Semantic Frame Parsing Model for Intent Detection and Slot Filling

Key Technology Research for Speech Recognition

Deep neural networks for syllable based acoustic modeling in Chinese speech recognition.

Experiments and Analysis for Speaker Dependent Mandarin Syllable Recognition.

Improving Mandarin End-to-End Speech Recognition with Word N-gram Language Model

Personalized Speech Recognizer With Keyword-Based Personalized Lexicon And Language Model Using Word Vector Representations