Abstract: Language modeling is a crucial component in a wide range of applications including speech recognition. Language models LMs are usually constructed by splitting a sentence into words and computing the probability of a word based on its word history. This sentence probability calculation, making use of conditional probability distributions, assumes that there is little impact from approximations used in the LMs, including the word history representations and finite training data. This motivates examining models that make use of additional information from the sentence. In this paper, future word information, in addition to the history, is used to predict the probability of the current word. For recurrent neural network LMs RNNLMs, this information can be encapsulated in a bi-directional model. However, if used directly, this form of model is computationally expensive when trained on large quantities of data, and can be problematic when used with word lattices. This paper proposes a novel neural network language model structure, the succeeding-word RNNLM, su-RNNLM, to address these issues. Instead of using a recurrent unit to capture the complete future word contexts, a feedforward unit is used to model a fixed finite number of succeeding words. This is more efficient in training than bi-directional models and can be applied to lattice rescoring. The generated lattices can be used for downstream applications, such as confusion network decoding and keyword search. Experimental results on speech recognition and keyword spotting tasks illustrate the empirical usefulness of future word information, and the flexibility of the proposed model to represent this information.

A Rescoring Approach for Keyword Search Using Lattice Context Information.

Neural Lattice Search for Speech Recognition.

Context-aware RNNLM Rescoring for Conversational Speech Recognition

Exploiting Future Word Contexts in Neural Network Language Models for Speech Recognition.

Contextualizing ASR Lattice Rescoring with Hybrid Pointer Network Language Model

Lattice Rescoring Based on Large Ensemble of Complementary Neural Language Models

LT-LM: a novel non-autoregressive language model for single-shot lattice rescoring

Improved Contextual Recognition In Automatic Speech Recognition Systems By Semantic Lattice Rescoring

Future word contexts in neural network language models

A Novel Discriminative Score Calibration Method for Keyword Search

Class-Based Neural Network Language Model for Second-Pass Rescoring in ASR

Discriminative Combination of Multiple Local Classifiers in Lattice Rescoring

Cross-utterance ASR Rescoring with Graph-based Label Propagation

Empirically Combining Unnormalized NNLM and Back-off N -Gram for Fast N -Best Rescoring in Speech Recognition

Low-Latency Convolutional Recurrent Neural Network for Keyword Spotting

DCCRN-KWS: an audio bias based model for noise robust small-footprint keyword spotting

Word lattice reranking for Chinese word segmentation and part-of-speech tagging

Syllable Based Audio Search Using Confusion Network Arc As Indexing Unit

CRF-based confidence measures of recognized candidates for lattice-based audio indexing

Improved lattice-based spoken document retrieval by directly learning from the evaluation measures

End-to-end keyword search system based on attention mechanism and energy scorer for low resource languages