Abstract:End-to-end (E2E) automatic speech recognition (ASR) models have become standard practice for various commercial applications. However, in real-world scenarios, the long-tailed nature of word distribution often leads E2E ASR models to perform well on common words but fall short in recognizing uncommon ones. Recently, the notion of a contextual adapter (CA) was proposed to infuse external knowledge represented by a context word list into E2E ASR models. Although CA can improve recognition performance on rare words, two crucial data imbalance problems remain. First, when using low-frequency words as context words during training, since these words rarely occur in the utterance, CA becomes prone to overfit on attending to the <no-context> token due to higher-frequency words not being present in the context list. Second, the long-tailed distribution within the context list itself still causes the model to perform poorly on low-frequency context words. In light of this, we explore in-depth the impact of altering the context list to have words with different frequency distributions on model performance, and meanwhile extend CA with a simple yet effective context-balanced learning objective. A series of experiments conducted on the AISHELL-1 benchmark dataset suggests that using all vocabulary words from the training corpus as the context list and pairing them with our balanced objective yields the best performance, demonstrating a significant reduction in character error rate (CER) by up to 1.21% and a more pronounced 9.44% reduction in the error rate of zero-shot words.

End-to-end Contextual Speech Recognition Using Class Language Models and a Token Passing Decoder.

End-to-End Speech Recognition Contextualization with Large Language Models

Joint Grapheme and Phoneme Embeddings for Contextual End-to-End ASR

End-to-End Contextual Speech Recognition with Word-Piece-Level Token Selection.

Learning Contextual Language Embeddings for Monaural Multi-Talker Speech Recognition.

CTC-Assisted LLM-Based Contextual ASR

Multistate Encoding with End-To-End Speech RNN Transducer Network

Modular End-to-End Automatic Speech Recognition Framework for Acoustic-to-Word Model

Two Stage Contextual Word Filtering for Context bias in Unified Streaming and Non-streaming Transducer

Contextualization of ASR with LLM using phonetic retrieval-based augmentation

Reducing Multilingual Context Confusion for End-to-end Code-switching Automatic Speech Recognition

An End-to-End Speech Recognition System Based on Shared Encoder

Using Large Language Model for End-to-End Chinese ASR and NER

Optimizing Alignment of Speech and Language Latent Spaces for End-To-End Speech Recognition and Understanding.

End-to-End Dereverberation, Beamforming, and Speech Recognition in a Cocktail Party.

Phoneme-Based Contextualization for Cross-Lingual Speech Recognition in End-to-End Models

Reducing language context confusion for end-to-end code-switching automatic speech recognition

Multi-encoder multi-resolution framework for end-to-end speech recognition

Dialog-context aware end-to-end speech recognition

High-Accuracy and Low-Latency Speech Recognition with Two-Head Contextual Layer Trajectory LSTM Model

An Effective Context-Balanced Adaptation Approach for Long-Tailed Speech Recognition