Abstract:Due to the dynamic nature of human language, automatic speech recognition (ASR) systems need to continuously acquire new vocabulary. Out-Of-Vocabulary (OOV) words, such as trending words and new named entities, pose problems to modern ASR systems that require long training times to adapt their large numbers of parameters. Different from most previous research focusing on language model post-processing, we tackle this problem on an earlier processing level and eliminate the bias in acoustic modeling to recognize OOV words acoustically. We propose to generate OOV words using text-to-speech systems and to rescale losses to encourage neural networks to pay more attention to OOV words. Specifically, we enlarge the classification loss used for training neural networks' parameters of utterances containing OOV words (sentence-level), or rescale the gradient used for back-propagation for OOV words (word-level), when fine-tuning a previously trained model on synthetic audio. To overcome catastrophic forgetting, we also explore the combination of loss rescaling and model regularization, i.e. L2 regularization and elastic weight consolidation (EWC). Compared with previous methods that just fine-tune synthetic audio with EWC, the experimental results on the LibriSpeech benchmark reveal that our proposed loss rescaling approach can achieve significant improvement on the recall rate with only a slight decrease on word error rate. Moreover, word-level rescaling is more stable than utterance-level rescaling and leads to higher recall rates and precision on OOV word recognition. Furthermore, our proposed combined loss rescaling and weight consolidation methods can support continual learning of an ASR system.

Wiki-En-ASR-Adapt: Large-scale synthetic dataset for English ASR Customization

Advancing African-Accented Speech Recognition: Epistemic Uncertainty-Driven Data Selection for Generalizable ASR Models

SpellMapper: A non-autoregressive neural spellchecker for ASR customization with candidate retrieval based on n-gram mappings

An Effective Context-Balanced Adaptation Approach for Long-Tailed Speech Recognition

Open-vocabulary Keyword-spotting with Adaptive Instance Normalization

SeACo-Paraformer: A Non-Autoregressive ASR System with Flexible and Effective Hotword Customization Ability

ASR-EC Benchmark: Evaluating Large Language Models on Chinese ASR Error Correction

Synthetic Cross-accent Data Augmentation for Automatic Speech Recognition

Enabling ASR for Low-Resource Languages: A Comprehensive Dataset Creation Approach

Contextual Biasing to Improve Domain-specific Custom Vocabulary Audio Transcription without Explicit Fine-Tuning of Whisper Model

LM-assisted keyword biasing with Aho-Corasick algorithm for Transducer-based ASR

A Light-weight contextual spelling correction model for customizing transducer-based speech recognition systems

Retrieve and Copy: Scaling ASR Personalization to Large Catalogs

Emphasizing Unseen Words: New Vocabulary Acquisition for End-to-End Speech Recognition

Auto-AVSR: Audio-Visual Speech Recognition with Automatic Labels

New Datasets and Controllable Iterative Data Augmentation Method for Code-switching ASR Error Correction.

SynthesizRR: Generating Diverse Datasets with Retrieval Augmentation

Improving Contextual Spelling Correction by External Acoustics Attention and Semantic Aware Data Augmentation

Universal Cross-Lingual Data Generation for Low Resource ASR

BordIRlines: A Dataset for Evaluating Cross-lingual Retrieval-Augmented Generation

Anatomy of Industrial Scale Multilingual ASR