Abstract:Due to the dynamic nature of human language, automatic speech recognition (ASR) systems need to continuously acquire new vocabulary. Out-Of-Vocabulary (OOV) words, such as trending words and new named entities, pose problems to modern ASR systems that require long training times to adapt their large numbers of parameters. Different from most previous research focusing on language model post-processing, we tackle this problem on an earlier processing level and eliminate the bias in acoustic modeling to recognize OOV words acoustically. We propose to generate OOV words using text-to-speech systems and to rescale losses to encourage neural networks to pay more attention to OOV words. Specifically, we enlarge the classification loss used for training neural networks' parameters of utterances containing OOV words (sentence-level), or rescale the gradient used for back-propagation for OOV words (word-level), when fine-tuning a previously trained model on synthetic audio. To overcome catastrophic forgetting, we also explore the combination of loss rescaling and model regularization, i.e. L2 regularization and elastic weight consolidation (EWC). Compared with previous methods that just fine-tune synthetic audio with EWC, the experimental results on the LibriSpeech benchmark reveal that our proposed loss rescaling approach can achieve significant improvement on the recall rate with only a slight decrease on word error rate. Moreover, word-level rescaling is more stable than utterance-level rescaling and leads to higher recall rates and precision on OOV word recognition. Furthermore, our proposed combined loss rescaling and weight consolidation methods can support continual learning of an ASR system.

Prediction of Neutrino Fluxes in the NOMAD Experiment

Emphasizing Unseen Words: New Vocabulary Acquisition for End-to-End Speech Recognition

Enhancing Out-of-Domain Detection for Speech Spoofing Countermeasure Via Supervised Contrastive Learning

Subword scheme for keyword search

Towards Vocabulary-Independent Speech Indexing for Large-Scale Repositories

Addressing the Out-of-vocabulary Problem for Large-Scale Chinese Spoken Term Detection

How Large a Vocabulary Does Text Classification Need? A Variational Approach to Vocabulary Selection

Enhancing Out-of-Vocabulary Performance of Indian TTS Systems for Practical Applications through Low-Effort Data Strategies

A Selection Method of Speech Vocabulary for Human-Robot Speech Interaction

Towards High Performance LVCSR in Speech-to-Speech Translation System on Smart Phones.

Vary: Scaling up the Vision Vocabulary for Large Vision-Language Models

A review of large-vocabulary continuous-speech

Combination Of Data Borrowing Strategies For Low-Resource Lvcsr

On Vocabulary Reliance in Scene Text Recognition

Open vocabulary keyword spotting through transfer learning from speech synthesis

A cost minimization approach to fix the vocabulary size in a tokenizer for an End-to-End ASR system

CNVSRC 2023: The First Chinese Continuous Visual Speech Recognition Challenge

Improving EEG based Continuous Speech Recognition

Toward On-Line Learning of Chinese Continuous Speech Recognition System.

Visual Information Assisted Mandarin Large Vocabulary Continuous Speech Recognition

A Systematic Analysis of Vocabulary and BPE Settings for Optimal Fine-tuning of NMT: A Case Study of In-domain Translation