Abstract:Due to the dynamic nature of human language, automatic speech recognition (ASR) systems need to continuously acquire new vocabulary. Out-Of-Vocabulary (OOV) words, such as trending words and new named entities, pose problems to modern ASR systems that require long training times to adapt their large numbers of parameters. Different from most previous research focusing on language model post-processing, we tackle this problem on an earlier processing level and eliminate the bias in acoustic modeling to recognize OOV words acoustically. We propose to generate OOV words using text-to-speech systems and to rescale losses to encourage neural networks to pay more attention to OOV words. Specifically, we enlarge the classification loss used for training neural networks' parameters of utterances containing OOV words (sentence-level), or rescale the gradient used for back-propagation for OOV words (word-level), when fine-tuning a previously trained model on synthetic audio. To overcome catastrophic forgetting, we also explore the combination of loss rescaling and model regularization, i.e. L2 regularization and elastic weight consolidation (EWC). Compared with previous methods that just fine-tune synthetic audio with EWC, the experimental results on the LibriSpeech benchmark reveal that our proposed loss rescaling approach can achieve significant improvement on the recall rate with only a slight decrease on word error rate. Moreover, word-level rescaling is more stable than utterance-level rescaling and leads to higher recall rates and precision on OOV word recognition. Furthermore, our proposed combined loss rescaling and weight consolidation methods can support continual learning of an ASR system.

Augmenting Statistical Machine Translation with Subword Translation of Out-of-Vocabulary Words

Data Augmentation and Terminology Integration for Domain-Specific Sinhala-English-Tamil Statistical Machine Translation

Finding Better Subword Segmentation for Neural Machine Translation

Enhancing Out-of-Vocabulary Performance of Indian TTS Systems for Practical Applications through Low-Effort Data Strategies

How Effective is Byte Pair Encoding for Out-Of-Vocabulary Words in Neural Machine Translation?

Designing effective web mining-based techniques for OOV translation

English-Chinese Bi-Directional OOV Translation based on Web Mining and Supervised Learning.

AdvAug: Robust Adversarial Augmentation for Neural Machine Translation

Improving Pre-Trained Multilingual Model with Vocabulary Expansion

OOVs in the Spotlight: How to Inflect them?

Research on Out-of-vocabulary Words' Recognition in Uyghur-Chinese Machine Translation

Memory-augmented Neural Machine Translation.

Vocabulary Learning Via Optimal Transport for Neural Machine Translation

Data Augmentation Techniques for Machine Translation of Code-Switched Texts: A Comparative Study

Towards Continual Learning for Multilingual Machine Translation via Vocabulary Substitution

Emphasizing Unseen Words: New Vocabulary Acquisition for End-to-End Speech Recognition

Improving Data Augmentation for Low-Resource NMT Guided by POS-Tagging and Paraphrase Embedding

Improving Statistical Machine Translation for a Resource-Poor Language Using Related Resource-Rich Languages

Improving Many-to-Many Neural Machine Translation Via Selective and Aligned Online Data Augmentation

A Morphologically-Aware Dictionary-based Data Augmentation Technique for Machine Translation of Under-Represented Languages

A Systematic Analysis of Vocabulary and BPE Settings for Optimal Fine-tuning of NMT: A Case Study of In-domain Translation