Abstract:In AI, generalization refers to a model's ability to perform well on out-of-distribution data related to the given task, beyond the data it was trained on. For an AI agent to excel, it must also possess the continual learning capability, whereby an agent incrementally learns to perform a sequence of tasks without forgetting the previously acquired knowledge to solve the old tasks. Intuitively, generalization within a task allows the model to learn underlying features that can readily be applied to novel tasks, facilitating quicker learning and enhanced performance in subsequent tasks within a continual learning framework. Conversely, continual learning methods often include mechanisms to mitigate catastrophic forgetting, ensuring that knowledge from earlier tasks is retained. This preservation of knowledge over tasks plays a role in enhancing generalization for the ongoing task at hand. Despite the intuitive appeal of the interplay of both abilities, existing literature on continual learning and generalization has proceeded separately. In the preliminary effort to promote studies that bridge both fields, we first present empirical evidence showing that each of these fields has a mutually positive effect on the other. Next, building upon this finding, we introduce a simple and effective technique known as Shape-Texture Consistency Regularization (STCR), which caters to continual learning. STCR learns both shape and texture representations for each task, consequently enhancing generalization and thereby mitigating forgetting. Remarkably, extensive experiments validate that our STCR, can be seamlessly integrated with existing continual learning methods, where its performance surpasses these continual learning methods in isolation or when combined with established generalization techniques by a large margin. Our data and source code will be made publicly available upon publication.

Less Forgetting for Better Generalization: Exploring Continual-learning Fine-tuning Methods for Speech Self-supervised Representations

Continual Learning for End-to-End ASR by Averaging Domain Experts

Less-forgetting Multi-lingual Fine-tuning

Progressive Multi-scale Self-supervised Learning for Speech Recognition

Recall and Learn: Fine-tuning Deep Pretrained Language Models with Less Forgetting

INCREMENTAL LEARNING FOR END-TO-END AUTOMATIC SPEECH RECOGNITION

A dual task learning approach to fine-tune a multilingual semantic speech encoder for Spoken Language Understanding

Improving Children's Speech Recognition by Fine-tuning Self-supervised Adult Speech Representations

Continual Speaker Adaptation for Text-to-Speech Synthesis

Rehearsal-Free Online Continual Learning for Automatic Speech Recognition

Sequential Editing for Lifelong Training of Speech Recognition Models

Replay to Remember: Continual Layer-Specific Fine-tuning for German Speech Recognition

Continual Learning for On-Device Speech Recognition using Disentangled Conformers

Automatic Data Augmentation for Domain Adapted Fine-Tuning of Self-Supervised Speech Representations

Improving Automatic Speech Recognition Performance for Low-Resource Languages With Self-Supervised Models

Federated Self-Learning with Weak Supervision for Speech Recognition

Controlling Forgetting with Test-Time Data in Continual Learning

Unveiling the Tapestry: the Interplay of Generalization and Forgetting in Continual Learning

Continual Pre-Training Mitigates Forgetting in Language and Vision

Continual Learning: Less Forgetting, More OOD Generalization via Adaptive Contrastive Replay

An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning