Abstract:Deep neural networks (DNNs) have gained great success in information fusion. However, recent studies report that DNNs are suffering from catastrophic forgetting, i.e., DNNs would forget the knowledge learned from previous tasks when training on the current task. To address this issue, continual learning is proposed to enhance long-term memories for DNNs. Since continual learning is very challenging, existing work simplifies the setting to simulate the sequentially online multi-task learning paradigm. Specifically, existing works commonly split one dataset into multiple disjoint categories to get multiple tasks that follow the same marginal distribution. We argue that this setting is too simple to approximate the real-world applications. In real-world scenarios, the data distributions of sequentially arrived tasks would change significantly from time to time, e.g., the lighting from day to night, and the background from spring to winter. Thus, the real-world applications are in a multi-view manner, yet existing methods ignore this challenge. To tame this, we propose Adaptive Online Continual Multi-view Learning (AOCML) to align distributions and reduce catastrophic forgetting as new tasks arrive. AOCML integrates experience replay and adversarial learning in an end-to-end framework, which stores samples in a memory buffer to replay previous tasks, while leveraging a discriminator to adaptively align distributions across views on-the-fly. In addition to common replay buffer, we also incorporate a soft label-based replay and an entropy-based reweighting to further prevent forgetting. Extensive experiments on four datasets verify that our method is able to significantly outperform previous CL methods and our method pushes CL one step forward towards practical multi-view orientation.

Mixture-of-Variational-Experts for Continual Learning

Progressive Learning without Forgetting

Theory on Mixture-of-Experts in Continual Learning

Online continual learning through unsupervised mutual information maximization

Learning an evolved mixture model for task-free continual learning

Boosting Continual Learning of Vision-Language Models via Mixture-of-Experts Adapters

Adaptive Progressive Continual Learning.

Low-Rank Mixture-of-Experts for Continual Medical Image Segmentation

Temporal-Difference Variational Continual Learning

Heterogeneous Continual Learning

Towards Continual Learning Desiderata via HSIC-Bottleneck Orthogonalization and Equiangular Embedding

Continual Learning by Modeling Intra-Class Variation

Uncertainty Estimation With Neural Processes for Meta-Continual Learning

Layerwise Optimization by Gradient Decomposition for Continual Learning

Representational Continuity for Unsupervised Continual Learning

Learning to Remember: A Synaptic Plasticity Driven Framework for Continual Learning

Automating Continual Learning

Learning to Modulate Random Weights: Neuromodulation-inspired Neural Networks For Efficient Continual Learning

Rethinking the Representational Continuity: Towards Unsupervised Continual Learning

Mixture of Experts Meets Prompt-Based Continual Learning

Adaptive online continual multi-view learning