Abstract:Humans learn all their life long. They accumulate knowledge from a sequence of learning experiences and remember the essential concepts without forgetting what they have learned previously. Artificial neural networks struggle to learn similarly. They often rely on data rigorously preprocessed to learn solutions to specific problems such as classification or regression. In particular, they forget their past learning experiences if trained on new ones. Therefore, artificial neural networks are often inept to deal with real-life settings such as an autonomous-robot that has to learn on-line to adapt to new situations and overcome new problems without forgetting its past learning-experiences. Continual learning (CL) is a branch of machine learning addressing this type of problem. Continual algorithms are designed to accumulate and improve knowledge in a curriculum of learning-experiences without forgetting. In this thesis, we propose to explore continual algorithms with replay processes. Replay processes gather together rehearsal methods and generative replay methods. Generative Replay consists of regenerating past learning experiences with a generative model to remember them. Rehearsal consists of saving a core-set of samples from past learning experiences to rehearse them later. The replay processes make possible a compromise between optimizing the current learning objective and the past ones enabling learning without forgetting in sequences of tasks settings. We show that they are very promising methods for continual learning. Notably, they enable the re-evaluation of past data with new knowledge and the confrontation of data from different learning-experiences. We demonstrate their ability to learn continually through unsupervised learning, supervised learning and reinforcement learning tasks.

Learning Fast, Learning Slow: A General Continual Learning Method based on Complementary Learning System

Progressive Learning without Forgetting

SYNERgy between SYNaptic consolidation and Experience Replay for general continual learning

Dual Memory Model for Experience-Once Task-Incremental Lifelong Learning.

Continual Learning with Strong Experience Replay

MGSER-SAM: Memory-Guided Soft Experience Replay with Sharpness-Aware Optimization for Enhanced Continual Learning

Saliency-Guided Hidden Associative Replay for Continual Learning

SynapNet: A Complementary Learning System Inspired Algorithm With Real-Time Application in Multimodal Perception

Learning Bayesian Sparse Networks with Full Experience Replay for Continual Learning

Slowing Down Forgetting in Continual Learning

Adaptive Memory Replay for Continual Learning

ER-FSL: Experience Replay with Feature Subspace Learning for Online Continual Learning

Continual Learning: Tackling Catastrophic Forgetting in Deep Neural Networks with Replay Processes

A Unified and General Framework for Continual Learning

Sparse Coding in a Dual Memory System for Lifelong Learning

Memory Enhanced Replay for Continual Learning

Brain-Inspired Continual Learning-Robust Feature Distillation and Re-Consolidation for Class Incremental Learning

Learning to Continually Learn Rapidly from Few and Noisy Data

DualNet: Continual Learning, Fast and Slow

Online Continual Learning with Declarative Memory

Interactive Continual Learning: Fast and Slow Thinking