Abstract:Humans learn all their life long. They accumulate knowledge from a sequence of learning experiences and remember the essential concepts without forgetting what they have learned previously. Artificial neural networks struggle to learn similarly. They often rely on data rigorously preprocessed to learn solutions to specific problems such as classification or regression. In particular, they forget their past learning experiences if trained on new ones. Therefore, artificial neural networks are often inept to deal with real-life settings such as an autonomous-robot that has to learn on-line to adapt to new situations and overcome new problems without forgetting its past learning-experiences. Continual learning (CL) is a branch of machine learning addressing this type of problem. Continual algorithms are designed to accumulate and improve knowledge in a curriculum of learning-experiences without forgetting. In this thesis, we propose to explore continual algorithms with replay processes. Replay processes gather together rehearsal methods and generative replay methods. Generative Replay consists of regenerating past learning experiences with a generative model to remember them. Rehearsal consists of saving a core-set of samples from past learning experiences to rehearse them later. The replay processes make possible a compromise between optimizing the current learning objective and the past ones enabling learning without forgetting in sequences of tasks settings. We show that they are very promising methods for continual learning. Notably, they enable the re-evaluation of past data with new knowledge and the confrontation of data from different learning-experiences. We demonstrate their ability to learn continually through unsupervised learning, supervised learning and reinforcement learning tasks.

Replay in Deep Learning: Current Approaches and Missing Biological Elements

Brain-Like Replay Naturally Emerges in Reinforcement Learning Agents

Learning offline: memory replay in biological and artificial reinforcement learning

A model of hippocampal replay driven by experience and environmental structure facilitates spatial learning

A unifying account of replay as context-driven memory reactivation

Continual Learning: Tackling Catastrophic Forgetting in Deep Neural Networks with Replay Processes

Continual Learning with Deep Generative Replay

Model-Based and Model-Free Replay Mechanisms for Reinforcement Learning in Neurorobotics

Augmented Replay Memory in Reinforcement Learning With Continuous Control

A Benchmark and Empirical Analysis for Replay Strategies in Continual Learning

Prioritized Sweeping Neural DynaQ with Multiple Predecessors, and Hippocampal Replays

Map-based Experience Replay: A Memory-Efficient Solution to Catastrophic Forgetting in Reinforcement Learning

Replay in human visual cortex is linked to the formation of successor representations and independent of consciousness

Unsupervised Replay Strategies for Continual Learning with Limited Data

Replay and compositional computation

Replay-enhanced Continual Reinforcement Learning

Paradoxical replay can protect contextual task representations from destructive interference when experience is unbalanced

Memory Enhanced Replay for Continual Learning

Replay-triggered brain-wide activation in humans

Memory Efficient Experience Replay for Streaming Learning

Prioritized Experience Replay