Abstract:Our brains extract durable, generalizable knowledge from transient experiences of the world. Artificial neural networks come nowhere close to this ability. When tasked with learning to classify objects by training on non-repeating video frames in temporal order (online stream learning), models that learn well from shuffled datasets catastrophically forget old knowledge upon learning new stimuli. We propose a new continual learning algorithm, Compositional Replay Using Memory Blocks (CRUMB), which mitigates forgetting by replaying feature maps reconstructed by combining generic parts. CRUMB concatenates trainable and re-usable "memory block" vectors to compositionally reconstruct feature map tensors in convolutional neural networks. Storing the indices of memory blocks used to reconstruct new stimuli enables memories of the stimuli to be replayed during later tasks. This reconstruction mechanism also primes the neural network to minimize catastrophic forgetting by biasing it towards attending to information about object shapes more than information about image textures, and stabilizes the network during stream learning by providing a shared feature-level basis for all training examples. These properties allow CRUMB to outperform an otherwise identical algorithm that stores and replays raw images, while occupying only 3.6% as much memory. We stress-tested CRUMB alongside 13 competing methods on 7 challenging datasets. To address the limited number of existing online stream learning datasets, we introduce 2 new benchmarks by adapting existing datasets for stream learning. With only 3.7-4.1% as much memory and 15-43% as much runtime, CRUMB mitigates catastrophic forgetting more effectively than the state-of-the-art. Our code is available at <a class="link-external link-https" href="https://github.com/MorganBDT/crumb.git" rel="external noopener nofollow">this https URL</a>.

Memory Efficient Experience Replay for Streaming Learning

Tuned Compositional Feature Replays for Efficient Stream Learning

Dual Memory Model for Experience-Once Task-Incremental Lifelong Learning.

Map-based Experience Replay: A Memory-Efficient Solution to Catastrophic Forgetting in Reinforcement Learning

Summarizing Stream Data for Memory-Constrained Online Continual Learning

Replay-enhanced Continual Reinforcement Learning

A Benchmark and Empirical Analysis for Replay Strategies in Continual Learning

A rehearsal framework for computational efficiency in online continual learning

Lifelong Machine Learning with Deep Streaming Linear Discriminant Analysis

Learning What to Remember: Long-term Episodic Memory Networks for Learning from Streaming Data

Augmented Replay Memory in Reinforcement Learning With Continuous Control

Streaming LifeLong Learning With Any-Time Inference

AdaER: An Adaptive Experience Replay Approach for Continual Lifelong Learning

Memory-efficient training with streaming dimensionality reduction

Adaptive Memory Replay for Continual Learning

Continual Learning: Tackling Catastrophic Forgetting in Deep Neural Networks with Replay Processes

SLER: Self-generated long-term experience replay for continual reinforcement learning

Replay in Deep Learning: Current Approaches and Missing Biological Elements

TEAL: New Selection Strategy for Small Buffers in Experience Replay Class Incremental Learning

Ddper - Decentralized Distributed Prioritized Experience Replay.

A Unified Replay-based Continuous Learning Framework for Spatio-Temporal Prediction on Streaming Data