Abstract:Selective experience replay is a popular strategy for integrating lifelong learning with deep reinforcement learning. Selective experience replay aims to recount selected experiences from previous tasks to avoid catastrophic forgetting. Furthermore, selective experience replay based techniques are model agnostic and allow experiences to be shared across different models. However, storing experiences from all previous tasks make lifelong learning using selective experience replay computationally very expensive and impractical as the number of tasks increase. To that end, we propose a reward distribution-preserving coreset compression technique for compressing experience replay buffers stored for selective experience replay. We evaluated the coreset compression technique on the brain tumor segmentation (BRATS) dataset for the task of ventricle localization and on the whole-body MRI for localization of left knee cap, left kidney, right trochanter, left lung, and spleen. The coreset lifelong learning models trained on a sequence of 10 different brain MR imaging environments demonstrated excellent performance localizing the ventricle with a mean pixel error distance of 12.93 for the compression ratio of 10x. In comparison, the conventional lifelong learning model localized the ventricle with a mean pixel distance of 10.87. Similarly, the coreset lifelong learning models trained on whole-body MRI demonstrated no significant difference (p=0.28) between the 10x compressed coreset lifelong learning models and conventional lifelong learning models for all the landmarks. The mean pixel distance for the 10x compressed models across all the landmarks was 25.30, compared to 19.24 for the conventional lifelong learning models. Our results demonstrate that the potential of the coreset-based ERB compression method for compressing experiences without a significant drop in performance.

COBRA: A Continual Learning Approach to Vision-Brain Understanding

Extracting Semantic-Dynamic Features for Long-Term Stable Brain Computer Interface.

Personalized Continual EEG Decoding Framework for Knowledge Retention and Transfer

Brainformer: Mimic Human Visual Brain Functions to Machine Vision Models via fMRI

Neuro-Vision to Language: Enhancing Brain Recording-based Visual Reconstruction and Language Interaction

Vision Enhanced Neuro-Cognitive Structure for Robotic Spatial Cognition

A Biologically Plausible Audio-Visual Integration Model for Continual Learning

Incremental Concept Formation over Visual Images Without Catastrophic Forgetting

Continual Learning of a Transformer-Based Deep Learning Classifier Using an Initial Model from Action Observation EEG Data to Online Motor Imagery Classification

BrainVis: Exploring the Bridge between Brain and Visual Signals via Image Reconstruction

Neuro-Symbolic Continual Learning: Knowledge, Reasoning Shortcuts and Concept Rehearsal

Unidirectional brain-computer interface: Artificial neural network encoding natural images to fMRI response in the visual cortex

BRACTIVE: A Brain Activation Approach to Human Visual Brain Learning

Online Continual Learning with Contrastive Vision Transformer

Vision-Language Navigation with Continual Learning

Preventing Catastrophic Forgetting through Memory Networks in Continuous Detection

A unified framework to exploit information in BCI data for continuous prediction

See Through Their Minds: Learning Transferable Neural Representation from Cross-Subject fMRI

UMBRAE: Unified Multimodal Brain Decoding

Quantum-Brain: Quantum-Inspired Neural Network Approach to Vision-Brain Understanding

Selective experience replay compression using coresets for lifelong deep reinforcement learning in medical imaging