Abstract:Humans and animals learn throughout their lives from limited amounts of sensed data, both with and without supervision. Autonomous, intelligent robots of the future are often expected to do the same. The existing continual learning (CL) methods are usually not directly applicable to robotic settings: they typically require buffering and a balanced replay of training data. A few-shot online continual learning (FS-OCL) setting has been proposed to address more realistic scenarios where robots must learn from a non-repeated sparse data stream. To enable truly autonomous life-long learning, an additional challenge of detecting novelties and learning new items without supervision needs to be addressed. We address this challenge with our new prototype-based approach called Continually Learning Prototypes (CLP). In addition to being capable of FS-OCL learning, CLP also detects novel objects and learns them without supervision. To mitigate forgetting, CLP utilizes a novel metaplasticity mechanism that adapts the learning rate individually per prototype. CLP is rehearsal-free, hence does not require a memory buffer, and is compatible with neuromorphic hardware, characterized by ultra-low power consumption, real-time processing abilities, and on-chip learning. Indeed, we have open-sourced a simple version of CLP in the neuromorphic software framework Lava, targetting Intel's neuromorphic chip Loihi 2. We evaluate CLP on a robotic vision dataset, OpenLORIS. In a low-instance FS-OCL scenario, CLP shows state-of-the-art results. In the open world, CLP detects novelties with superior precision and recall and learns features of the detected novel classes without supervision, achieving a strong baseline of 99% base class and 65%/76% (5-shot/10-shot) novel class accuracy.

Replay Consolidation with Label Propagation for Continual Object Detection

RECL: Responsive Resource-Efficient Continuous Learning for Video Analytics

nVFNet-RDC: Replay and Non-Local Distillation Collaboration for Continual Object Detection

Latent Distillation for Continual Object Detection at the Edge

Augmented Box Replay: Overcoming Foreground Shift for Incremental Object Detection

Label-Efficient Online Continual Object Detection in Streaming Video

SDDGR: Stable Diffusion-based Deep Generative Replay for Class Incremental Object Detection

Pseudo Object Replay and Mining for Incremental Object Detection

Continual Learning: Less Forgetting, More OOD Generalization via Adaptive Contrastive Replay

Contrastive Correlation Preserving Replay for Online Continual Learning

RECALL+: Adversarial Web-based Replay for Continual Learning in Semantic Segmentation

Watch Your Step: Optimal Retrieval for Continual Learning at Scale

Continual Learning for Class- and Domain-Incremental Semantic Segmentation

IOR: Inversed Objects Replay for Incremental Object Detection

One-Shot Replay: Boosting Incremental Object Detection via Retrospecting One Object

Saliency-Guided Hidden Associative Replay for Continual Learning

May the Forgetting Be with You: Alternate Replay for Learning with Noisy Labels

An Attention-based Representation Distillation Baseline for Multi-Label Continual Learning

Continual Object Detection: A review of definitions, strategies, and challenges

Continual Learning for Autonomous Robots: A Prototype-based Approach

Prototype-Sample Relation Distillation: Towards Replay-Free Continual Learning