Continual Learning for Autonomous Robots: A Prototype-based Approach

Elvin Hajizada,Balachandran Swaminathan,Yulia Sandamirskaya

2024-03-31

Abstract:Humans and animals learn throughout their lives from limited amounts of sensed data, both with and without supervision. Autonomous, intelligent robots of the future are often expected to do the same. The existing continual learning (CL) methods are usually not directly applicable to robotic settings: they typically require buffering and a balanced replay of training data. A few-shot online continual learning (FS-OCL) setting has been proposed to address more realistic scenarios where robots must learn from a non-repeated sparse data stream. To enable truly autonomous life-long learning, an additional challenge of detecting novelties and learning new items without supervision needs to be addressed. We address this challenge with our new prototype-based approach called Continually Learning Prototypes (CLP). In addition to being capable of FS-OCL learning, CLP also detects novel objects and learns them without supervision. To mitigate forgetting, CLP utilizes a novel metaplasticity mechanism that adapts the learning rate individually per prototype. CLP is rehearsal-free, hence does not require a memory buffer, and is compatible with neuromorphic hardware, characterized by ultra-low power consumption, real-time processing abilities, and on-chip learning. Indeed, we have open-sourced a simple version of CLP in the neuromorphic software framework Lava, targetting Intel's neuromorphic chip Loihi 2. We evaluate CLP on a robotic vision dataset, OpenLORIS. In a low-instance FS-OCL scenario, CLP shows state-of-the-art results. In the open world, CLP detects novelties with superior precision and recall and learns features of the detected novel classes without supervision, achieving a strong baseline of 99% base class and 65%/76% (5-shot/10-shot) novel class accuracy.

Machine Learning,Computer Vision and Pattern Recognition,Robotics

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to achieve continual learning (CL) in autonomous robots. Especially in an open - world environment, robots need to be able to learn new knowledge online from a small number of samples, detect and learn new categories without supervision, and at the same time avoid catastrophic forgetting. Specifically, the paper proposes a new method named "Continually Learning Prototypes (CLP)", aiming to address the following challenges:

1. **Online Continual Learning**: Robots need to be able to keep learning in non - repeating data streams. However, existing CL methods usually require buffers to store training data, which is not practical on robot platforms.
2. **Few - shot Learning**: Robots need to be able to learn new categories with only a small number of labeled samples.
3. **Open - world Recognition**: Robots need to be able to recognize and learn new categories that have not been seen before, not just the known categories.
4. **Unsupervised Learning**: Robots need to be able to detect and learn new categories without external supervision.
5. **Avoiding Catastrophic Forgetting**: Robots need to be able to retain old knowledge while learning new knowledge and avoid catastrophic forgetting.

To meet these challenges, CLP introduces the following key mechanisms:

- **Prototype - based Learning**: Use prototypes as representatives of categories and learn new knowledge by updating the most similar prototypes.
- **Metaplasticity Mechanism**: Dynamically adjust the learning rate of each prototype to resolve the contradiction between stability and plasticity, thereby avoiding catastrophic forgetting.
- **Novelty Detection**: Recognize and learn new categories by detecting samples that do not belong to the known categories.
- **Multi - modal Representation Learning**: Allow multiple prototypes to be assigned to each category to adapt to the representation of complex categories.

The paper verifies the effectiveness of CLP through experiments on the OpenLORIS dataset, demonstrating its superior performance in online continual learning, few - shot learning, open - world recognition, and unsupervised learning.