Abstract:The recent breakthroughs in computer vision have benefited from the availability of large representative datasets (e.g. ImageNet and COCO) for training. Yet, robotic vision poses unique challenges for applying visual algorithms developed from these standard computer vision datasets due to their implicit assumption over non-varying distributions for a fixed set of tasks. Fully retraining models each time a new task becomes available is infeasible due to computational, storage and sometimes privacy issues, while naïve incremental strategies have been shown to suffer from catastrophic forgetting. It is crucial for the robots to operate continuously under open-set and detrimental conditions with adaptive visual perceptual systems, where lifelong learning is a fundamental capability. However, very few datasets and benchmarks are available to evaluate and compare emerging techniques. To fill this gap, we provide a new lifelong robotic vision dataset ("OpenLORIS-Object") collected via RGB-D cameras. The dataset embeds the challenges faced by a robot in the real-life application and provides new benchmarks for validating lifelong object recognition algorithms. Moreover, we have provided a testbed of $9$ state-of-the-art lifelong learning algorithms. Each of them involves $48$ tasks with $4$ evaluation metrics over the OpenLORIS-Object dataset. The results demonstrate that the object recognition task in the ever-changing difficulty environments is far from being solved and the bottlenecks are at the forward/backward transfer designs. Our dataset and benchmark are publicly available at at \href{<a class="link-external link-https" href="https://lifelong-robotic-vision.github.io/dataset/object" rel="external noopener nofollow">this https URL</a>}{\underline{<a class="link-external link-https" href="https://lifelong-robotic-vision.github.io/dataset/object" rel="external noopener nofollow">this https URL</a>}}.

LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning

Lifelong Robot Library Learning: Bootstrapping Composable and Generalizable Skills for Embodied Control with Language Models

CLFR-M: Continual Learning Framework for Robots Via Human Feedback and Dynamic Memory

Learning with Recoverable Forgetting

Active Task Selection for Lifelong Machine Learning

A Versatile Agent for Fast Learning from Human Instructors

OpenLORIS-Object: A Robotic Vision Dataset and Benchmark for Lifelong Deep Learning

Task-unaware Lifelong Robot Learning with Retrieval-based Weighted Local Adaptation

Generalized Robot Learning Framework

Lifelong Robot Learning with Human Assisted Language Planners

OpenLORIS-Object: A Dataset and Benchmark towards Lifelong Object Recognition

Continuous Coordination As a Realistic Scenario for Lifelong Learning

A Simple Lifelong Learning Approach

Balanced Meta Learning and Diverse Sampling for Lifelong Task-Oriented Dialogue Systems.

Towards Lifelong Object Recognition: A Dataset and Benchmark

RoboCoder: Robotic Learning from Basic Skills to General Tasks with Large Language Models

Policy Stitching: Learning Transferable Robot Policies

Active Lifelong Learning with "Watchdog"

Evaluations of the Gap Between Supervised and Reinforcement Lifelong Learning on Robotic Manipulation Tasks

VLABench: A Large-Scale Benchmark for Language-Conditioned Robotics Manipulation with Long-Horizon Reasoning Tasks

LOTUS: Continual Imitation Learning for Robot Manipulation Through Unsupervised Skill Discovery