Abstract:The recent breakthroughs in computer vision have benefited from the availability of large representative datasets (e.g. ImageNet and COCO) for training. Yet, robotic vision poses unique challenges for applying visual algorithms developed from these standard computer vision datasets due to their implicit assumption over non-varying distributions for a fixed set of tasks. Fully retraining models each time a new task becomes available is infeasible due to computational, storage and sometimes privacy issues, while naïve incremental strategies have been shown to suffer from catastrophic forgetting. It is crucial for the robots to operate continuously under open-set and detrimental conditions with adaptive visual perceptual systems, where lifelong learning is a fundamental capability. However, very few datasets and benchmarks are available to evaluate and compare emerging techniques. To fill this gap, we provide a new lifelong robotic vision dataset ("OpenLORIS-Object") collected via RGB-D cameras. The dataset embeds the challenges faced by a robot in the real-life application and provides new benchmarks for validating lifelong object recognition algorithms. Moreover, we have provided a testbed of $9$ state-of-the-art lifelong learning algorithms. Each of them involves $48$ tasks with $4$ evaluation metrics over the OpenLORIS-Object dataset. The results demonstrate that the object recognition task in the ever-changing difficulty environments is far from being solved and the bottlenecks are at the forward/backward transfer designs. Our dataset and benchmark are publicly available at at \href{<a class="link-external link-https" href="https://lifelong-robotic-vision.github.io/dataset/object" rel="external noopener nofollow">this https URL</a>}{\underline{<a class="link-external link-https" href="https://lifelong-robotic-vision.github.io/dataset/object" rel="external noopener nofollow">this https URL</a>}}.

Wake Vision: A Tailored Dataset and Benchmark Suite for TinyML Computer Vision Applications

Visual Wake Words Dataset

MWIRSTD: A MWIR Small Target Detection Dataset

Descriptor: Face Detection Dataset for Programmable Threshold-Based Sparse-Vision

DarkVision: A Benchmark for Low-light Image/Video Perception

OpenLORIS-Object: A Robotic Vision Dataset and Benchmark for Lifelong Deep Learning

WebVision Database: Visual Learning and Understanding from Web Data

Benchmark for Generic Product Detection: A Low Data Baseline for Dense Object Detection

Towards Vision Mixture of Experts for Wildlife Monitoring on the Edge

MindSet: Vision. A toolbox for testing DNNs on key psychological experiments

BVI-RLV: A Fully Registered Dataset and Benchmarks for Low-Light Video Enhancement

Image Classification with Small Datasets: Overview and Benchmark

Webly Supervised Fine-Grained Recognition: Benchmark Datasets and An Approach

DailyDVS-200: A Comprehensive Benchmark Dataset for Event-Based Action Recognition

Evaluation of Human and Machine Face Detection using a Novel Distinctive Human Appearance Dataset

Finding Tiny Faces

Ultra-High-Definition Low-Light Image Enhancement: A Benchmark and Transformer-Based Method

BEHAVIOR Vision Suite: Customizable Dataset Generation via Simulation

Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments

MOS: A Low Latency and Lightweight Framework for Face Detection, Landmark Localization, and Head Pose Estimation

Toward RAW Object Detection: A New Benchmark and a New Model