Abstract:Despite the recent progress on 6D object pose estimation methods for robotic grasping, a substantial performance gap persists between the capabilities of these methods on existing datasets and their efficacy in real-world grasping and mobile manipulation tasks, particularly when robots rely solely on their monocular egocentric field of view (FOV). Existing real-world datasets primarily focus on table-top grasping scenarios, where a robot arm is placed in a fixed position and the objects are centralized within the FOV of fixed external camera(s). Assessing performance on such datasets may not accurately reflect the challenges encountered in everyday grasping and mobile manipulation tasks within kitchen environments such as retrieving objects from higher shelves, sinks, dishwashers, ovens, refrigerators, or microwaves. To address this gap, we present KITchen, a novel benchmark designed specifically for estimating the 6D poses of objects located in diverse positions within kitchen settings. For this purpose, we recorded a comprehensive dataset comprising around 205k real-world RGBD images for 111 kitchen objects captured in two distinct kitchens, utilizing a humanoid robot with its egocentric perspectives. Subsequently, we developed a semi-automated annotation pipeline, to streamline the labeling process of such datasets, resulting in the generation of 2D object labels, 2D object segmentation masks, and 6D object poses with minimal human effort. The benchmark, the dataset, and the annotation pipeline will be publicly available at <a class="link-external link-https" href="https://kitchen-dataset.github.io/KITchen" rel="external noopener nofollow">this https URL</a>.

The RBO Dataset of Articulated Objects and Interactions

6DoF assembly pose estimation dataset for robotic manipulation

PokeFlex: Towards a Real-World Dataset of Deformable Objects for Robotic Manipulation

HANDAL: A Dataset of Real-World Manipulable Object Categories with Pose Annotations, Affordances, and Reconstructions

CORE4D: A 4D Human-Object-Human Interaction Dataset for Collaborative Object REarrangement

BEHAVE: Dataset and Method for Tracking Human Object Interactions

PokeFlex: A Real-World Dataset of Deformable Objects for Robotics

DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset

GAPartManip: A Large-scale Part-centric Dataset for Material-Agnostic Articulated Object Manipulation

HBOD: A Novel Dataset with Synchronized Hand, Body, and Object Manipulation Data for Human-Robot Interaction

LHManip: A Dataset for Long-Horizon Language-Grounded Manipulation Tasks in Cluttered Tabletop Environments

RPMArt: Towards Robust Perception and Manipulation for Articulated Objects

AKB-48: A Real-World Articulated Object Knowledge Base

KITchen: A Real-World Benchmark and Dataset for 6D Object Pose Estimation in Kitchen Environments

Recovering Articulated Object Models from 3D Range Data

TACO: Benchmarking Generalizable Bimanual Tool-ACtion-Object Understanding

JRDB-Pose: A Large-scale Dataset for Multi-Person Pose Estimation and Tracking

IKEA Object State Dataset: A 6DoF object pose estimation dataset and benchmark for multi-state assembly objects

Building Rearticulable Models for Arbitrary 3D Objects from 4D Point Clouds

RoCap: A Robotic Data Collection Pipeline for the Pose Estimation of Appearance-Changing Objects

ADL4D: Towards A Contextually Rich Dataset for 4D Activities of Daily Living