Abstract:The International Journal of Robotics Research, Ahead of Print. In this paper, we propose a real-world benchmark for studying robotic learning in the context of functional manipulation: a robot needs to accomplish complex long-horizon behaviors by composing individual manipulation skills in functionally relevant ways. The core design principles of our Functional Manipulation Benchmark (FMB) emphasize a harmonious balance between complexity and accessibility. Tasks are deliberately scoped to be narrow, ensuring that models and datasets of manageable scale can be utilized effectively to track progress. Simultaneously, they are diverse enough to pose a significant generalization challenge. Furthermore, the benchmark is designed to be easily replicable, encompassing all essential hardware and software components. To achieve this goal, FMB consists of a variety of 3D-printed objects designed for easy and accurate replication by other researchers. The objects are procedurally generated, providing a principled framework to study generalization in a controlled fashion. We focus on fundamental manipulation skills, including grasping, repositioning, and a range of assembly behaviors. The FMB can be used to evaluate methods for acquiring individual skills, as well as methods for effectively combining and ordering such skills in order to solve complex, multi-stage manipulation tasks. We also offer an imitation learning framework that includes a suite of policies trained to solve the proposed tasks. This enables researchers to utilize our tasks as a versatile toolkit for examining various parts of the pipeline. For example, researchers could propose a better design for a grasping controller and evaluate it in combination with our baseline reorientation and assembly policies as part of a pipeline for solving multi-stage tasks. Our dataset, object CAD files, code, and evaluation videos can be found on our project website: https://functional-manipulation-benchmark.github.io.

An Empirical Study and Analysis of Learning Generalizable Manipulation Skill in the SAPIEN Simulator

Learning Robot Manipulation Skills from Human Demonstration Videos Using Two-Stream 2-D/3-D Residual Networks with Self-Attention

Silver-Bullet-3D at ManiSkill 2021: Learning-from-Demonstrations and Heuristic Rule-based Methods for Object Manipulation

Grounding Language for Robotic Manipulation via Skill Library

ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skills

A Two-stage Fine-tuning Strategy for Generalizable Manipulation Skill of Embodied AI

Ag2Manip: Learning Novel Manipulation Skills with Agent-Agnostic Visual and Action Representations

Unsupervised Skill Discovery for Robotic Manipulation through Automatic Task Generation

Learning Category-Level Generalizable Object Manipulation Policy Via Generative Adversarial Self-Imitation Learning from Demonstrations

Learning Generalizable 3D Manipulation With 10 Demonstrations

Kinematic-aware Prompting for Generalizable Articulated Object Manipulation with LLMs

Learning the Generalizable Manipulation Skills on Soft-body Tasks via Guided Self-attention Behavior Cloning Policy

Learning Manipulation by Predicting Interaction

PartManip: Learning Cross-Category Generalizable Part Manipulation Policy from Point Cloud Observations

ManiGaussian: Dynamic Gaussian Splatting for Multi-task Robotic Manipulation

Efficient Robot Skill Learning with Imitation from a Single Video for Contact-Rich Fabric Manipulation

Language-Conditioned Imitation Learning with Base Skill Priors under Unstructured Data

ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AI

SkillMimicGen: Automated Demonstration Generation for Efficient Skill Learning and Deployment

FMB: A functional manipulation benchmark for generalizable robotic learning