Abstract:Human life is populated with articulated objects. Current Category-level Articulation Pose Estimation (CAPE) methods are studied under the single-instance setting with a fixed kinematic structure for each category. Considering these limitations, we aim to study the problem of estimating part-level 6D pose for multiple articulated objects with unknown kinematic structures in a single RGB-D image, and reform this problem setting for real-world environments and suggest a CAPE-Real (CAPER) task setting. This setting allows varied kinematic structures within a semantic category, and multiple instances to co-exist in an observation of real world. To support this task, we build an articulated model repository ReArt-48 and present an efficient dataset generation pipeline, which contains Fast Articulated Object Modeling (FAOM) and Semi-Authentic MixEd Reality Technique (SAMERT). Accompanying the pipeline, we build a large-scale mixed reality dataset ReArtMix and a real world dataset ReArtVal. Accompanying the CAPER problem and the dataset, we propose an effective framework that exploits RGB-D input to estimate part-level pose for multiple instances in a single forward pass. In our method, we introduce object detection from RGB-D input to handle the multi-instance problem and segment each instance into several parts. To address the unknown kinematic structure issue, we propose an Articulation Parsing Network to analyze the structure of detected instance, and also build a Pair Articulation Pose Estimation module to estimate per-part 6D pose as well as joint property from connected part pairs. Extensive experiments demonstrate that the proposed method can achieve good performance on CAPER, CAPE and instance-level Robot Arm pose estimation problems. We believe it could serve as a strong baseline for future research on the CAPER task. The datasets and codes in our work will be made publicly available.

Generalizable Articulated Object Perception with Superpoints

GAPartNet: Cross-Category Domain-Generalizable Object Perception and Manipulation via Generalizable and Actionable Parts

3D Object Segmentation Using Cross-Window Point Transformer with Latent Semantic Boundary Guidance

RPMArt: Towards Robust Perception and Manipulation for Articulated Objects

Articulated Object Manipulation using Online Axis Estimation with SAM2-Based Tracking

GAPartManip: A Large-scale Part-centric Dataset for Material-Agnostic Articulated Object Manipulation

Superpoint-guided Semi-supervised Semantic Segmentation of 3D Point Clouds

Articulated Object Manipulation with Coarse-to-fine Affordance for Mitigating the Effect of Point Cloud Noise

ManipGPT: Is Affordance Segmentation by Large Vision Models Enough for Articulated Object Manipulation?

One Point, One Object: Simultaneous 3D Object Segmentation and 6-DOF Pose Estimation

APEX: Unsupervised, Object-Centric Scene Segmentation and Tracking for Robot Manipulation

Efficient LiDAR Point Cloud Oversegmentation Network

PointNet++ Grasping: Learning an End-to-end Spatial Grasp Generation Algorithm from Sparse Point Clouds

Building Rearticulable Models for Arbitrary 3D Objects from 4D Point Clouds

Refining Object Proposals Using Structured Edge and Superpixel Contrast in Robotic Grasping

Unseen Object Few-Shot Semantic Segmentation for Robotic Grasping

FlowBot++: Learning Generalized Articulated Objects Manipulation via Articulation Projection

PartManip: Learning Cross-Category Generalizable Part Manipulation Policy from Point Cloud Observations

SUGrasping: a Semantic Grasping Framework Based on Multi-Head 3D U-Net

Superpoint Transformer for 3D Scene Instance Segmentation.

Toward Real-World Category-Level Articulation Pose Estimation