Abstract:Human life is populated with articulated objects. Current Category-level Articulation Pose Estimation (CAPE) methods are studied under the single-instance setting with a fixed kinematic structure for each category. Considering these limitations, we aim to study the problem of estimating part-level 6D pose for multiple articulated objects with unknown kinematic structures in a single RGB-D image, and reform this problem setting for real-world environments and suggest a CAPE-Real (CAPER) task setting. This setting allows varied kinematic structures within a semantic category, and multiple instances to co-exist in an observation of real world. To support this task, we build an articulated model repository ReArt-48 and present an efficient dataset generation pipeline, which contains Fast Articulated Object Modeling (FAOM) and Semi-Authentic MixEd Reality Technique (SAMERT). Accompanying the pipeline, we build a large-scale mixed reality dataset ReArtMix and a real world dataset ReArtVal. Accompanying the CAPER problem and the dataset, we propose an effective framework that exploits RGB-D input to estimate part-level pose for multiple instances in a single forward pass. In our method, we introduce object detection from RGB-D input to handle the multi-instance problem and segment each instance into several parts. To address the unknown kinematic structure issue, we propose an Articulation Parsing Network to analyze the structure of detected instance, and also build a Pair Articulation Pose Estimation module to estimate per-part 6D pose as well as joint property from connected part pairs. Extensive experiments demonstrate that the proposed method can achieve good performance on CAPER, CAPE and instance-level Robot Arm pose estimation problems. We believe it could serve as a strong baseline for future research on the CAPER task. The datasets and codes in our work will be made publicly available.

VoCAPTER: Voting-based Pose Tracking for Category-level Articulated Object Via Inter-frame Priors

CAPTRA: CAtegory-level Pose Tracking for Rigid and Articulated Objects from Point Clouds

Toward Real-World Category-Level Articulation Pose Estimation

ICK-Track: A Category-Level 6-Dof Pose Tracker Using Inter-Frame Consistent Keypoints for Aerial Manipulation.

PA-Pose: Partial Point Cloud Fusion Based on Reliable Alignment for 6D Pose Tracking

Category-Independent Articulated Object Tracking with Factor Graphs

3D Articulated Hand Tracking Based on Behavioral Model

A Robust Framework for 2D Human Pose Tracking with Spatial and Temporal Constraints

Articulated Object Manipulation using Online Axis Estimation with SAM2-Based Tracking

Towards Real-World Category-level Articulation Pose Estimation

Corr-Track: Category-Level 6D Pose Tracking with Soft-Correspondence Matrix Estimation

U-COPE: Taking a Further Step to Universal 9D Category-level Object Pose Estimation

CatTrack: Single-Stage Category-Level 6D Object Pose Tracking Via Convolution and Vision Transformer

Efficient Sampling of Two-Stage Multi-Person Pose Estimation and Tracking from Spatiotemporal

Towards Real-World Aerial Vision Guidance with Categorical 6D Pose Tracker

2D-3D Pose Tracking with Multi-View Constraints

Multi-Person Articulated Tracking With Spatial and Temporal Embeddings

Learning Variational Motion Prior for Video-based Motion Capture

vmTracking: Virtual Markers Overcome Occlusion and Crowding in Multi-Animal Pose Tracking

PoseVocab: Learning Joint-structured Pose Embeddings for Human Avatar Modeling

Articulated point pattern matching in optical motion capture systems