AutoURDF: Unsupervised Robot Modeling from Point Cloud Frames Using Cluster Registration

Jiong Lin,Lechen Zhang,Kwansoo Lee,Jialong Ning,Judah Goldfeder,Hod Lipson
2024-12-07
Abstract:Robot description models are essential for simulation and control, yet their creation often requires significant manual effort. To streamline this modeling process, we introduce AutoURDF, an unsupervised approach for constructing description files for unseen robots from point cloud frames. Our method leverages a cluster-based point cloud registration model that tracks the 6-DoF transformations of point clusters. Through analyzing cluster movements, we hierarchically address the following challenges: (1) moving part segmentation, (2) body topology inference, and (3) joint parameter estimation. The complete pipeline produces robot description files that are fully compatible with existing simulators. We validate our method across a variety of robots, using both synthetic and real-world scan data. Results indicate that our approach outperforms previous methods in registration and body topology estimation accuracy, offering a scalable solution for automated robot modeling.
Robotics,Computer Vision and Pattern Recognition
What problem does this paper attempt to address?
The problem that this paper attempts to solve is: **How to automatically construct a robot description file, especially a URDF (Unified Robot Description Format) file, from point cloud frames without relying on manual annotation or prior knowledge**. ### Detailed Explanation #### 1. **Problem Background** Accurate and structured robot models are crucial for applications such as real - time control, motion planning, and physics - based simulation. Although existing robot description files (such as URDF) can clearly capture the geometric, kinematic, and dynamic characteristics of robots, creating these files usually requires a great deal of manual effort, including CAD model conversion or cumbersome XML file modification. This makes the automatic generation of robot description files an urgent problem to be solved. #### 2. **Existing Challenges** - **Manual Modeling is Time - consuming and Labor - intensive**: The traditional process of creating robot description files depends on manual operations and is inefficient. - **Difficult to Handle Complex Robot Structures**: Existing methods are mainly for simple objects (such as scissors, laptops, etc.), and for robots with multiple joints and complex kinematic structures, existing methods are difficult to be directly applied. - **Lack of Large - scale Datasets**: Large - scale datasets representing diverse robot morphologies are still insufficient, which limits the application of supervised learning methods. #### 3. **Research Objectives** This paper proposes an unsupervised method named AutoURDF, aiming to automatically generate robot description files (URDF) from time - series point cloud frames. Specifically, this method is achieved through the following steps: - **Segment Moving Parts**: Cluster the points in the point cloud frames and segment different moving parts according to the motion patterns of the point clusters. - **Infer Body Topology**: Use the minimum spanning tree (MST) algorithm to infer the connection relationships between components. - **Estimate Joint Parameters**: Calculate the transformation matrix of each sub - component relative to its parent component, thereby deriving the joint parameters. #### 4. **Innovations** - **No Prior Knowledge Required**: AutoURDF does not require prior knowledge about robot structures or kinematics, nor does it require ground - truth annotations or manual interventions. - **Purely Driven by Visual Data**: This method only depends on visual data (point cloud frames) and does not depend on sensor motion data. - **Wide Applicability**: This method is applicable to various types of robots, including single - branch and multi - branch robotic arms with complex structures of 5 to 18 degrees of freedom. #### 5. **Experimental Verification** Researchers have verified the effectiveness of AutoURDF through a series of experiments, including testing multiple robots using synthetic data and real - scan data. The results show that this method outperforms existing methods in point cloud registration and body topology inference and can generate description files that are highly consistent with real URDF files. ### Summary The core problem of this paper is to automatically construct robot description files from point cloud frames through unsupervised learning, which solves the problems of time - consuming and labor - intensive manual modeling and difficult handling of complex robot structures in traditional methods.