Multi-level Reasoning for Robotic Assembly: From Sequence Inference to Contact Selection

Xinghao Zhu,Devesh K. Jha,Diego Romeres,Lingfeng Sun,Masayoshi Tomizuka,Anoop Cherian

2023-12-17

Abstract:Automating the assembly of objects from their parts is a complex problem with innumerable applications in manufacturing, maintenance, and recycling. Unlike existing research, which is limited to target segmentation, pose regression, or using fixed target blueprints, our work presents a holistic multi-level framework for part assembly planning consisting of part assembly sequence inference, part motion planning, and robot contact optimization. We present the Part Assembly Sequence Transformer (PAST) -- a sequence-to-sequence neural network -- to infer assembly sequences recursively from a target blueprint. We then use a motion planner and optimization to generate part movements and contacts. To train PAST, we introduce D4PAS: a large-scale Dataset for Part Assembly Sequences (D4PAS) consisting of physically valid sequences for industrial objects. Experimental results show that our approach generalizes better than prior methods while needing significantly less computational time for inference.

Robotics,Artificial Intelligence,Computer Vision and Pattern Recognition,Machine Learning

What problem does this paper attempt to address?

### Problems the Paper Attempts to Solve

This paper aims to address the complex problem of assembling parts into a whole by robots and proposes a multi-level framework to achieve this goal. Specifically, the research focuses on the following aspects:

1. **Assembly Sequence Inference**: Inferring a reasonable assembly sequence by inputting the target blueprint and unassembled parts' point cloud data.
2. **Parts Motion Planning**: Planning collision-free paths for each part.
3. **Contact Point Optimization**: Determining the contact points that the robot can utilize during the assembly process.

### Main Contributions

1. Proposed an assembly planning algorithm, including assembly sequence inference, parts motion planning, and contact point optimization.
2. Introduced the Part Assembly Sequence Transformer (PAST) model, which generates assembly sequences in an autoregressive manner and can generalize to new blueprints and part geometries.
3. Constructed a dataset containing assembly sequences, trajectories, and feasible contact points (D4PAS), providing a foundation for future research.

### Summary

The goal of the paper is to achieve the automated assembly process from parts to a complete object given a target blueprint. By proposing a multi-level framework, it addresses the issues in traditional methods that only focus on specific tasks (such as blueprint segmentation or fixed target assembly) and presents an efficient and general approach to accomplish complex robotic assembly tasks.