YOLOPose V2: Understanding and Improving Transformer-based 6D Pose Estimation

Arul Selvam Periyasamy,Arash Amini,Vladimir Tsaturyan,Sven Behnke

2023-07-21

Abstract:6D object pose estimation is a crucial prerequisite for autonomous robot manipulation applications. The state-of-the-art models for pose estimation are convolutional neural network (CNN)-based. Lately, Transformers, an architecture originally proposed for natural language processing, is achieving state-of-the-art results in many computer vision tasks as well. Equipped with the multi-head self-attention mechanism, Transformers enable simple single-stage end-to-end architectures for learning object detection and 6D object pose estimation jointly. In this work, we propose YOLOPose (short form for You Only Look Once Pose estimation), a Transformer-based multi-object 6D pose estimation method based on keypoint regression and an improved variant of the YOLOPose model. In contrast to the standard heatmaps for predicting keypoints in an image, we directly regress the keypoints. Additionally, we employ a learnable orientation estimation module to predict the orientation from the keypoints. Along with a separate translation estimation module, our model is end-to-end differentiable. Our method is suitable for real-time applications and achieves results comparable to state-of-the-art methods. We analyze the role of object queries in our architecture and reveal that the object queries specialize in detecting objects in specific image regions. Furthermore, we quantify the accuracy trade-off of using datasets of smaller sizes to train our model.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The paper primarily focuses on addressing a key technical issue in autonomous robotic operations—6D object pose estimation. Specifically, the research team proposed YOLOPose V2, a multi-object 6D pose estimation method based on the Transformer architecture. The main contributions of this method include:

1. **Proposing a real-time single-stage model based on keypoint regression**: This model can perform monocular 6D pose estimation for multiple objects simultaneously by directly regressing the positions of keypoints.

2. **Introducing a learnable direction estimation module**: This module can estimate the direction of objects from the predicted keypoints, thereby constructing an end-to-end differentiable pose estimation architecture.

3. **Achieving results comparable to existing top pose estimators**: It achieved comparable results on the YCB-Video dataset and has the fastest inference time.

Additionally, the paper analyzes the role of object queries in the YOLOPose architecture and improves the accuracy of the YOLOPose model by introducing new model variants. These improvements include adding extra inputs to the pose estimation multi-layer perceptrons (MLPs), quantifying the robustness of the learned perspective-n-point (PnP) module, and quantifying the accuracy trade-offs when training models with smaller datasets.

In summary, this research aims to enhance the accuracy and efficiency of 6D pose estimation by leveraging the advantages of the Transformer architecture, thereby advancing the application of autonomous robots in real-world scenarios.