Abstract:In computer vision, estimating the six-degree-of-freedom pose from an RGB image is a fundamental task. However, this task becomes highly challenging in multi-object scenes. Currently, the best methods typically employ an indirect strategy, which identifies 2D and 3D correspondences, and then solves with the Perspective-n-Points method. Yet, this approach cannot be trained end-to-end. Direct methods, on the other hand, suffer from lower accuracy due to challenges such as varying object sizes and occlusions. To address these issues, we propose SEMPose, an end-to-end multi-object pose estimation network. SEMPose utilizes a well-designed texture-shape guided feature pyramid network, effectively tackling the challenge of object size variations. Additionally, it employs an iterative refinement head structure, progressively regressing rotation and translation separately to enhance estimation accuracy. During training, we alleviate the impact of occlusion by selecting positive samples from visible parts. Experimental results demonstrate that SEMPose can perform inference at 32 FPS without requiring inputs other than the RGB image. It can accurately estimate the poses of multiple objects in real time, with inference time unaffected by the number of target objects. On the LM-O and YCB-V datasets, our method outperforms other RGB-based single-model methods, achieving higher accuracy. Even when compared with multi-model methods and approaches that use additional refinement, our results remain competitive.
What problem does this paper attempt to address?
### What problems does this paper attempt to solve?
This paper aims to solve the six - degree - of - freedom (6D) pose estimation problem in multi - object scenes. Specifically, the author proposes a single end - to - end network named SEMPose for simultaneously estimating the poses of multiple objects from RGB images. The following are the key problems that this paper attempts to solve:
1. **Challenges in pose estimation in multi - object scenes**:
- **Limitations of indirect methods**: Currently, the state - of - the - art methods usually adopt an indirect strategy, that is, first identify 2D and 3D corresponding points, and then use the perspective - n - point (PnP) method to solve the pose. However, this method cannot be trained end - to - end.
- **Accuracy problems of direct methods**: Although direct methods can be trained end - to - end, they have lower accuracy when dealing with objects of different sizes and occlusion problems.
2. **The influence of object size changes and occlusion**:
- In multi - object scenes, the sizes and proportions of objects vary greatly, resulting in unbalanced feature extraction.
- The occlusion problem makes it difficult to detect the key points of some objects, which affects the accuracy of pose estimation.
3. **Complexity and resource consumption of existing methods**:
- Existing methods usually design networks for specific types of objects, and it is difficult to generalize to other types of objects, resulting in increased system complexity and resource consumption.
- In multi - object scenes, as the number of objects increases, the demand for computing resources increases significantly.
### SEMPose's solutions
To address the above challenges, SEMPose proposes the following innovations:
1. **Texture - Shape - Guided Feature Pyramid Network (TS - FPN)**:
- A structure for hierarchically capturing the fused features of different - sized objects is designed, which effectively solves the problem of object size changes.
2. **Iterative Refinement Head Structure**:
- Rotation and translation are regressed step by step respectively, which improves the accuracy of pose estimation.
3. **Positive Sample Selection Strategy Based on Visible Parts**:
- During the training process, by selecting the unoccluded parts as positive samples, the impact of occlusion on model training is reduced.
4. **Only RGB Image Input Required**:
- No additional 3D models, depth images, object symmetry information or real - region - information (RoI) are required, simplifying the input requirements.
### Experimental results
The experimental results show that SEMPose can perform inference at a speed of 32 FPS and is not affected by the number of target objects. On the LM - O and YCB - V datasets, SEMPose performs better than other single - model methods based on RGB images, and in some cases even outperforms multi - model methods and methods using additional refinement steps.
In summary, through innovative network structures and training strategies, SEMPose successfully solves multiple challenges in 6D pose estimation in multi - object scenes and achieves efficient and accurate pose estimation.