Toward 3D Object Reconstruction from Stereo Images

Haozhe Xie,Hongxun Yao,Shangchen Zhou,Shengping Zhang,Xiaoshuai Sun,Wenxiu Sun

DOI: https://doi.org/10.48550/arXiv.1910.08223

2019-10-21

Abstract:Inferring the 3D shape of an object from an RGB image has shown impressive results, however, existing methods rely primarily on recognizing the most similar 3D model from the training set to solve the problem. These methods suffer from poor generalization and may lead to low-quality reconstructions for unseen objects. Nowadays, stereo cameras are pervasive in emerging devices such as dual-lens smartphones and robots, which enables the use of the two-view nature of stereo images to explore the 3D structure and thus improve the reconstruction performance. In this paper, we propose a new deep learning framework for reconstructing the 3D shape of an object from a pair of stereo images, which reasons about the 3D structure of the object by taking bidirectional disparities and feature correspondences between the two views into account. Besides, we present a large-scale synthetic benchmarking dataset, namely StereoShapeNet, containing 1,052,976 pairs of stereo images rendered from ShapeNet along with the corresponding bidirectional depth and disparity maps. Experimental results on the StereoShapeNet benchmark demonstrate that the proposed framework outperforms the state-of-the-art methods.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to reconstruct the complete 3D shape of an object from a stereo image pair. Existing methods based on a single RGB image mainly rely on identifying the most similar 3D model from the training set to solve the problem, and these methods show poor generalization ability and low - quality reconstruction effects when dealing with unseen objects. In addition, although depth cameras can provide depth information, they are very sensitive to lighting conditions, and as the light intensity decreases, the depth accuracy will degrade rapidly. Therefore, this study proposes a new deep - learning framework that utilizes bidirectional disparity and feature correspondences in stereo image pairs to improve the inference of 3D structures, thereby enhancing the reconstruction performance.

Specifically, the main contributions of the paper include:
1. **Proposing a new framework**: This framework reconstructs the 3D shape of an object in the form of 3D volumes and point clouds by exploring the bidirectional disparity and feature correspondences between a pair of stereo images.
2. **Constructing a large - scale benchmark dataset**: Named StereoShapeNet, it contains more than 1 million pairs of stereo RGB images rendered from ShapeNet and their corresponding bidirectional depth maps and disparity maps, which is the first benchmark dataset for stereo 3D object reconstruction.
3. **Experimental results show**: The experimental results on the StereoShapeNet dataset show that the proposed method outperforms existing methods in both point - cloud and 3D - volume reconstruction.

Through these contributions, this study not only overcomes the limitations of existing methods in dealing with unseen objects but also provides an important foundation and direction for future research.