Abstract:In the domain of single-view 3D reconstruction, traditional techniques have frequently relied on expensive and time-intensive 3D annotation data. Facing the challenge of annotation acquisition, semi-supervised learning strategies offer an innovative approach to reduce the dependence on labeled data. Despite these developments, the utilization of this learning paradigm in 3D reconstruction tasks remains relatively constrained. In this research, we created an innovative semi-supervised framework for 3D reconstruction that distinctively uniquely introduces a multi shape prior fusion strategy, intending to guide the creation of more realistic object structures. Additionally, to improve the quality of shape generation, we integrated a self-attention module into the traditional decoder. In benchmark tests on the ShapeNet dataset, our method substantially outperformed existing supervised learning methods at diverse labeled ratios of 1\%, 10\%, and 20\%. Moreover, it showcased excellent performance on the real-world Pix3D dataset. Through comprehensive experiments on ShapeNet, our framework demonstrated a 3.3\% performance improvement over the baseline. Moreover, stringent ablation studies further confirmed the notable effectiveness of our approach. Our code has been released on <a class="link-external link-https" href="https://github.com/NWUzhouwei/SSMP" rel="external noopener nofollow">this https URL</a>

What problem does this paper attempt to address?

This paper aims to solve several key problems in single - view 3D reconstruction: 1. **Labeled data - dependence problem**: Traditional techniques for single - view 3D reconstruction usually rely on a large amount of expensive and time - consuming 3D labeled data. This not only increases the cost of data acquisition but also limits the generalization ability of the model. The paper proposes a semi - supervised learning framework, which reduces the dependence on labeled data by combining a small amount of labeled data with a large amount of unlabeled data. 2. **Reconstruction accuracy problem**: In the single - view 3D reconstruction task, the complex non - linear relationship between 2D images and 3D structures makes model learning difficult. Moreover, the spatial information carried by single - view images is limited, resulting in great uncertainty in reconstruction results. The paper improves the quality of 3D shape generation by introducing a multi - shape prior fusion strategy and a self - attention mechanism. 3. **Detail - capturing problem**: Traditional point - cloud methods perform poorly when dealing with slender or narrow parts of objects and are prone to missing key details. The paper generates more complete point clouds by using a multi - shape prior fusion strategy, thereby better capturing and extracting these details. Specifically, the main contributions of the paper include: - **Semi - supervised learning paradigm**: A semi - supervised 3D shape reconstruction network is proposed, which can efficiently realize 3D point - cloud reconstruction with only a small amount of labeled data. - **Multi - shape prior fusion strategy**: Different from the traditional method of using spherical point clouds as input, the paper uses a multi - shape prior fusion strategy to generate an average point cloud, thereby more comprehensively capturing and fusing the features of various shapes. - **Self - attention mechanism decoder**: The self - attention mechanism is introduced into the decoder, which enhances the model's ability to capture salient features during the reconstruction process, thereby restoring the input 3D shape structure at a finer granularity and improving the reconstruction quality. Through these innovations, the paper has achieved significant performance improvements in multiple benchmark tests and has shown good results in practical applications.

Semi-supervised Single-view 3D Reconstruction via Multi Shape Prior Fusion Strategy and Self-Attention

Semi-Supervised Single-View 3D Reconstruction Via Prototype Shape Priors

Single-view 3D Scene Reconstruction with High-fidelity Shape and Texture

Self-Supervised 3d Face Reconstruction Based On Multi-View Uv Fusion

Video Supervised for 3D Reconstruction from Single Image

Diffusion-Driven Self-Supervised Learning for Shape Reconstruction and Pose Estimation

Self-Supervised 3D Mesh Reconstruction from Single Images

Multi-view 3D Object Reconstruction and Uncertainty Modelling with Neural Shape Prior

Single-view 3D Mesh Reconstruction for Seen and Unseen Categories

2L3: Lifting Imperfect Generated 2D Images into Accurate 3D

Self-supervised reflectance-guided 3d shape reconstruction from single-view images

Segmentation-aware Prior Assisted Joint Global Information Aggregated 3D Building Reconstruction

Semi-Supervised 3D Shape Segmentation via Self Refining

Semi-supervised 3D shape segmentation with multilevel consistency and part substitution

3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction

Learning Shape Priors for Single-View 3D Completion and Reconstruction

3D Shape Completion on Unseen Categories: A Weakly-Supervised Approach

Feature Sharing Attention 3D Face Reconstruction with Unsupervised Learning from In-the-Wild Photo Collection

Semi-supervised Three-dimensional Reconstruction Framework with GAN.

Attention Aware Cost Volume Pyramid Based Multi-view Stereo Network for 3D Reconstruction

3D3M: 3D Modulated Morphable Model for Monocular Face Reconstruction