Abstract:Category-specific 3D object shape models have greatly boosted the recent advances in object detection, recognition and segmentation. However, even the most advanced approach for learning 3D object shapes still requires heavy manual annotations on large-scale 2D images. Such annotations include object categories, object keypoints, and figure-ground segmentation for the instances in each image. In particular, annotating figure-ground segmentation is unbearably labor-intensive and time-consuming. To address this problem, this paper devotes to learn category-specific 3D shape models under weak supervision, where only object categories and keypoints are required to be manually annotated on the training 2D images. By exploring the underlying relationship between two tasks: object segmentation and category-specific 3D shape reconstruction, we propose a novel weakly-supervised learning framework to jointly address these two tasks and combine them to boost the final performance of the learned 3D shape models. Moreover, learning without using figure-ground segmentation leads to ambiguous solutions. To this end, we develop the confidence weighting schemes in the viewpoint estimation and 3D shape learning procedure. These schemes effectively reduce the confusion caused by the noisy data and thus increase the chances for recovering more reliable 3D object shapes. Comprehensive experiments on the challenging PASCAL VOC benchmark show that our framework achieves comparable performance with the state-of-the-art methods that use expensive manual segmentation-level annotations. In addition, our experiments also demonstrate that our 3D shape models improve object segmentation performance.

Weakly Supervised Part-wise 3D Shape Reconstruction from Single-View RGB Images

View-Aware Geometry-Structure Joint Learning for Single-View 3D Shape Reconstruction

Reconstructing Part-Level 3D Models from a Single Image.

Shape-Pose Ambiguity in Learning 3D Reconstruction from Images.

Weakly-Supervised Single-view Dense 3D Point Cloud Reconstruction Via Differentiable Renderer

Learning pose-invariant 3D object reconstruction from single-view images

Single-view 3D Reconstruction Algorithm Based on View-aware

Weakly Supervised Learning for Single Depth-Based Hand Shape Recovery

Do 2D GANs Know 3D Shape? Unsupervised 3D shape reconstruction from 2D Image GANs

View Inter-Prediction GAN: Unsupervised Representation Learning for 3D Shapes by Learning Global Shape Memories to Support Local View Predictions.

Video Supervised for 3D Reconstruction from Single Image

Im2Struct: Recovering 3D Shape Structure from a Single RGB Image

Learning Shape Priors for Single-View 3D Completion and Reconstruction

Deep 3D Shape Reconstruction from Single-View Sketch Image

Conditional Single-View Shape Generation for Multi-View Stereo Reconstruction.

Weakly-Supervised Learning of Category-Specific 3D Object Shapes.

Semi-Supervised Single-View 3D Reconstruction Via Prototype Shape Priors

Single-view 3D Mesh Reconstruction for Seen and Unseen Categories

3D-Mask-GAN:Unsupervised Single-View 3D Object Reconstruction

Semi-supervised Single-view 3D Reconstruction via Multi Shape Prior Fusion Strategy and Self-Attention

Unsupervised 3D Shape Reconstruction by Part Retrieval and Assembly