Abstract:Category-specific 3D object shape models have greatly boosted the recent advances in object detection, recognition and segmentation. However, even the most advanced approach for learning 3D object shapes still requires heavy manual annotations on large-scale 2D images. Such annotations include object categories, object keypoints, and figure-ground segmentation for the instances in each image. In particular, annotating figure-ground segmentation is unbearably labor-intensive and time-consuming. To address this problem, this paper devotes to learn category-specific 3D shape models under weak supervision, where only object categories and keypoints are required to be manually annotated on the training 2D images. By exploring the underlying relationship between two tasks: object segmentation and category-specific 3D shape reconstruction, we propose a novel weakly-supervised learning framework to jointly address these two tasks and combine them to boost the final performance of the learned 3D shape models. Moreover, learning without using figure-ground segmentation leads to ambiguous solutions. To this end, we develop the confidence weighting schemes in the viewpoint estimation and 3D shape learning procedure. These schemes effectively reduce the confusion caused by the noisy data and thus increase the chances for recovering more reliable 3D object shapes. Comprehensive experiments on the challenging PASCAL VOC benchmark show that our framework achieves comparable performance with the state-of-the-art methods that use expensive manual segmentation-level annotations. In addition, our experiments also demonstrate that our 3D shape models improve object segmentation performance.

Weakly Supervised Learning for Single Depth-Based Hand Shape Recovery

Weakly-Supervised 3D Hand Reconstruction with Knowledge Prior and Uncertainty Guidance

WHSP-Net: A Weakly-Supervised Approach for 3D Hand Shape and Pose Recovery from a Single Depth Image

Model-based 3D Hand Reconstruction via Self-Supervised Learning

3D Hand Pose Estimation and Shape Reconstruction Based on Multi-task Learning

Weakly-Supervised 3d Hand Pose Estimation From Monocular Rgb Images

Weakly Supervised Part-wise 3D Shape Reconstruction from Single-View RGB Images

DeepHPS: End-to-end Estimation of 3D Hand Pose and Shape by Learning from Synthetic Depth

HandFormer: Hand Pose Reconstructing from a Single RGB Image

Weakly Supervised 3D Human Pose and Shape Reconstruction with Normalizing Flows

End-to-End Weakly-Supervised Single-Stage Multiple 3d Hand Mesh Reconstruction from a Single Rgb Image

Interacting Two-Hand 3D Pose and Shape Reconstruction from Single Color Image

Weakly Supervised 3D Hand Pose Estimation via Biomechanical Constraints

3D Hand Pose Estimation Using Synthetic Data and Weakly Labeled RGB Images.

3D Hand Shape and Pose Estimation from a Single RGB Image

Semi-supervised Hand Appearance Recovery Via Structure Disentanglement and Dual Adversarial Discrimination

A Dual-Branch Self-Boosting Framework for Self-Supervised 3D Hand Pose Estimation

Consistent 3D Hand Reconstruction in Video Via Self-Supervised Learning

Hand Pose Estimation through Semi-Supervised and Weakly-Supervised Learning

3D Reconstruction of Objects in Hands without Real World 3D Supervision

Weakly-Supervised Learning of Category-Specific 3D Object Shapes.