Abstract:As an important solution for 3D shape retrieval, a multi-view shape descriptor has achieved impressive performance. One crucial part of view-based shape descriptors is to interpret 3D structures through various 2D observations. Most existing methods like MVCNN believe that a strong classification model trained with deep learning, can often provide an efficient shape embedding for 3D shape retrieval. However, these methods pay much attention to discriminative models and none of them necessarily incorporate the underlying 3D properties of the objects from 2D images. In this paper, we present a novel encoder-decoder recurrent feature aggregation network (ERFA-Net) to address this problem. Aiming at emphasizing the 3D properties of 3D shapes in the fusion of multiple view features, 3D properties prediction tasks are introduced into the 3D shape retrieval. Specifically, an image sequence of the shape is recurrently aggregated into a discriminative shape embedding based on LSTM network, and then this latent shape embedding is trained to predict the original voxel grids and estimate images of unseen viewpoints. This generation task gives an effective supervision which makes the network exploit 3D properties of shapes through various 2D images. Our method achieves the state-of-the-art performance for 3D shape retrieval, on two large-scale 3D shape datasets, ModelNet and ShapeNetCore55. Extensive experiments show that the proposed 3D representation performs robust discrimination against view occlusion, and strong generation ability for various 3D shape tasks.

View Inter-Prediction GAN: Unsupervised Representation Learning for 3D Shapes by Learning Global Shape Memories to Support Local View Predictions.

View-Aware Geometry-Structure Joint Learning for Single-View 3D Shape Reconstruction

Hierarchical View Predictor: Unsupervised 3D Global Feature Learning Through Hierarchical Prediction among Unordered Views.

Virtual View Generation Based on 3D-Dense-attentive GAN Networks

Learning Discriminative and Generative Shape Embeddings for Three-Dimensional Shape Retrieval

3Dviewgraph: Learning Global Features for 3D Shapes from A Graph of Unordered Views with Attention

Do 2D GANs Know 3D Shape? Unsupervised 3D shape reconstruction from 2D Image GANs

Learning View-Based Graph Convolutional Network for Multi-View 3D Shape Analysis

A Unified Feature Representation and Learning Framework for 3D Shape

2D GANs Meet Unsupervised Single-view 3D Reconstruction

3D-Mask-GAN:Unsupervised Single-View 3D Object Reconstruction

Multi-Angle Point Cloud-VAE: Unsupervised Feature Learning for 3D Point Clouds from Multiple Angles by Joint Self-Reconstruction and Half-to-Half Prediction

Shape-Pose Ambiguity in Learning 3D Reconstruction from Images.

Conditional Single-View Shape Generation for Multi-View Stereo Reconstruction.

Shape Inpainting using 3D Generative Adversarial Network and Recurrent Convolutional Networks

Progressive Learning of 3D Reconstruction Network from 2D GAN Data

Look, Cast and Mold: Learning 3D Shape Manifold from Single-view Synthetic Data.

Compositionally Generalizable 3D Structure Prediction

Learning Canonical View Representation for 3D Shape Recognition with Arbitrary Views

Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling

Parts4Feature: Learning 3D Global Features from Generally Semantic Parts in Multiple Views