Abstract:The sketch-based 3D shape retrieval has been an active but challenging task for several decades. In this paper, we deeply analyze the challenges and propose a novel Hierarchical Domain-Augmented Adaptive Learning (HDA2L) for sketch-based 3D shape retrieval. The first notable challenge is the vast cross-modality discrepancies between sketches and 3D shapes. To this issue, the existing methods restrict the consistency of the final features by establishing a shared cross-domain loss but ignore the feature extraction process, resulting in a limited effect. We argue that the mutual information of the samples from the same class but different domains can provide an important cue to enhance common features captured in the feature extraction process. Thus, we design an Inter-Domain Augmented Network (Inter-DAN) by employing inter-domain feature correlation learning to capture cross-domain mutual information to learn augmented common global features for both sketches and 3D shapes. Another challenge is that the input sketch is various: it may be particularly abstract and contains only the overall outline of the target model, or it may be too sketchy and only contains some salient local regions of the target model. Though existing methods have demonstrated their capability to capture overall features, they always ignore the learning of local discriminative features and fail to adapt to the various changes of input sketches. To address this issue, we design an Intra-Domain Augmented Network (Intra-DAN) for sketches and 3D shapes, respectively, which learns augmented local discriminative features by adopting cascading cross-layer bilinear pooling operations. In addition, we design a Source-Agnostic Adversarial Network (SAAN) to accomplish the adaptive hierarchical domain features fusion, which forces the network to adaptively focus on more discriminative information from global features and local features and further adapt the diversity of the input sketches. The experiments on three benchmark datasets demonstrate that our method obtains superior retrieval performance than the state-of-the-art sketch-based 3D shape retrieval approaches.

Learning Barycentric Representations of 3D Shapes for Sketch-Based 3D Shape Retrieval.

Sketch-Based 3D Model Retrieval via Multi-feature Fusion

A Comparison of 3D Shape Retrieval Methods Based on a Large-Scale Benchmark Supporting Multimodal Queries

Extended Large Scale Sketch-Based 3D Shape Retrieval.

Viewpoint-Aware Representation for Sketch-Based 3D Model Retrieval

Deep Common Semantic Space Embedding for Sketch-Based 3D Model Retrieval.

A Unified Feature Representation and Learning Framework for 3D Shape

Deep Correlated Holistic Metric Learning for Sketch-Based 3D Shape Retrieval

Training Deep Convolutional Neural Networks to Acquire the Best View of a 3D Shape.

Sketch-based 3D Shape Retrieval Via Teacher–student Learning

A Dimensional Reduction Guiding Deep Learning Architecture for 3D Shape Retrieval

Cross-modal guidance network for sketch-based 3d shape retrieval

Learning Cross-Domain Neural Networks For Sketch-Based 3d Shape Retrieval

Toward Fine-Grained Sketch-Based 3D Shape Retrieval

3D Sketching for 3D Object Retrieval

A Metric Learning Method for Image-based 3D Shape Retrieval

Deep Sketch-Shape Hashing With Segmented 3D Stochastic Viewing

HDA2L: Hierarchical Domain-Augmented Adaptive Learning for Sketch-Based 3D Shape Retrieval

Discriminative Sketch-Based 3d Model Retrieval Via Robust Shape Matching

Novel Sketch-Based 3D Model Retrieval Via Cross-domain Feature Clustering and Matching

Sketch-Based 3D Shape Retrieval with Multi-View Fusion Transformer