Abstract:The sketch-based 3D shape retrieval has been an active but challenging task for several decades. In this paper, we deeply analyze the challenges and propose a novel Hierarchical Domain-Augmented Adaptive Learning (HDA2L) for sketch-based 3D shape retrieval. The first notable challenge is the vast cross-modality discrepancies between sketches and 3D shapes. To this issue, the existing methods restrict the consistency of the final features by establishing a shared cross-domain loss but ignore the feature extraction process, resulting in a limited effect. We argue that the mutual information of the samples from the same class but different domains can provide an important cue to enhance common features captured in the feature extraction process. Thus, we design an Inter-Domain Augmented Network (Inter-DAN) by employing inter-domain feature correlation learning to capture cross-domain mutual information to learn augmented common global features for both sketches and 3D shapes. Another challenge is that the input sketch is various: it may be particularly abstract and contains only the overall outline of the target model, or it may be too sketchy and only contains some salient local regions of the target model. Though existing methods have demonstrated their capability to capture overall features, they always ignore the learning of local discriminative features and fail to adapt to the various changes of input sketches. To address this issue, we design an Intra-Domain Augmented Network (Intra-DAN) for sketches and 3D shapes, respectively, which learns augmented local discriminative features by adopting cascading cross-layer bilinear pooling operations. In addition, we design a Source-Agnostic Adversarial Network (SAAN) to accomplish the adaptive hierarchical domain features fusion, which forces the network to adaptively focus on more discriminative information from global features and local features and further adapt the diversity of the input sketches. The experiments on three benchmark datasets demonstrate that our method obtains superior retrieval performance than the state-of-the-art sketch-based 3D shape retrieval approaches.

Cross-modal guidance network for sketch-based 3d shape retrieval

JFLN: Joint Feature Learning Network for 2D Sketch Based 3D Shape Retrieval

Sketch-Based 3D Model Retrieval via Multi-feature Fusion

Learning Cross-Domain Neural Networks For Sketch-Based 3d Shape Retrieval

SceneSketcher-v2: Fine-Grained Scene-Level Sketch-Based Image Retrieval Using Adaptive GCNs

Learning Barycentric Representations of 3D Shapes for Sketch-Based 3D Shape Retrieval.

3D Graph-S<SUP>2</SUP>Net: Shape-Aware Self-ensembling Network for Semi-supervised Segmentation with Bilateral Graph Convolution

HDA2L: Hierarchical Domain-Augmented Adaptive Learning for Sketch-Based 3D Shape Retrieval

Uncertainty-Aware Cross-Modal Transfer Network for Sketch-Based 3D Shape Retrieval

Cross-domain Retrieving Sketch and Shape Using Cycle CNNs.

PAGML: Precise Alignment Guided Metric Learning for sketch-based 3D shape retrieval

A New Sketch-Based 3D Model Retrieval Approach by Using Global and Local Features

CLN: Cross-Domain Learning Network for 2D Image-Based 3D Shape Retrieval

Semi-Heterogeneous Three-Way Joint Embedding Network for Sketch-Based Image Retrieval

A Unified Feature Representation and Learning Framework for 3D Shape

Deep Correlated Holistic Metric Learning for Sketch-Based 3D Shape Retrieval

3D VR Sketch Guided 3D Shape Prototyping and Exploration

Progressive Cross-Modal Semantic Network for Zero-Shot Sketch-Based Image Retrieval

Cross Task Modality Alignment Network for Sketch Face Recognition

Discriminative Sketch-Based 3d Model Retrieval Via Robust Shape Matching

Domain Disentangled Generative Adversarial Network for Zero-Shot Sketch-Based 3D Shape Retrieval.