Abstract:State-of-the-art methods on sketch classification and retrieval are based on deep convolutional neural network to learn representations. Although deep neural networks have the ability to model images with hierarchical representations by convolution kernels, they can not automatically extract the structural representations of object categories in a human-perceptible way. Furthermore, sketch images usually have large scale visual variations caused by the styles of drawing or viewpoints, which make it difficult to develop generalized representations using the fixed computational mode of convolutional kernel. In this paper, our aim is to address the problem of fixed computational mode in feature extraction process without extra supervision. We propose a novel architecture to dynamically discover the object landmarks and learn the discriminative structural representations. Our model is composed of two components: a representative landmark discovering module that localizes the key points on the object, and a category-aware representation learning module that develops the category-specific features. Specifically, we develop a structure-aware offset layer to dynamically localize the representative landmarks, which is optimized based on the category labels without extra supervision. After that, a diversity branch is introduced to extract the global discriminative features for each category. Finally, we employ a multi-task loss function to develop an end-to-end trainable architecture. At testing time, we fuse all the predictions with different number of landmarks to achieve the final results. Through extensive experiments, we compare our model with several state-of-the-art methods on two challenging datasets TU-Berlin and Sketchy for sketch classification and retrieval, and the experimental results demonstrate the effectiveness of our proposed model.

DCR: Disentangled Component Representation for Sketch Generation.

Sketchformer++: A Hierarchical Transformer Architecture for Vector Sketch Representation

Model-driven sketch reconstruction with structure-oriented retrieval

On Learning Semantic Representations for Large-Scale Abstract Sketches

Component segmentation of engineering drawings using Graph Convolutional Networks

Sketch Recognition with Deep Visual-Sequential Fusion Model

Enhance Sketch Recognition's Explainability via Semantic Component-Level Parsing

SketchTrans: Disentangled Prototype Learning With Transformer for Sketch-Photo Recognition

Learning Deep Sketch Abstraction

Deep3DSketch+: Rapid 3D Modeling from Single Free-hand Sketches

Enhancing Sketch-Based Image Retrieval Via Deep Discriminative Representation.

Sketch-pix2seq: a Model to Generate Sketches of Multiple Categories

Deep3DSketch: 3D modeling from Free-hand Sketches with View- and Structural-Aware Adversarial Training

Disentangled Representations in Neural Models

Open Vocabulary Semantic Scene Sketch Understanding

Component-based Sketching for Deep ReLU Nets

SketchDesc: Learning Local Sketch Descriptors for Multi-View Correspondence

Sketch-BERT: Learning Sketch Bidirectional Encoder Representation from Transformers by Self-supervised Learning of Sketch Gestalt

End-to-End Photo-Sketch Generation via Fully Convolutional Representation Learning

Learning Structural Representations via Dynamic Object Landmarks Discovery for Sketch Recognition and Retrieval

CHIP: Channel-wise Disentangled Interpretation of Deep Convolutional Neural Networks