Abstract:We study the problem of fine-grained sketch-based image retrieval. By performing instance-level (rather than category-level) retrieval, it embodies a timely and practical application, particularly with the ubiquitous availability of touchscreens. Three factors contribute to the challenging nature of the problem: (i) free-hand sketches are inherently abstract and iconic, making visual comparisons with photos more difficult, (ii) sketches and photos are in two different visual domains, i.e. black and white lines vs. color pixels, and (iii) fine-grained distinctions are especially challenging when executed across domain and abstraction-level. To address this, we propose to detect visual attributes at part-level, in order to build a new representation that not only captures fine-grained characteristics but also traverses across visual domains. More specifically, (i) we propose a dataset with 304 photos and 912 sketches, where each sketch and photo is annotated with its semantic parts and associated part-level attributes, and with the help of this dataset, we investigate (ii) how strongly-supervised deformable part-based models can be learned that subsequently enable automatic detection of part-level attributes, and (iii) a novel matching framework that synergistically integrates low-level features, mid-level geometric structure and high-level semantic attributes to boost retrieval performance. Extensive experiments conducted on our new dataset demonstrate value of the proposed method.

Rst-Invariant Sketch Retrieval Based On Circular Description

SceneSketcher: Fine-Grained Image Retrieval with Scene Sketches

SceneSketcher-v2: Fine-Grained Scene-Level Sketch-Based Image Retrieval Using Adaptive GCNs

Sketch-Based Image Retrieval by Salient Contour Reinforcement

Sketch-Based Retrieval in Large-Scale Image Database Via Position-Aware Silhouette Matching.

RISAS: A Novel Rotation, Illumination, Scale Invariant Appearance and Shape Feature.

Image Retrieval Based on RST Invariant Features Extracted from Scale Invariant Keypoints

Image Retrieval Based On Rst-Invariant Features

A Novel Visual-Region-Descriptor-based Approach to Sketch-based Image Retrieval

Sketch-based image retrieval using contour segments

Conventional SBIR Fine-grained SBIR Conventional Methods Our Method

Sketch4Image: a novel framework for sketch-based image retrieval based on product quantization with coding residuals

Sketch-Based Image Retrieval Using Sketch Tokens

Sketch Less for More: On-the-Fly Fine-Grained Sketch Based Image Retrieval

Sketch-Based Image Retrieval via Adaptive Weighting

Sketch-based image retrieval method using local geometry moment invariant

Single Color Sketch-Based Image Retrieval in HSV Color Space

Zero-Shot Sketch-Based Image Retrieval with Structure-aware Asymmetric Disentanglement

Sketch-based Image Retrieval Using Hierarchical Partial Matching

M-SBIR: An Improved Sketch-Based Image Retrieval Method Using Visual Word Mapping

M-SBIR: An Improved Sketch-Based Image Retrieval Method Using Visual Word Mapping.