Abstract:Effectively and efficiently retrieving images from remote-sensing databases is a critical challenge in the realm of remote-sensing big data. Utilizing hand-drawn sketches as retrieval inputs offers intuitive and user-friendly advantages, yet the potential of multi-level feature integration from sketches remains underexplored, leading to suboptimal retrieval performance. To address this gap, our study introduces a novel zero-shot, sketch-based retrieval method for remote-sensing images, leveraging multi-level feature extraction, self-attention-guided tokenization and filtering, and cross-modality attention update. This approach employs only vision information and does not require semantic knowledge concerning the sketch and image. It starts by employing multi-level self-attention guided feature extraction to tokenize the query sketches, as well as self-attention feature extraction to tokenize the candidate images. It then employs cross-attention mechanisms to establish token correspondence between these two modalities, facilitating the computation of sketch-to-image similarity. Our method significantly outperforms existing sketch-based remote-sensing image retrieval techniques, as evidenced by tests on multiple datasets. Notably, it also exhibits robust zero-shot learning capabilities in handling unseen categories and strong domain adaptation capabilities in handling unseen novel remote-sensing data. The method's scalability can be further enhanced by the pre-calculation of retrieval tokens for all candidate images in a database. This research underscores the significant potential of multi-level, attention-guided tokenization in cross-modal remote-sensing image retrieval. For broader accessibility and research facilitation, we have made the code and dataset used in this study publicly available online.

A Preliminary Study On Extracting Objects In Sketches

SceneSketcher: Fine-Grained Image Retrieval with Scene Sketches

An on-line sketch recognition algorithm for composite shape

SceneSketcher-v2: Fine-Grained Scene-Level Sketch-Based Image Retrieval Using Adaptive GCNs

Sketch-Based Retrieval in Large-Scale Image Database Via Position-Aware Silhouette Matching.

Sketchformer++: A Hierarchical Transformer Architecture for Vector Sketch Representation

Component Segmentation of Sketches Used in 3D Model Retrieval

Free hand-drawn sketch segmentation

On-Line Sketch Recognition Using Direction Feature

Sketch-Specific Data Augmentation for Freehand Sketch Recognition

Why Does the Sketch Look So Vivid?

FS-COCO: Towards Understanding of Freehand Sketches of Common Objects in Context

Sketch-based Video Object Segmentation: Benchmark and Analysis

Sketch2Saliency: Learning to Detect Salient Objects from Human Drawings

Discovering Discriminative Patches for Free-Hand Sketch Analysis

Stroke-based semantic segmentation for scene-level free-hand sketches

3D Sketching for 3D Object Retrieval

Learning Dense Correspondences between Photos and Sketches

Research on Freehand Sketchy Graphics Recognition

Zero-Shot Sketch-Based Remote-Sensing Image Retrieval Based on Multi-Level and Attention-Guided Tokenization

Research progress of image retrieval based on hand-drawn sketches