Abstract:Freehand sketching is a dynamic process where points are sequentially sampled and grouped as strokes for sketch acquisition on electronic devices. To recognize a sketched object, most existing methods discard such important temporal ordering and grouping information from human and simply rasterize sketches into binary images for classification. In this paper, we propose a novel single-branch attentive network architecture RNN-Rasterization-CNN (Sketch-R2CNN for short) to fully leverage the dynamics in sketches for recognition. Sketch-R2CNN takes as input only a vector sketch with grouped sequences of points, and uses an RNN for stroke attention estimation in the vector space and a CNN for 2D feature extraction in the pixel space respectively. To bridge the gap between these two spaces in neural networks, we propose a neural line rasterization module to convert the vector sketch along with the attention estimated by RNN into a bitmap image, which is subsequently consumed by CNN. The neural line rasterization module is designed in a differentiable way to yield a unified pipeline for end-to-end learning. We perform experiments on existing large-scale sketch recognition benchmarks and show that by exploiting the sketch dynamics with the attention mechanism, our method is more robust and achieves better performance than the state-of-the-art methods.

Sketch2Tag: automatic hand-drawn sketch recognition.

An online sketching and gesture editing system for conceptual design

Pen-based gesture design and recognition for sketch editing

Research and Implementation of the Sketching Design System

Sketch-R2CNN: an Attentive Network for Vector Sketch Recognition

SceneSketcher: Fine-Grained Image Retrieval with Scene Sketches

Sketchformer++: A Hierarchical Transformer Architecture for Vector Sketch Representation

<i>Sketch-R2CNN</i>: An RNN-Rasterization-CNN Architecture for Vector Sketch Recognition

An on-line sketch recognition algorithm for composite shape

Sketch-Based Retrieval in Large-Scale Image Database Via Position-Aware Silhouette Matching.

On-Line Sketch Recognition Using Direction Feature

A Structural Method for Online Sketched Symbol Recognition.

Iterative Collection Annotation for Sketch Recognition

Sketch Recognition With Natural Correction And Editing

An Efficient Primitive-Based Method to Recognize Online Sketched Symbols with Autocompletion

Online Sketch Recognition Using Contexts

Enhance Sketch Recognition's Explainability via Semantic Component-Level Parsing

Two-Dimensional Sketch Recognition with Design Intent Capture

Iterative Samples Labeling for Sketch Recognition.

Sketch Input Method Editor: A Comprehensive Dataset and Methodology for Systematic Input Recognition

Sketch2Photo: Internet Image Montage