Abstract:Freehand sketching is a dynamic process where points are sequentially sampled and grouped as strokes for sketch acquisition on electronic devices. To recognize a sketched object, most existing methods discard such important temporal ordering and grouping information from human and simply rasterize sketches into binary images for classification. In this paper, we propose a novel single-branch attentive network architecture RNN-Rasterization-CNN (Sketch-R2CNN for short) to fully leverage the dynamics in sketches for recognition. Sketch-R2CNN takes as input only a vector sketch with grouped sequences of points, and uses an RNN for stroke attention estimation in the vector space and a CNN for 2D feature extraction in the pixel space respectively. To bridge the gap between these two spaces in neural networks, we propose a neural line rasterization module to convert the vector sketch along with the attention estimated by RNN into a bitmap image, which is subsequently consumed by CNN. The neural line rasterization module is designed in a differentiable way to yield a unified pipeline for end-to-end learning. We perform experiments on existing large-scale sketch recognition benchmarks and show that by exploiting the sketch dynamics with the attention mechanism, our method is more robust and achieves better performance than the state-of-the-art methods.

An Efficient Primitive-Based Method to Recognize Online Sketched Symbols with Autocompletion

A Structural Method for Online Sketched Symbol Recognition.

An on-line sketch recognition algorithm for composite shape

An online sketching and gesture editing system for conceptual design

On-line hand-drawn electric circuit diagram recognition using 2D dynamic programming

On-Line Sketch Recognition Using Direction Feature

Sketch-R2CNN: an Attentive Network for Vector Sketch Recognition

Online Diagramming Recognition Based on Automatic Stroke Parsing and Bayesian Classifier

Online Sketch Recognition Using Contexts

On-line Handwritten Flowchart Recognition Based on Logical Structure and Graph Grammar

Online Composite Sketchy Shape Recognition Using Dynamic Programming

Directional Blurred Shape Model Descriptor for Sketched Symbol Recognition

An Online Composite Graphics Recognition Approach Based on Matching of Spatial Relation Graphs.

An Online Multi-Stroke Sketch Recognition Method Integrated with Stroke Segmentation

Research on Freehand Sketchy Graphics Recognition

Algorithm for Recognizing Symbols from Vector Engineering Drawings Based on a Two-Layer Structure

Hand-drawn Electric Circuit Diagram Understanding Using 2D Dynamic programming

Offline Sketch Parsing Via Shapeness Estimation

A Multilevel Approach For Recognizing Characters In Engineering Drawings

Research of the Structural-Learning-based Symbol Recognition Mechanism for Engineering Drawings

A SRG-based Online Composite Graphic Recognition Strategy for Sketch-Based User Interface