Abstract:We propose a deep learning approach to free-hand sketch recognition that achieves state-of-the-art performance, significantly surpassing that of humans. Our superior performance is a result of modelling and exploiting the unique characteristics of free-hand sketches, i.e., consisting of an ordered set of strokes but lacking visual cues such as colour and texture, being highly iconic and abstract, and exhibiting extremely large appearance variations due to different levels of abstraction and deformation. Specifically, our deep neural network, termed Sketch-a-Net has the following novel components: (i) we propose a network architecture designed for sketch rather than natural photo statistics. (ii) Two novel data augmentation strategies are developed which exploit the unique sketch-domain properties to modify and synthesise sketch training data at multiple abstraction levels. Based on this idea we are able to both significantly increase the volume and diversity of sketches for training, and address the challenge of varying levels of sketching detail commonplace in free-hand sketches. (iii) We explore different network ensemble fusion strategies, including a re-purposed joint Bayesian scheme, to further improve recognition performance. We show that state-of-the-art deep networks specifically engineered for photos of natural objects fail to perform well on sketch recognition, regardless whether they are trained using photos or sketches. Furthermore, through visualising the learned filters, we offer useful insights in to where the superior performance of our network comes from.

S3Net:Graph Representational Network for Sketch Recognition

Sketch-R2CNN: an Attentive Network for Vector Sketch Recognition

Sketch-Snet: Deeper Subdivision Of Temporal Cues For Sketch Recognition

<i>Sketch-R2CNN</i>: An RNN-Rasterization-CNN Architecture for Vector Sketch Recognition

SceneSketcher-v2: Fine-Grained Scene-Level Sketch-Based Image Retrieval Using Adaptive GCNs

Sketchformer++: A Hierarchical Transformer Architecture for Vector Sketch Representation

SketchGNN: Semantic Sketch Segmentation with Graph Neural Networks

AI-Sketcher : A Deep Generative Model for Producing High-Quality Sketches.

Sketchpointnet: A Compact Network for Robust Sketch Recognition

Sketch-a-Net That Beats Humans

Sketch-a-Net: A Deep Neural Network That Beats Humans

SketchNet: Sketch Classification with Web Images

Sketchsegnet: A Rnn Model for Labeling Sketch Strokes.

Four‐branch Siamese Network Based on Sketch‐specific Data Augmentation for Sketch Recognition

SSR-GNNs: Stroke-based Sketch Representation with Graph Neural Networks

SketchSegNet+: An End-to-End Learning of RNN for Multi-Class Sketch Semantic Segmentation

Research on Recognition Effect of DSCN Network Structure in Hand-Drawn Sketch

SketchGCN: Semantic Sketch Segmentation with Graph Convolutional Networks

A hierarchical residual network with compact triplet-center loss for sketch recognition

SFSegNet: Parse Freehand Sketches using Deep Fully Convolutional Networks

Enhance Sketch Recognition's Explainability via Semantic Component-Level Parsing