Abstract:Sketch is a well-researched topic in the vision community by now. Sketch semantic segmentation in particular, serves as a fundamental step towards finer-level sketch interpretation. Recent works use various means of extracting discriminative features from sketches and have achieved considerable improvements on segmentation accuracy. Common approaches for this include attending to the sketch-image as a whole, its stroke-level representation or the sequence information embedded in it. However, they mostly focus on only a part of such multi-facet information. In this paper, we for the first time demonstrate that there is complementary information to be explored across all these three facets of sketch data, and that segmentation performance consequently benefits as a result of such exploration of sketch-specific information. Specifically, we propose the Sketch-Segformer, a transformer-based framework for sketch semantic segmentation that inherently treats sketches as stroke sequences other than pixel-maps. In particular, Sketch-Segformer introduces two types of self-attention modules having similar structures that work with different receptive fields (i.e., whole sketch or individual stroke). The order embedding is then further synergized with spatial embeddings learned from the entire sketch as well as localized stroke-level information. Extensive experiments show that our sketch-specific design is not only able to obtain state-of-the-art performance on traditional figurative sketches (such as SPG, SketchSeg-150K datasets), but also performs well on creative sketches that do not conform to conventional object semantics (CreativeSketch dataset) thanks for our usage of multi-facet sketch information. Ablation studies, visualizations, and invariance tests further justifies our design choice and the effectiveness of Sketch-Segformer. Codes are available at https://github.com/PRIS-CV/Sketch-SF.

Sketchformer++: A Hierarchical Transformer Architecture for Vector Sketch Representation

Sketch-R2CNN: an Attentive Network for Vector Sketch Recognition

<i>Sketch-R2CNN</i>: An RNN-Rasterization-CNN Architecture for Vector Sketch Recognition

Sketch-Segformer: Transformer-Based Segmentation for Figurative and Creative Sketches.

General Virtual Sketching Framework for Vector Line Art

AI-Sketcher : A Deep Generative Model for Producing High-Quality Sketches.

A Sketch-Transformer Network for Face Photo-Sketch Synthesis

Scene Sketch Semantic Segmentation with Hierarchical Transformer.

Sketch-a-Net That Beats Humans

A hierarchical residual network with compact triplet-center loss for sketch recognition

Adapting Hierarchical Transformer for Scene-Level Sketch-Based Image Retrieval.

SketchAA: Abstract Representation for Abstract Sketches

Coupling Deep Textural and Shape Features for Sketch Recognition.

SketchHealer: A Graph-to-Sequence Network for Recreating Partial Human Sketches.

Sketch Recognition with Deep Visual-Sequential Fusion Model

On Learning Semantic Representations for Million-Scale Free-Hand Sketches

Sketch-Snet: Deeper Subdivision Of Temporal Cues For Sketch Recognition

Sketch-a-Net: A Deep Neural Network That Beats Humans

Reconstructing 3D Model from Single-View Sketch with Deep Neural Network.

Sketch-R2CNN : An RNN-Rasterization-CNN Architecture for Vector Sketch Recognition

BézierSketch: A generative model for scalable vector sketches