Abstract:Automatic sketch colorization is a challenging task that aims to generate a color image from a sketch, primarily due to its inherently ill-posed nature. While many approaches have shown promising results, two significant challenges remain: limited color patterns and a wide range of artifacts such as color bleeding and semantic inconsistencies among relevant regions. These issues stem from the operation of traditional convolutional structures, which capture structural features in a pixel-wise manner, resulting in inadequate utilization of regional information within the sketch. Therefore, we propose the Region-Assisted Sketch Coloring (RASC) method, which introduces an intermediate representation called the 'Region Map' to explicitly characterize the regional information of the sketch. This Region Map is derived from the input sketch and is effectively formulated by our RASC architecture, enhancing the perception of region-wise features beyond the original pixel-wise features. Specifically, we start by employing the sketch encoder to extract hierarchical feature maps from the input sketches. Subsequently, we introduce a coarse-to-fine decoder comprising a series of Region-based Modulation (RM) blocks. This decoder modulates features that combine the modulation results of its previous block and the sketch features of the corresponding encoder block with our Region Formulation module. Each module explicitly formulates the sketch features in a region-wise manner. This accurately captures both the inner-region local style and inter-region global context dependency, resulting in various color patterns and fewer synthesis artifacts. Our experimental results show that our proposed method surpasses state-of-the-art methods in both synthetic and real sketch datasets.

Color Sketch Generation

Language-based colorization of scene sketches

PencilArt: A Chromatic Penciling Style Generation Framework.

Colorsketch: A Drawing Assistant for Generating Color Sketches from Photos

Region Assisted Sketch Colorization

Mural Sketch Generation via Style-aware Convolutional Neural Network.

Automatic Image-Based Pencil Sketch Rendering

Sketch Generation with Drawing Process Guided by Vector Flow and Grayscale

Sketch3D: Style-Consistent Guidance for Sketch-to-3D Generation

SketchDeco: Decorating B&W Sketches with Colour

AI supported computer-generated pen-and-ink illustration

Hand-drawn Grayscale Image Colorful Colorization Based on Natural Image

Feature-preserving color pencil drawings from photographs

Dual Color Space Guided Sketch Colorization

Example-Based Facial Sketch Generation With Non-Parametric Sampling

LUCSS: Language-based User-customized Colourization of Scene Sketches

UGSC‐GAN: User‐guided sketch colorization with deep convolution generative adversarial networks

Sketch-R2CNN: an Attentive Network for Vector Sketch Recognition

Picture that Sketch: Photorealistic Image Generation from Abstract Sketches

An Example-Based Facial Sketch Generation System

Inversion-by-Inversion: Exemplar-based Sketch-to-Photo Synthesis via Stochastic Differential Equations without Training