Abstract:Automatic sketch colorization is a challenging task that aims to generate a color image from a sketch, primarily due to its inherently ill-posed nature. While many approaches have shown promising results, two significant challenges remain: limited color patterns and a wide range of artifacts such as color bleeding and semantic inconsistencies among relevant regions. These issues stem from the operation of traditional convolutional structures, which capture structural features in a pixel-wise manner, resulting in inadequate utilization of regional information within the sketch. Therefore, we propose the Region-Assisted Sketch Coloring (RASC) method, which introduces an intermediate representation called the 'Region Map' to explicitly characterize the regional information of the sketch. This Region Map is derived from the input sketch and is effectively formulated by our RASC architecture, enhancing the perception of region-wise features beyond the original pixel-wise features. Specifically, we start by employing the sketch encoder to extract hierarchical feature maps from the input sketches. Subsequently, we introduce a coarse-to-fine decoder comprising a series of Region-based Modulation (RM) blocks. This decoder modulates features that combine the modulation results of its previous block and the sketch features of the corresponding encoder block with our Region Formulation module. Each module explicitly formulates the sketch features in a region-wise manner. This accurately captures both the inner-region local style and inter-region global context dependency, resulting in various color patterns and fewer synthesis artifacts. Our experimental results show that our proposed method surpasses state-of-the-art methods in both synthetic and real sketch datasets.

Colorsketch: A Drawing Assistant for Generating Color Sketches from Photos

Research and Implementation of the Sketching Design System

Language-based colorization of scene sketches

PencilArt: A Chromatic Penciling Style Generation Framework.

Sketching Feature-Based Modeling by Capturing Design Intention

Color Sketch Generation

An online sketching and gesture editing system for conceptual design

Automatic Image-Based Pencil Sketch Rendering

SweepCanvas: Sketch-based 3D Prototyping on an RGB-D Image

Sketch Generation with Drawing Process Guided by Vector Flow and Grayscale

Feature-preserving color pencil drawings from photographs

SketchDeco: Decorating B&W Sketches with Colour

Unsupervised Scene Sketch to Photo Synthesis

AI supported computer-generated pen-and-ink illustration

SketchMaker: Sketch Extraction and Reuse for Interactive Scene Sketch Composition

Region Assisted Sketch Colorization

PhotoSketch: Internet Image Montage

A Freehand Sketchy Graphic Input System: SketchGIS

Sketch3D: Style-Consistent Guidance for Sketch-to-3D Generation

CustomSketching: Sketch Concept Extraction for Sketch-based Image Synthesis and Editing

Sketch2Photo: Internet Image Montage