Abstract:Automatic sketch colorization is a challenging task that aims to generate a color image from a sketch, primarily due to its inherently ill-posed nature. While many approaches have shown promising results, two significant challenges remain: limited color patterns and a wide range of artifacts such as color bleeding and semantic inconsistencies among relevant regions. These issues stem from the operation of traditional convolutional structures, which capture structural features in a pixel-wise manner, resulting in inadequate utilization of regional information within the sketch. Therefore, we propose the Region-Assisted Sketch Coloring (RASC) method, which introduces an intermediate representation called the 'Region Map' to explicitly characterize the regional information of the sketch. This Region Map is derived from the input sketch and is effectively formulated by our RASC architecture, enhancing the perception of region-wise features beyond the original pixel-wise features. Specifically, we start by employing the sketch encoder to extract hierarchical feature maps from the input sketches. Subsequently, we introduce a coarse-to-fine decoder comprising a series of Region-based Modulation (RM) blocks. This decoder modulates features that combine the modulation results of its previous block and the sketch features of the corresponding encoder block with our Region Formulation module. Each module explicitly formulates the sketch features in a region-wise manner. This accurately captures both the inner-region local style and inter-region global context dependency, resulting in various color patterns and fewer synthesis artifacts. Our experimental results show that our proposed method surpasses state-of-the-art methods in both synthetic and real sketch datasets.

Cross-Domain Learning for Reference-Based Sketch Colorization with Structural and Colorific Strategy.

Exemplar-Based Sketch Colorization with Cross-Domain Dense Semantic Correspondence

Language-based colorization of scene sketches

Self-driven Dual-path Learning for Reference-based Line Art Colorization under Limited Data

Improving reference-based image colorization for line arts via feature aggregation and contrastive learning

Dual Color Space Guided Sketch Colorization

Semi-supervised reference-based sketch extraction using a contrastive learning framework

Region Assisted Sketch Colorization

Learning Spatially Decoupled Color Representations for Facial Image Colorization

Cooperative Colorization: Exploring Latent Cross-Domain Priors for NIR Image Spectrum Translation

Exemplar-Based Image Colorization with A Learning Framework

Cross-Domain Adversarial Feature Learning For Sketch Re-Identification

Cross-domain representation learning by domain-migration generative adversarial network for sketch based image retrieval

Adversarial Open Domain Adaptation for Sketch-to-Photo Synthesis

Active Colorization for Cartoon Line Drawings

Deep Exemplar-based Colorization

Semantic-Sparse Colorization Network for Deep Exemplar-based Colorization

UGSC‐GAN: User‐guided sketch colorization with deep convolution generative adversarial networks

Single Color Sketch-Based Image Retrieval in HSV Color Space

MultiColor: Image Colorization by Learning from Multiple Color Spaces

L-C4: Language-Based Video Colorization for Creative and Consistent Color