Abstract:We implement high-quality sketch colorization using two-stage conditional generative adversarial network (GAN) training based on different intermediate features. The intermediate features used in autonomous colorization are the grayscale parsing and interval pixel-level color parsing. The autonomous colorization based on grayscale parsing feature can learn the spacial topology of pixels in the first stage to guide the colorization in the second stage. The autonomous colorization based on pixel-level color parsing feature can learn the color information of few feature points in the first stage to guide the colorization of all pixels in the second stage. Additionally, we use the intermediate feature of sampling points as constraint and achieve the color reconstruction using Laplacian mesh editing as a special second stage. Furthermore, the interactive colorization uses the superpixel color parsing as the intermediate feature. Specifically, we use the simple linear iterative cluster (SLIC) to obtain a palette that maintains the edges in the first stage to guide the colorization in the second stage. As for evaluation metrics, we propose a color-coded local binary pattern (CCLBP) score based on color distances from the first-order 8 pixels to the central pixel, to measure the degrees of color blurring and mess. We also propose a light-sensitivity (LS) score based on the reversed grayscale map, to measure the degrees of auto painting and overfitting of the color hint. According to the L1 distances between the original and generated color images based on these scores, compared with state-of-the-art methods including one stage approaches such as pix2pix and PaintsChainer and two-stage approaches such as Style2Paints and DeepColor, our model can achieve the highest-quality autonomous colorization. Moreover, compared with pix2pix, PaintsChainer and Style2Paints with color hints, according to the proposed objective evaluation as well as the user visual study, our model can achieve the highest-quality interactive colorization as well.

Two-Stage Sketch Colorization with Color Parsing

Language-based colorization of scene sketches

Dual Color Space Guided Sketch Colorization

UGSC‐GAN: User‐guided sketch colorization with deep convolution generative adversarial networks

Automatic Sketch Colorization with Tandem Conditional Adversarial Networks.

Self-driven Dual-path Learning for Reference-based Line Art Colorization under Limited Data

Adversarial Interactive Cartoon Sketch Colourization with Texture Constraint and Auxiliary Auto-Encoder.

User-Guided Deep Anime Line Art Colorization with Conditional Adversarial Networks.

Dual Stage Augmented Colorful Texture Synthesis from Hand Sketch

Automatic Colorization of Anime Style Illustrations Using a Two-Stage Generator

CoPaint: Guiding Sketch Painting with Consistent Color and Coherent Generative Adversarial Networks

Color Hint-guided Ink Wash Painting Colorization with Ink Style Prediction Mechanism

Auto-painter: Cartoon Image Generation from Sketch by Using Conditional Wasserstein Generative Adversarial Networks

User Tailored Colorization Using Automatic Scribbles and Hierarchical Features

Double-Channel Guided Generative Adversarial Network for Image Colorization

Region Assisted Sketch Colorization

Automatic Color Sketch Generation Using Deep Style Transfer

Interactive Grayscale Image Colorization with Generative Adversarial Networks

Staged Sketch-to-Image Synthesis via Semi-supervised Generative Adversarial Networks

GGD-GAN: Gradient-Guided Dual-Branch Adversarial Networks for Relic Sketch Generation

Satellite Remote Sensing Grayscale Image Colorization Based on Denoising Generative Adversarial Network