Abstract:Sketch colorization is usually expensive and time-consuming for artists, and automating this process can have many pragmatic applications in the animation, comic book, and video game industry. However, automatic image colorization faces many challenges, because sketches not only lack texture information but also potentially entail complicated objects that require acute coloring. These difficulties usually result in incorrect color assignments that can ruin the aesthetic appeal of the final output. In this paper, we present a novel component-based matching framework that combines deep learned features and quadratic programming with a new cost function to solve this colorization problem. The proposed framework inputs a character's sketches as well as a colored image in the same cut of a movie, and outputs a high-quality sequence of colorized frames based on the color assignment in the reference colored image. To carry out this colorization task, we first utilize a pretrained ResNet-34 model to extract elementary components' features to match certain pairs of components (one component from the sketch and one from reference). Next, a graph representation is constructed in order to process and match the remaining components that could not be done in the first step. Since the first step has reduced the number of components to be matched by the graph, we can solve this graph problem in a short computing time even when there are hundreds of different components present in each sketch. We demonstrate the effectiveness of the proposed solution by conducting comprehensive experiments and producing aesthetically pleasing results. To the best of our knowledge, our framework is the first work that combines deep learning extraction and graph representation to colorize anime sketches and achieves a high pixel-level accuracy at a reasonable time cost.

Outline Colorization through Tandem Adversarial Networks

PencilArt: A Chromatic Penciling Style Generation Framework.

Language-based colorization of scene sketches

Creative and Diverse Artwork Generation Using Adversarial Networks

Dual Color Space Guided Sketch Colorization

Self-driven Dual-path Learning for Reference-based Line Art Colorization under Limited Data

Adversarial Colorization Of Icons Based On Structure And Color Conditions

Coloring with Words: Guiding Image Colorization Through Text-based Palette Generation

Line Art Colorization of Fakemon using Generative Adversarial Neural Networks

Automatic Colorization of Anime Style Illustrations Using a Two-Stage Generator

Improving reference-based image colorization for line arts via feature aggregation and contrastive learning

Deep Line Art Video Colorization with a Few References

Active Colorization for Cartoon Line Drawings

Anime Sketch Colorization by Component-based Matching using Deep Appearance Features and Graph Representation

Line Art Correlation Matching Feature Transfer Network for Automatic Animation Colorization

UGSC‐GAN: User‐guided sketch colorization with deep convolution generative adversarial networks

Broad Colorization

User-Guided Anime Line Art Colorization with Spatially-adaptive Normalization

Exemplar-Based Sketch Colorization with Cross-Domain Dense Semantic Correspondence

Animation line art colorization based on the optical flow method

Paint Bucket Colorization Using Anime Character Color Design Sheets