Abstract:Being essential in animation creation, colorizing anime line drawings is usually a tedious and time-consuming manual task. Reference-based line drawing colorization provides an intuitive way to automatically colorize target line drawings using reference images. The prevailing approaches are based on generative adversarial networks (GANs), yet these methods still cannot generate high-quality results comparable to manually-colored ones. In this article, a new AnimeDiffusion approach is proposed via hybrid diffusions for the automatic colorization of anime face line drawings. This is the first attempt to utilize the diffusion model for reference-based colorization, which demands a high level of control over the image synthesis process. To do so, a hybrid end-to-end training strategy is designed, including phase 1 for training diffusion model with classifier-free guidance and phase 2 for efficiently updating color tone with a target reference colored image. The model learns denoising and structure-capturing ability in phase 1, and in phase 2, the model learns more accurate color information. Utilizing our hybrid training strategy, the network convergence speed is accelerated, and the colorization performance is improved. Our AnimeDiffusion generates colorization results with semantic correspondence and color consistency. In addition, the model has a certain generalization performance for line drawings of different line styles. To train and evaluate colorization methods, an anime face line drawing colorization benchmark dataset, containing 31,696 training data and 579 testing data, is introduced and shared. Extensive experiments and user studies have demonstrated that our proposed AnimeDiffusion outperforms state-of-the-art GAN-based methods and another diffusion-based model, both quantitatively and qualitatively.

Bridging the Gap: Sketch to Color Diffusion Model with Semantic Prompt Learning.

Language-based colorization of scene sketches

ColorizeDiffusion: Adjustable Sketch Colorization with Reference Image and Text

Multimodal Semantic-Aware Automatic Colorization with Diffusion Prior

DiffSketching: Sketch Control Image Synthesis with Diffusion Models

Self-driven Dual-path Learning for Reference-based Line Art Colorization under Limited Data

DiffColor: Toward High Fidelity Text-Guided Image Colorization with Diffusion Models

AnimeDiffusion: Anime Diffusion Colorization

Improving reference-based image colorization for line arts via feature aggregation and contrastive learning

Region Assisted Sketch Colorization

Draw Like an Artist: Complex Scene Generation with Diffusion Model via Composition, Painting, and Retouching

Improved Diffusion-based Image Colorization via Piggybacked Models

Diffusing Colors: Image Colorization with Text Guided Diffusion

Prompt-Free Diffusion: Taking "text" out of Text-to-Image Diffusion Models

AnimeDiffusion: Anime Face Line Drawing Colorization via Diffusion Models

SketchScene: Scene Sketch to Image Generation with Diffusion Models.

SketchFFusion: Sketch-guided image editing with diffusion model

Sketch-Guided Scene Image Generation

Inversion-by-Inversion: Exemplar-based Sketch-to-Photo Synthesis via Stochastic Differential Equations without Training

Exemplar-Based Sketch Colorization with Cross-Domain Dense Semantic Correspondence

Semantic-Sparse Colorization Network for Deep Exemplar-based Colorization