Paint Bucket Colorization Using Anime Character Color Design Sheets

Yuekun Dai,Qinyue Li,Shangchen Zhou,Yihang Luo,Chongyi Li,Chen Change Loy

2024-10-25

Abstract:Line art colorization plays a crucial role in hand-drawn animation production, where digital artists manually colorize segments using a paint bucket tool, guided by RGB values from character color design sheets. This process, often called paint bucket colorization, involves two main tasks: keyframe colorization, where colors are applied according to the character's color design sheet, and consecutive frame colorization, where these colors are replicated across adjacent frames. Current automated colorization methods primarily focus on reference-based and segment-matching approaches. However, reference-based methods often fail to accurately assign specific colors to each region, while matching-based methods are limited to consecutive frame colorization and struggle with issues like significant deformation and occlusion. In this work, we introduce inclusion matching, which allows the network to understand the inclusion relationships between segments, rather than relying solely on direct visual correspondences. By integrating this approach with segment parsing and color warping modules, our inclusion matching pipeline significantly improves performance in both keyframe colorization and consecutive frame colorization. To support our network's training, we have developed a unique dataset named PaintBucket-Character, which includes rendered line arts alongside their colorized versions and shading annotations for various 3D characters. To replicate industry animation data formats, we also created color design sheets for each character, with semantic information for each color and standard pose reference images. Experiments highlight the superiority of our method, demonstrating accurate and consistent colorization across both our proposed benchmarks and hand-drawn animations.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to color line drawings automatically and accurately during the animation production process. Specifically, the paper focuses on the "paint bucket coloring" technique based on the character color design sheet, which involves two main tasks:

1. **Keyframe Colorization**: Color the keyframes according to the character's color design sheet. This requires precisely assigning the colors in the design sheet to each area in the line drawing.
2. **Consecutive Frame Colorization**: Copy the colors of the keyframes to adjacent frames to achieve automatic coloring of the entire sequence.

Current automated coloring methods are mainly divided into two categories: reference image coloring and patch matching. However, these methods have some limitations:
- **Reference Image Coloring**: It is unable to accurately assign the predefined colors in the design sheet to each area.
- **Patch Matching**: It performs poorly when dealing with occlusions or scenes with large - scale movements because these situations will disrupt the similarity between patches.

To solve these problems, the paper introduces the concept of "Inclusion Matching", allowing the network to understand the inclusion relationships between patches, rather than just direct visual correspondence. By combining the patch analysis module and the color deformation module, the method proposed in the paper has achieved significant performance improvements in both keyframe coloring and consecutive frame coloring.

### Main Contributions
1. **Introducing the Concept of "Inclusion Matching"**: By calculating the probability that each patch in the target frame is included in a specific color in the reference frame, the need for direct patch correspondence is avoided.
2. **Proposing a Coloring Pipeline Based on the Color Design Sheet**: Including keyframe coloring and consecutive frame coloring, which can be seamlessly integrated into the existing animation production process.
3. **Constructing a New Dataset**: Developed a dataset named PaintBucket - Character, which contains rendered line drawings, their colored versions, and shadow annotations, to support network training and evaluation.
4. **Detailed Data Generation Process**: Described how to generate a high - quality dataset containing color design sheets and shadow annotations.
5. **Experimental Verification**: Demonstrated the effectiveness of the method through extensive experiments, including support for different input formats and comparison with other methods.

### Dataset Features
- **Flat Color Style**: Conforms to the standards of animation production.
- **Accurate Index Labels and Clear Line Drawings**: Define each color patch.
- **Color Design Sheet**: Contains semantic categories.
- **Shadow Annotations**: Include highlight and shadow areas, helping to distinguish confusing areas.

Through these innovations, the paper provides a more accurate and consistent automatic coloring method, which is suitable for the production process of hand - drawn animations.