Joint Stroke Tracing and Correspondence for 2D Animation

Haoran Mo,Chengying Gao,Ruomei Wang

DOI: https://doi.org/10.1145/3649890

IF: 6.2

2024-02-29

ACM Transactions on Graphics

Abstract:To alleviate human labor in redrawing keyframes with ordered vector strokes for automatic inbetweening, we for the first time propose a joint stroke tracing and correspondence approach. Given consecutive raster keyframes along with a single vector image of the starting frame as a guidance, the approach generates vector drawings for the remaining keyframes while ensuring one-to-one stroke correspondence. Our framework trained on clean line drawings generalizes to rough sketches and the generated results can be imported into inbetweening systems to produce inbetween sequences. Hence, the method is compatible with standard 2D animation workflow. An adaptive spatial transformation module (ASTM) is introduced to handle non-rigid motions and stroke distortion. We collect a dataset for training, with 10k+ pairs of raster frames and their vector drawings with stroke correspondence. Comprehensive validations on real clean and rough animated frames manifest the effectiveness of our method and superiority to existing methods.

computer science, software engineering

What problem does this paper attempt to address?

### Problems Addressed by the Paper

This paper aims to address the issue of inbetweening in 2D animation production, specifically reducing the manual effort required to redraw keyframes with sequential vector strokes.

#### Main Contributions:
1. **Joint Stroke Tracking and Correspondence Framework**: This method introduces, for the first time, a joint stroke tracking and correspondence approach. Given a series of consecutive pixel keyframes and a manually tracked vector image of the initial frame as guidance, the method automatically generates vector drawings for the remaining keyframes, ensuring a one-to-one correspondence between strokes. This approach uses clean line drawing data during training but generalizes well to rough sketches.
2. **Adaptive Spatial Transformation Module (ASTM)**: A self-supervised learning-based spatial transformation module is introduced to handle large movements or stroke distortions in keyframes.
3. **Dataset**: A dataset containing over 10,000 pairs of pixel frames and their vector drawings with stroke correspondence annotations was collected for training.
4. **Validation Experiments**: Extensive validation was conducted using high-resolution real drawing images, demonstrating the effectiveness and superiority of the method.

### Solution Overview
1. **Stroke Representation**: Cubic Bézier curves are used to represent strokes at each step.
2. **Starting Point Matching Model**: This model handles the starting point matching process for each stroke chain, using the starting points in the reference frame as guidance to locate matching points in the target image.
3. **Stroke Tracking and Correspondence Model**: This model recursively generates vector strokes corresponding to the reference frame.
4. **Adaptive Spatial Transformation Module**: This module predicts spatial transformation parameters to align the content in local windows, reducing differences caused by deformation and thus improving the accuracy of subsequent predictions.

### Summary
This paper proposes a novel method that significantly reduces the manual workload in 2D animation production through automated stroke tracking and correspondence techniques, thereby enhancing production efficiency.