Contour Counts: Restricting Deformation for Accurate Animation Interpolation

Lei Li,Xin Xu,Kui Jiang,Wei Liu,Zheng Wang

DOI: https://doi.org/10.1109/lsp.2024.3404138

2024-05-31

IEEE Signal Processing Letters

Abstract:Animated videos with low frame rates commonly degrade the visual experience with choppy motion. Recently, video frame interpolation has been growing rapidly which can increase frame rates. However, the sparse texture information and complex motion scenes of animated videos make the objects in the frames generated by existing video frame interpolation methods appear significantly deformation, causing distortion in the content of generated frames. To address this issue, the Restricting Deformation by Contour Network (RDC-Net) is proposed to repair and fill the content by optimizing object contours and leveraging context features for high-quality animation interpolation. Specifically, the RDC-Net was proposed to learn optical flow maps utilized to effectively capture the spatial shifts in the motion subject's contours across time intervals. Furthermore, contour information is used to refine the structure of the object in optical flow estimation and moderate the object deformation scale in the generated frame. In addition, the context information is explored to characterize the motion between adjacent frames via bidirectional optical flow learning, enabling the filling of distortion in the content of generated frames by feature-filling technology. Experiments on commonly used benchmarks show our state-of-the-art performance.

engineering, electrical & electronic