Abstract:Advances in deep neural networks have considerably improved the art of animating a still image without operating in 3D domain. Whereas, prior arts can only animate small images (typically no larger than 512x512) due to memory limitations, difficulty of training and lack of high-resolution (HD) training datasets, which significantly reduce their potential for applications in movie production and interactive systems. Motivated by the idea that HD images can be generated by adding high-frequency residuals to low-resolution results produced by a neural network, we propose a novel framework known as Animating Through Warping (ATW) to enable efficient animation of HD images. Specifically, the proposed framework consists of two modules, a novel two-stage neural-network generator and a novel post-processing module known as Animating Through Warping (ATW). It only requires the generator to be trained on small images and can do inference on an image of any size. During inference, an HD input image is decomposed into a low-resolution component(128x128) and its corresponding high-frequency residuals. The generator predicts the low-resolution result as well as the motion field that warps the input face to the desired status (e.g., expressions categories or action units [11]). Finally, the ResWarp module warps the residuals based on the motion field and adding the warped residuals to generates the final HD results from the naively up-sampled low-resolution results. Experiments show the effectiveness and efficiency of our method in generating high-resolution animations. Our proposed framework successfully animates a 4K facial image, which has never been achieved by prior neural models. In addition, our method generally guarantee the temporal coherency of the generated animations. Source codes will be made publicly available.

Warp-Guided GANs for Single-Photo Facial Animation

Realistic Dynamic Facial Textures from a Single Image Using GANs

Generating Animatable 3D Cartoon Faces from Single Portraits

Animating Portrait Line Drawings from a Single Face Photo and a Speech Signal

Towards Realistic Face Photo-Sketch Synthesis via Composition-Aided GANs

Toward Realistic Face Photo–Sketch Synthesis via Composition-Aided GANs

AniPortraitGAN: Animatable 3D Portrait Generation from 2D Image Collections

Audio-driven Talking Face Video Generation with Natural Head Pose

Animating Through Warping An Efficient Method for High-Quality Facial Expression Animation

Two Birds with One Stone: Transforming and Generating Facial Images with Iterative GAN

MyPortrait: Morphable Prior-Guided Personalized Portrait Generation

Mask-Guided Portrait Editing With Conditional GANs

StyleAvatar: Real-time Photo-realistic Portrait Avatar from a Single Video

G3FA: Geometry-guided GAN for Face Animation

Generative Adversarial Talking Head: Bringing Portraits to Life with a Weakly Supervised Neural Network

WarpGAN: Automatic Caricature Generation

Towards Realistic Landmark-Guided Facial Video Inpainting Based on GANs

Hallo3: Highly Dynamic and Realistic Portrait Image Animation with Diffusion Transformer Networks

Composition-Aided Face Photo-Sketch Synthesis.

Controllable Facial Caricaturization with Localized Deformation and Personalized Semantic Attentions

PuppeteerGAN: Arbitrary Portrait Animation With Semantic-Aware Appearance Transformation