Abstract:Different people speak with diverse personalized speaking styles. Although existing one-shot talking head methods have made significant progress in lip sync, natural facial expressions, and stable head motions, they still cannot generate diverse speaking styles in the final talking head videos. To tackle this problem, we propose a one-shot style-controllable talking face generation framework. In a nutshell, we aim to attain a speaking style from an arbitrary reference speaking video and then drive the one-shot portrait to speak with the reference speaking style and another piece of audio. Specifically, we first develop a style encoder to extract dynamic facial motion patterns of a style reference video and then encode them into a style code. Afterward, we introduce a style-controllable decoder to synthesize stylized facial animations from the speech content and style code. In order to integrate the reference speaking style into generated videos, we design a style-aware adaptive transformer, which enables the encoded style code to adjust the weights of the feed-forward layers accordingly. Thanks to the style-aware adaptation mechanism, the reference speaking style can be better embedded into synthesized videos during decoding. Extensive experiments demonstrate that our method is capable of generating talking head videos with diverse speaking styles from only one portrait image and an audio clip while achieving authentic visual effects. Project Page: <a class="link-external link-https" href="https://github.com/FuxiVirtualHuman/styletalk" rel="external noopener nofollow">this https URL</a>.

A Stylized Method for Facial Cartoon Based on Feature Tracking and Rending

Expressive Facial Style Transfer for Personalized Memes Mimic

A real-time interactive system for facial makeup of Peking Opera

Feature-Based Automatic Portrait Generation System

Video Tracked Facial Expression Animation

Feature-based multi-style cartoon system

Creative cartoon face synthesis system for mobile entertainment

Video-Based Facial Animation with Detailed Appearance Texture

Generating Animatable 3D Cartoon Faces from Single Portraits

Performance-Driven Animation of Hand-Drawn Cartoon Faces

Video-driven state-aware facial animation

Synthesis, Style Editing, and Animation of 3D Cartoon Face

Animating Cartoon Faces by Multi-View Drawings

Performance-Driven Two-Dimensional Vector Expression Animation

Quick Customization of Particular Human Face and Muscle-Driven Expression Animation

Template Fitting Non Parametric Sampling Stroke Library Stroke Rendering Interactive Hair Contour Extraction Interactive Face Alignment

3Dtoonify: Creating Your High-Fidelity 3D Stylized Avatar Easily from 2D Portrait Images

Animating Portrait Line Drawings from a Single Face Photo and a Speech Signal

StyleTalk: One-shot Talking Head Generation with Controllable Speaking Styles

Cartoon-like Stylization of Video for Real-Time Applications.

FACTS: Facial Animation Creation using the Transfer of Styles