Abstract:The emergence of text-driven motion synthesis technique provides animators with great potential to create efficiently. However, in most cases, textual expressions only contain general and qualitative motion descriptions, while lack fine depiction and sufficient intensity, leading to the synthesized motions that either (a) semantically compliant but uncontrollable over specific pose details, or (b) even deviates from the provided descriptions, bringing animators with undesired cases. In this paper, we propose DiffKFC, a conditional diffusion model for text-driven motion synthesis with KeyFrames Collaborated, enabling realistic generation with collaborative and efficient dual-level control: coarse guidance at semantic level, with only few keyframes for direct and fine-grained depiction down to body posture level. Unlike existing inference-editing diffusion models that incorporate conditions without training, our conditional diffusion model is explicitly trained and can fully exploit correlations among texts, keyframes and the diffused target frames. To preserve the control capability of discrete and sparse keyframes, we customize dilated mask attention modules where only partial valid tokens participate in local-to-global attention, indicated by the dilated keyframe mask. Additionally, we develop a simple yet effective smoothness prior, which steers the generated frames towards seamless keyframe transitions at inference. Extensive experiments show that our model not only achieves state-of-the-art performance in terms of semantic fidelity, but more importantly, is able to satisfy animator requirements through fine-grained guidance without tedious labor.

Example-based automatic migration method for dynamic text special effect

Dyntypo: Example-Based Dynamic Text Effects Transfer

Image texture migration method based on structure guidance

Text Effects Transfer Via Distribution-Aware Texture Synthesis.

ff ects transfer via distribution-aware texture synthesis

Sketch Based Multi-source 3D Animation Transfer

A new texture morphing method for visual presentation of textile product

Artistic Text Style Transfer: an Overview of State-of-the-art Methods and Datasets [SP Forum].

Awesome Typography: Statistics-Based Text Effects Transfer.

Application of artificial intelligence-based style transfer algorithm in animation special effects design

Shape-Matching GAN++: Scale Controllable Dynamic Artistic Text Style Transfer

Intelligent Typography: Artistic Text Style Transfer for Complex Texture and Structure

Enhanced Fine-Grained Motion Diffusion for Text-Driven Human Motion Synthesis

Dynamic Typography: Bringing Text to Life via Video Diffusion Prior

A Texture Synthesis Method Based on Adaptive Patches

Image based high realistic visual pattern effect achieving method

FET-GAN: Font and Effect Transfer via K-shot Adaptive Instance Normalization

A Novel Method of Dynamic Textures Analysis and Synthesis.

Application of Style Transfer Algorithm in the Design of Animation Pattern Special Effect

Understanding Text-driven Motion Synthesis with Keyframe Collaboration via Diffusion Models

Make-It-Vivid: Dressing Your Animatable Biped Cartoon Characters from Text