Abstract:Designing stylized cinemagraphs is challenging due to the difficulty in customizing complex and expressive flow motions. To achieve intuitive and detailed control of the generated cinemagraphs, freehand sketches can provide a better solution to convey personalized design requirements than only text inputs. In this paper, we propose Sketch2Cinemagraph, a sketch-guided framework that enables the conditional generation of stylized cinemagraphs from freehand sketches. Sketch2Cinemagraph adopts text prompts for initial content generation and provides hand-drawn sketch controls for both spatial and motion cues. The latent diffusion model is adopted to generate target stylized landscape images along with realistic versions. Then, a pre-trained object detection model is utilized to segment and obtain masks for the flow regions. We proposed a novel latent motion diffusion model to estimate the motion field in the fluid regions of the generated landscape images. The input motion sketches serve as the conditions to control the generated vector fields in the masked fluid regions with the prompt. To synthesize the cinemagraph frames, the pixels within fluid regions are subsequently warped to the target locations for each timestep using a frame generator. The results verified that Sketch2Cinemagraph can generate high-fidelity and aesthetically appealing stylized cinemagraphs with continuous temporal flow from intuitive sketch inputs. We showcase the advantages of Sketch2Cinemagraph through quantitative comparisons against the state-of-the-art generation approaches.

Hand-Drawn Cloth Animation with Kinetic Simulation

Cartoon Character Animation from Multi-view Hand-Drawings

Performance-Driven Animation of Hand-Drawn Cartoon Faces

Computer Generation of Water Animation with the Style of Paper-Cuts

D-Cloth: Skinning-based Cloth Dynamic Prediction with a Three-stage Network

Sketch Based Multi-source 3D Animation Transfer

Synthesis of Cartoon Fire Based on Hand-Drawn Samples

Physically-based Real-Time Animation of Draped Cloth

Hand-drawn animation with self-shaped canvas

Animating Cartoon Faces by Multi-View Drawings

Data-Driven Motion Exaggeration

SimulCap : Single-View Human Performance Capture With Cloth Simulation

Animating Water In Chinese Painting

GPU-based Real-time Cloth Simulation for Virtual Try-on.

Sketch-Guided Motion Diffusion for Stylized Cinemagraph Synthesis

Efficient and Simple Cloth Animation

Hand Tracking And Animation

Motion-Inspired Real-Time Garment Synthesis with Temporal-Consistency

Animating Water in Chinese Painting Using Autoregressive Model

A hybrid approach to animating the murals with Dunhuang style

Non‐linear Rough 2D Animation using Transient Embeddings