Abstract:In this paper, we propose Occlusion-Aware Warping GAN (OAW-GAN), a unified Human Video Synthesis (HVS) framework that can uniformly tackle human video motion transfer, attribute editing, as well as inpainting. This is the first work to our knowledge that can handle all these tasks within a one-time trained model. Although existing GAN-based HVS methods have achieved great success, they either can’t preserve appearance details due to the loss of spatial consistency between the synthesized target frames and the input source images, or generate incoherent video results due to the loss of temporal consistency among frames. Besides, most of them lack the ability to create new contents while keeping existing ones, failing especially when some regions in the target are invisible in the source due to self-occlusion. To address these limitations, we first introduce Coarse-to-Fine Flow Warping Network (C2F-FWN) to estimate spatial-temporal consistent transformation between source and target, as well as occlusion mask indicating which parts in the target are invisible in the source. Then, the flow and the mask are scaled and fed into the pyramidal stages of our OAW-GAN, guiding Occlusion-Aware Synthesis (OAS) that can be abstracted into visible part re-utilization and invisible part inpainting at the feature level, which effectively alleviates the self-occlusion problem. Extensive experiments conducted on both human video (i.e., iPER, SoloDance)Keywords are desired. please provide if necessary. and image (i.e., DeepFashion) datasets demonstrate the superiority of our approach to existing state-of-the-arts. We also show that, besides motion transfer task that previous works concern, our framework can further achieve attribute editing and texture inpainting, which paves the way towards unified HVS.

Part-Preserving Pose Manipulation For Person Image Synthesis

OAW-GAN: Occlusion-Aware Warping GAN for Unified Human Video Synthesis

Soft-Gated Warping-GAN for Pose-Guided Person Image Synthesis

Pose- and Attribute-consistent Person Image Synthesis

PoT-GAN: Pose Transform GAN for Person Image Synthesis.

Pose with Style: Detail-Preserving Pose-Guided Image Synthesis with Conditional StyleGAN

PISE-V: person image and video synthesis with decoupled GAN

Mutually Activated Residual Linear Modeling GAN for Pose-Guided Person Image Generation

ACGAN: Attribute Controllable Person Image Synthesis GAN for Pose Transfer

HumanGAN: A Generative Model of Humans Images

Precise Correspondence Enhanced GAN for Person Image Generation

Attentional pixel-wise deformation for pose-based human image generation

PoNA: Pose-Guided Non-Local Attention for Human Pose Transfer

Pose Guided Human Video Generation

Progressive and Aligned Pose Attention Transfer for Person Image Generation

Pose Generator ( G ) : Head : R arm : L arm : Chest : R leg : L leg Plausible Pose

Pose Guided Fashion Image Synthesis Using Deep Generative Model

Attention-Guided GANs for Human Pose Transfer

Pose with style

Progressive Pose Attention Transfer For Person Image Generation

Person Image Synthesis in Arbitrary 3D Poses Based on Part Affinity Fields.