Abstract:In this paper, we propose Occlusion-Aware Warping GAN (OAW-GAN), a unified Human Video Synthesis (HVS) framework that can uniformly tackle human video motion transfer, attribute editing, as well as inpainting. This is the first work to our knowledge that can handle all these tasks within a one-time trained model. Although existing GAN-based HVS methods have achieved great success, they either can’t preserve appearance details due to the loss of spatial consistency between the synthesized target frames and the input source images, or generate incoherent video results due to the loss of temporal consistency among frames. Besides, most of them lack the ability to create new contents while keeping existing ones, failing especially when some regions in the target are invisible in the source due to self-occlusion. To address these limitations, we first introduce Coarse-to-Fine Flow Warping Network (C2F-FWN) to estimate spatial-temporal consistent transformation between source and target, as well as occlusion mask indicating which parts in the target are invisible in the source. Then, the flow and the mask are scaled and fed into the pyramidal stages of our OAW-GAN, guiding Occlusion-Aware Synthesis (OAS) that can be abstracted into visible part re-utilization and invisible part inpainting at the feature level, which effectively alleviates the self-occlusion problem. Extensive experiments conducted on both human video (i.e., iPER, SoloDance)Keywords are desired. please provide if necessary. and image (i.e., DeepFashion) datasets demonstrate the superiority of our approach to existing state-of-the-arts. We also show that, besides motion transfer task that previous works concern, our framework can further achieve attribute editing and texture inpainting, which paves the way towards unified HVS.

HMGAN: A Hierarchical Multi-Modal Generative Adversarial Network Model for Wearable Human Activity Recognition

OAW-GAN: Occlusion-Aware Warping GAN for Unified Human Video Synthesis

Wearable Sensor Based Multimodal Human Activity Recognition Exploiting the Diversity of Classifier Ensemble.

RGC: Reliable Gesture Classification Via Wearables Using GANs-Based Data Augmentation.

Continual Activity Recognition with Generative Adversarial Networks

A multi-graph convolutional network based wearable human activity recognition method using multi-sensors

Data augmentation for Human Activity Recognition with Generative Adversarial Networks

Unsupervised Domain Adaptation in Activity Recognition: A GAN-Based Approach

Guided-GAN: Adversarial Representation Learning for Activity Recognition with Wearables

ContrasGAN: Unsupervised Domain Adaptation in Human Activity Recognition Via Adversarial and Contrastive Learning

Spatial-Temporal Masked Autoencoder for Multi-Device Wearable Human Activity Recognition

BSDGAN: Balancing Sensor Data Generative Adversarial Networks for Human Activity Recognition

Modality Consistency-Guided Contrastive Learning for Wearable-Based Human Activity Recognition

SALIENCE: An Unsupervised User Adaptation Model for Multiple Wearable Sensors Based Human Activity Recognition

Multi-Task GANs for View-Specific Feature Learning in Gait Recognition

Deep Wavelet Convolutional Neural Networks for Multimodal Human Activity Recognition Using Wearable Inertial Sensors

MAG-Res2Net: a novel deep learning network for human activity recognition

Multi-STMT: Multi-Level Network for Human Activity Recognition Based on Wearable Sensors

Towards a Dynamic Inter-Sensor Correlations Learning Framework for Multi-Sensor-Based Wearable Human Activity Recognition.

A GAN‐based Data Augmentation Method for Human Activity Recognition Via the Caching Ability

A Survey on Multimodal Wearable Sensor-based Human Action Recognition