Abstract:Inspired by the recent advances in generative models, we introduce a human action generation model in order to generate a consecutive sequence of human motions to formulate novel actions. We propose a framework of an autoencoder and a generative adversarial network (GAN) to produce multiple and consecutive human actions conditioned on the initial state and the given class label. The proposed model is trained in an end-to-end fashion, where the autoencoder is jointly trained with the GAN. The model is trained on the NTU RGB+D dataset and we show that the proposed model can generate different styles of actions. Moreover, the model can successfully generate a sequence of novel actions given different action labels as conditions. The conventional human action prediction and generation models lack those features, which are essential for practical applications.

What problem does this paper attempt to address?

This paper aims to solve the problem of generating continuous human action sequences. Specifically, the author proposes a new human - action - generation model based on Generative Adversarial Networks (GAN) to generate diverse human - action categories and their continuous action sequences. Compared with existing human - action prediction and generation models, this model has the following characteristics: 1. **Generate new - type actions**: Existing models mainly focus on predicting future action frames, while the new model can generate completely new action sequences that have not appeared in the training data. 2. **Control action categories and styles**: By introducing Conditional Generative Adversarial Networks (conditional GAN), the model can consider action - category labels and initial postures when generating actions, thereby generating actions of specific categories and generating actions of different styles by changing the random vector \( z \). 3. **Generate continuous actions**: The model can generate a series of continuous actions without obvious discontinuity between each action, which is very important for practical applications. ### Main contributions - **Introduce semi - supervised model**: The semi - supervised model based on GAN can generate new - type human actions. - **Consider action categories and initial states**: The model considers action - category labels and initial postures when generating actions. - **Generate multiple styles**: By changing the random vector \( z \), the model can generate actions of different styles within the same category. - **Generate continuous actions**: The model can generate a series of continuous actions to ensure a smooth transition between actions. ### Method overview - **Generative Adversarial Networks (GAN)**: The model contains a generator and a discriminator. The generator is responsible for generating action sequences, and the discriminator is responsible for distinguishing between the generated action sequences and the real data. - **Conditional Generative Adversarial Networks (Conditional GAN)**: Add action - category labels and initial postures to the data input to the generator and the discriminator to control the generated action categories and styles. - **Autoencoder**: Used to compress high - dimensional human - pose data into a low - dimensional space and reconstruct the human - pose sequence during the generation stage. - **Consistency loss**: To ensure the consistency of the generated action sequences between adjacent frames, the model introduces a consistency - loss function \( L_{\text{cons}} \). ### Experimental results - **Dataset**: The model is trained and tested on the NTU RGB + D dataset, which contains 60 different action categories. - **Generation effect**: By changing the random vector \( z \), the model can generate action sequences of different styles. Meanwhile, by changing the action - category labels, the model can generate actions of different categories. - **Generate continuous actions**: The model can generate a series of continuous actions, such as "sit down" followed by "stand up", demonstrating the effectiveness of the model in generating continuous actions. In conclusion, this paper proposes an innovative method that can generate new - type actions while controlling the categories and styles of actions and generating continuous action sequences, providing a new solution for the field of human - action generation.

Human Action Generation with Generative Adversarial Networks

3D Human motion anticipation and classification

Pose-guided Generative Adversarial Net for Novel View Action Synthesis

Active Generation Network of Human Skeleton for Action Recognition

Pose Guided Human Video Generation

Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks

Early Action Prediction with Generative Adversarial Networks

Generative Tweening: Long-term Inbetweening of 3D Human Motions

HumanGAN: A Generative Model of Humans Images

Deep Video Generation, Prediction and Completion of Human Action Sequences

Action-Conditioned 3D Human Motion Synthesis with Transformer VAE

GANimator: Neural Motion Synthesis from a Single Sequence

Generative Model for Skeletal Human Movements Based on Conditional DC-GAN Applied to Pseudo-Images

Generative adversarial networks

Continual Activity Recognition with Generative Adversarial Networks

Hierarchical Graph-Convolutional Variational AutoEncoding for Generative Modelling of Human Motion

The Imaginative Generative Adversarial Network: Automatic Data Augmentation for Dynamic Skeleton-Based Hand Gesture and Human Action Recognition

Human Behavior Modeling During Dialogue by Using Generative Adversarial Networks

Prediction-CGAN

Combining Recurrent Neural Networks and Adversarial Training for Human Motion Synthesis and Control