Abstract:Recognizing human actions in videos is a challenging problem owning to complex motion appearance, various backgrounds and semantic gap between low-level features and high-level semantics. Existing methods have scored some achievements and many new thoughts have been proposed for action recognition. They focus on designing a robust feature description and training an elaborate learning model, and many of them can benefit from a two-stream network with a stack of RGB frames and optical flow frames. However, these features for human action representation are struggling with the limited feature representation as RGB videos are confused by static appearance redundancy and optical flow videos cannot represent the detailed appearance. To solve these problems, we propose an efficient algorithm based on the spatial-optical data organization and the sequential learning framework. There are two contributions of our method: a novel data organization based on hierarchical weighting segmentation and optical flow for video representation, and a lightweight deep learning model based on the Convolutional 3D (C3D) network and the Recurrent Neural Network (RNN) for complicated action recognition. The new data organization aggregates the merits of motion appearance, movement trajectories and optical flow in a creative way to highlight the meaningful information. And the proposed lightweight model has an insight into patterns and semantics of sequential data by low-level spatiotemporal feature extraction and high-level information mining. The proposed method is evaluated on the state-of-the-art dataset and the results demonstrate that our method have a good performance for complex human action recognition.

Human behavior segmentation and recognition using Continuous Linear Dynamic System

A Method of Simultaneously Action Recognition and Video Segmentation of Video Streams.

Continuous Human Action Recognition in Real Time

Action recognition using linear dynamic systems

Continuous Action Recognition and Segmentation in Untrimmed Videos

Joint Segmentation and Recognition of Worker Actions Using Semi-Markov Models

Continuous Motion Classification and Segmentation Based on Improved Dynamic Time Warping Algorithm.

Continuous Action Recognition Based on Hybrid CNN-LDCRF Model

Human Action Recognition with Contextual Constraints Using a RGB-D Sensor

Linear Regression Motion Analysis for Unsupervised Temporal Segmentation of Human Actions

Online Robust Action Recognition Based on a Hierarchical Model

Human Action Recognition Using Deep Learning Methods.

Extreme Learning Machine for Linear Dynamical Systems Classification: Application to Human Activity Recognition

Joint Multi-Scale Residual and Motion Feature Learning for Action Recognition.

Action Recognition Using Spatial-Optical Data Organization and Sequential Learning Framework

Human Activity Recognition Using Smart Phone Embedded Sensors: A Linear Dynamical Systems Method

Nonlinear Dynamical System Analysis For Continuous Gesture Recognition

An Online Continuous Human Action Recognition Algorithm Based on the Kinect Sensor

Continuous Action Segmentation and Recognition Using Hybrid Convolutional Neural Network-Hidden Markov Model Model

Human Interaction Representation and Recognition Through Motion Decomposition.

Recognizing continuous human actions from RGB-D videos