Abstract:Recognizing human actions in videos is a challenging problem owning to complex motion appearance, various backgrounds and semantic gap between low-level features and high-level semantics. Existing methods have scored some achievements and many new thoughts have been proposed for action recognition. They focus on designing a robust feature description and training an elaborate learning model, and many of them can benefit from a two-stream network with a stack of RGB frames and optical flow frames. However, these features for human action representation are struggling with the limited feature representation as RGB videos are confused by static appearance redundancy and optical flow videos cannot represent the detailed appearance. To solve these problems, we propose an efficient algorithm based on the spatial-optical data organization and the sequential learning framework. There are two contributions of our method: a novel data organization based on hierarchical weighting segmentation and optical flow for video representation, and a lightweight deep learning model based on the Convolutional 3D (C3D) network and the Recurrent Neural Network (RNN) for complicated action recognition. The new data organization aggregates the merits of motion appearance, movement trajectories and optical flow in a creative way to highlight the meaningful information. And the proposed lightweight model has an insight into patterns and semantics of sequential data by low-level spatiotemporal feature extraction and high-level information mining. The proposed method is evaluated on the state-of-the-art dataset and the results demonstrate that our method have a good performance for complex human action recognition.

Saliency-based dense trajectories for action recognition using low-rank matrix decomposition.

Combined Trajectories for Action Recognition Based on Saliency Detection and Motion Boundary.

An Action Recognition Method Using Saliency Detection

Detecting Action-Relevant Regions for Action Recognition Using a Three-Stage Saliency Detection Technique

Features extraction approach based on dense salient trajectories in videos

Realistic action recognition with salient foreground trajectories

Human action recognition in videos based on dense trajectory selection

Action Recognition Based on Object Tracking and Dense Trajectories

Human action recognition with salient trajectories and multiple kernel learning

Activity Recognition Using Dense Long-Duration Trajectories

Action Recognition Using Edge Trajectories and Motion Acceleration Descriptor

Exploring the Influence of Motion Boundary Sampling to Improved Dense Trajectories for Action Recognition

Human Action Recognition by Fast Dense Trajectories.

A Study of Relative Motion Point Trajectories for Action Recognition

Human Action Recognition Based on Action Relevance Weighted Encoding

Motion Saliency Detection Using Low-Rank And Sparse Decomposition

Action Recognition by Jointly Using Video Proposal and Trajectory

Temporal Distinct Representation Learning for Action Recognition

Spatiotemporal Saliency Representation Learning for Video Action Recognition

Action Recognition Using Spatial-Optical Data Organization and Sequential Learning Framework

TSI: Temporal Saliency Integration for Video Action Recognition