Abstract:This paper presents an effective multiscale energy-based global ternary image (E-GTI) representation for action recognition from depth sequences. The unique property of our representation is that it takes the spatiotemporal discrimination and action speed variations into account, intending to solve the problems of distinguishing similar actions and identifying the actions with different speeds in one goal. The entire method is carried out in two stages. In the first stage, consecutive depth frames are used to generate global ternary image (GTI) features, which implicitly capture both inter-frame motion regions and motion directions. Specifically, each pixel in the GTI represents one of three possible states, namely, positive, negative, and neutral, which indicate the increased, decreased, and same depth values, respectively. To cope with speed variations in actions, energy-based sampling method is utilized, leading to multiscale E-GTI features, where the multiscale scheme can efficiently capture the temporal relationships among frames. In the second stage, all the E-GTI features are transformed by Radon transform (RT) as robust descriptors, which are aggregated by the bag-of-visual-words model as a compact representation. Extensive experiments on benchmark data sets show that our representation outperforms state-of-the-art approaches, since it captures discriminating spatiotemporal information of actions. Due to the merits of energy-based sampling and RT methods, our representation shows robustness to speed variations, depth noise, and partial occlusions.

Action Recognition Using Multi-Scale Histograms of Oriented Gradients Based Depth Motion Trail Images

Recognizing actions using depth motion maps-based histograms of oriented gradients

Human Action Recognition Based on DMMs, HOGs and Contourlet Transform

Combining Multi-scale Directed Depth Motion Maps and Log-Gabor Filters for Human Action Recognition

Human action recognition using Adaptive Hierarchical Depth Motion Maps and Gabor filter

Online Robust Action Recognition Based on a Hierarchical Model

Action Recognition from Depth Sequences Using Weighted Fusion of 2D and 3D Auto-Correlation of Gradients Features

3D Action Recognition Using Multi-Temporal Depth Motion Maps and Fisher Vector

Action Segmentation And Recognition Based On Depth Hog And Probability Distribution Difference

Real-Time Human Action Recognition System Using Depth Map Sequences

Human Action Recognition Based on Deep Images and Dense Trajectories

Deep Convolutional Neural Networks for Action Recognition Using Depth Map Sequences

Action Recognition Using Edge Trajectories and Motion Acceleration Descriptor

Human Action Recognition with Contextual Constraints Using a RGB-D Sensor

Multimodal human action recognition based on spatio-temporal action representation recognition model

3D Action Recognition Using Multi-Temporal Skeleton Visualization.

3D Action Recognition Using Multi-scale Energy-based Global Ternary Image

Multi-Temporal Depth Motion Maps-Based Local Binary Patterns for 3-D Human Action Recognition

Action Recognition By Learning Deep Multi-Granular Spatio-Temporal Video Representation

Action Recognition with Multi-Scale Trajectory-Pooled 3D Convolutional Descriptors

DMMs-Based Multiple Features Fusion for Human Action Recognition