Abstract:The local spatio-temporal descriptor and feature encoding algorithm are two crucial key steps for human action recognition based on spatio-temporal interest points (STIP). Since the local descriptors for STIP are essentially a type of motion information based on the texture, the key point of local feature description is to extract invariable, robust and distinguishable local texture features and motion information in reference spatio-temporal volume. Scattering transform is an image transform method based on directional wavelet transform and scale convolution, which has local translation invariance, rotation invariance and elastic deformation stability for local texture features. A novel local descriptor for STIP based on spatio-temporal three-dimensional scattering transform is proposed in this paper, which extends the original scattering transform to spatio-temporal three-dimensional space. Compared to the traditional descriptors, such as HOG, HOF and so on, the proposed scattering transform coefficients based histogram of oriented gradients (STC-HOG) descriptor can capture more robust and distinguishable motion information of local texture for STIP. In order to incorporate the local descriptors into action video representation, the feature encoding algorithm is indispensable. For the problem that vector of locally aggregated descriptors (VLAD) loses feature distribution location information during feature encoding, a histogram of distribution vector of locally aggregated descriptors (HOD-VALD) based on Gaussian kernel is proposed. We validated the proposed algorithm for human action recognition on multiple public available datasets, such as KTH, UCF Sports, HMDB51 and so on. The evaluation experiment results indicate that the proposed descriptor and encoding method can improve the efficiency of human action recognition and the recognition accuracy.

Human Action Recognition Using Spatio-Temoporal Descriptor

Robust Human Action Recognition Based on Spatio-Temporal Descriptors and Motion Temporal Templates

A Method of Human Action Recognition Based on Spatio-Temporal Interest Points and PLSA

Human Action Recognition Using Multi-Velocity STIPs and Motion Energy Orientation Histogram.

Human Action Recognition Based on Spatio-Temporal Three-Dimensional Scattering Transform Descriptor and an Improved VLAD Feature Encoding Algorithm

An Improved Interest Point Detector for Human Action Recognition

3D Action Recognition Using Multi-Temporal Depth Motion Maps and Fisher Vector

Human Action Recognition by Using Polyhedron Model-Based Spatio-Temporal Gradient Descriptor

A Global Spatio-Temporal Representation for Action Recognition

3d Human Action Recognition Based On The Spatial-Temporal Moving Skeleton Descriptor

A space-time SURF descriptor and its application to action recognition with video words

Local feature extraction using time domain information for human action recognition

A spatio-temporal interest point detector based on vorticity for action recognition

Spatio-temporal Semantic Features for Human Action Recognition.

Action Recognition Using 3D DAISY Descriptor

Human Activity Recognition Based on Pose Points Selection

Machine Vision-Based Human Action Recognition Using Spatio-Temporal Motion Features (STMF) with Difference Intensity Distance Group Pattern (DIDGP)

3D R Transform on Spatio-temporal Interest Points for Action Recognition

Action recognition based on video words with a space-time speeded up robust features descriptor

Human Action Recognition Algorithm Based on Spatial Temporal Depth Feature

Human Action Recognition By Som Considering The Probability Of Spatio-Temporal Features