Abstract:The local spatio-temporal descriptor and feature encoding algorithm are two crucial key steps for human action recognition based on spatio-temporal interest points (STIP). Since the local descriptors for STIP are essentially a type of motion information based on the texture, the key point of local feature description is to extract invariable, robust and distinguishable local texture features and motion information in reference spatio-temporal volume. Scattering transform is an image transform method based on directional wavelet transform and scale convolution, which has local translation invariance, rotation invariance and elastic deformation stability for local texture features. A novel local descriptor for STIP based on spatio-temporal three-dimensional scattering transform is proposed in this paper, which extends the original scattering transform to spatio-temporal three-dimensional space. Compared to the traditional descriptors, such as HOG, HOF and so on, the proposed scattering transform coefficients based histogram of oriented gradients (STC-HOG) descriptor can capture more robust and distinguishable motion information of local texture for STIP. In order to incorporate the local descriptors into action video representation, the feature encoding algorithm is indispensable. For the problem that vector of locally aggregated descriptors (VLAD) loses feature distribution location information during feature encoding, a histogram of distribution vector of locally aggregated descriptors (HOD-VALD) based on Gaussian kernel is proposed. We validated the proposed algorithm for human action recognition on multiple public available datasets, such as KTH, UCF Sports, HMDB51 and so on. The evaluation experiment results indicate that the proposed descriptor and encoding method can improve the efficiency of human action recognition and the recognition accuracy.

Multi-feature fusion based human action recognition algorithm

Fusing Mixed Visual Features for Human Action Recognition

Human Action Recognition Based on Multi-feature Fusion of SVD and 3D-SIFT Features

Human action recognition using late fusion and dimensionality reduction

DMMs-Based Multiple Features Fusion for Human Action Recognition

Discriminative Feature Fusion with Spectral Method for Human Action Recognition.

Human Action Recognition Based on Three-Stream Network with Frame Sequence Features

Human Action Recognition Based on Fusion Features Extraction of Adaptive Background Subtraction and Optical Flow Model

Action Recognition Using Hybrid Feature Descriptor And Vlad Video Encoding

Real-time Action Recognition by Feature-Level Fusion of Depth and Inertial Sensor

Multimodal human action recognition based on spatio-temporal action representation recognition model

Using a Selective Ensemble Support Vector Machine to Fuse Multimodal Features for Human Action Recognition

Action Recognition Using Spatiotemporal Features and Hybrid Generative/Discriminative Models

Local-aware spatio-temporal attention network with multi-stage feature fusion for human action recognition

Hmnan Action Recognition in Complex Scenes Based on Fuzzy Integral Fusion

Multi-dimension Feature Fusion for Action Recognition

A multidimensional feature fusion network based on MGSE and TAAC for video-based human action recognition

Feature Fusion of Triaxial Acceleration Signals and Depth Maps for Human Action Recognition

Multi-feature Fusion Action Recognition Based on Key Frames.

Action Recognition by Multiple Features and Hyper-Sphere Multi-class SVM

Human Action Recognition Based on Spatio-Temporal Three-Dimensional Scattering Transform Descriptor and an Improved VLAD Feature Encoding Algorithm