Abstract:In human activity recognition system both static and motion information play crucial role for efficient and competitive results. Most of the existing methods are insufficient to extract video features and unable to investigate the level of contribution of both (Static and Motion) components. Our work highlights this problem and proposes Static-Motion fused features descriptor (SMFD), which intelligently leverages both static and motion features in the form of descriptor. First, static features are learned by two-stream 3D convolutional neural network. Second, trajectories are extracted by tracking key points and only those trajectories have been selected which are located in central region of the original video frame in order to to reduce irrelevant background trajectories as well computational complexity. Then, shape and motion descriptors are obtained along with key points by using SIFT flow. Next, cholesky transformation is introduced to fuse static and motion feature vectors to guarantee the equal contribution of all descriptors. Finally, Long Short-Term Memory (LSTM) network is utilized to discover long-term temporal dependencies and final prediction. To confirm the effectiveness of the proposed approach, extensive experiments have been conducted on three well-known datasets i.e. UCF101, HMDB51 and YouTube. Findings shows that the resulting recognition system is on par with state-of-the-art methods.

Video Representation by Dense Trajectories Motion Map Applied to Human Activity Recognition

Video Representation Via Fusion of Static and Motion Features Applied to Human Activity Recognition.

Human Action Recognition Based on Action Relevance Weighted Encoding

Activity Recognition Using Dense Long-Duration Trajectories

Learning Deep Trajectory Descriptor for Action Recognition in Videos Using Deep Neural Networks.

Compressed Video Action Recognition Using Motion Vector Representation.

Human Action Recognition by Fast Dense Trajectories.

Trajectory-Based 3D Convolutional Descriptors for Human Action Recognition

Human Action Recognition From Digital Videos Based on Deep Learning.

Deep Trajectory for Recognition of Human Behaviours

Action Recognition Based on Dense Trajectories and Human Detection

Trajectories-based Motion Neighborhood Feature for Human Action Recognition

Disparity-Augmented Trajectories for Human Activity Recognition

Action Recognition Based on Object Tracking and Dense Trajectories

Human Action Recognition Based on DMMs, HOGs and Contourlet Transform

Human action recognition in videos based on dense trajectory selection

Joint Dynamic Pose Image and Space Time Reversal for Human Action Recognition from Videos

Action recognition via restricted dense trajectories and spatio-temporal co-occurrence feature

Tracking Salient Keypoints for Human Action Recognition

Temporal-Spatial Mapping for Action Recognition

Human Action Recognition with Trajectory Based Covariance Descriptor in Unconstrained Videos