Abstract:Real-time Human action classification in complex scenes has applications in various domains such as visual surveillance, video retrieval and human robot interaction. While, the task is challenging due to computation efficiency, cluttered backgrounds and intro-variability among same type of actions. Spatio-temporal interest point (STIP) based methods have shown promising results to tackle human action classification in complex scenes efficiently. However, the state-of-the-art works typically utilize bag-of-visual words (BoVW) model which only focuses on the word distribution of STIPs and ignore the distinctive character of word structure. In this paper, the distribution of STIPs is organized into a salient directed graph, which reflects salient motions and can be divided into a time salient directed graph and a space salient directed graph, aiming at adding spatio-temporal discriminant to BoVW. Generally speaking, both salient directed graphs are constructed by labeled STIPs in pairs. In detail, the "directional co-occurrence" property of different labeled pairwise STIPs in same frame is utilized to represent the time saliency, and the space saliency is reflected by the "geometric relationships" between same labeled pairwise STIPs across different frames. Then, new statistical features namely the Time Salient Pairwise feature (TSP) and the Space Salient Pairwise feature (SSP) are designed to describe two salient directed graphs, respectively. Experiments are carried out with a homogeneous kernel SVM classifier, on four challenging datasets KTH, ADL and UT-Interaction. Final results confirm the complementary of TSP and SSP, and our multi-cue representation TSP + SSP + BoVW can properly describe human actions with large intro-variability in real-time. Copyright (C) 2016, Chongqing University of Technology. Production and hosting by Elsevier B.V.

Bi-STAN: bilinear spatial-temporal attention network for wearable human activity recognition

Attention-based LSTM Network for Wearable Human Activity Recognition

Multi-STMT: Multi-Level Network for Human Activity Recognition Based on Wearable Sensors

MMTSA: Multimodal Temporal Segment Attention Network for Efficient Human Activity Recognition

An Attentional Spatial Temporal Graph Convolutional Network with Co-Occurrence Feature Learning for Action Recognition

A Novel Spatio-Temporal Network of Multi-channel CNN and GCN for Human Activity Recognition Based on BAN

STFNet: Enhanced and Lightweight Spatiotemporal Fusion Network for Wearable Human Activity Recognition

Attention-based Convolutional Neural Network for Weakly Labeled Human Activities Recognition with Wearable Sensors

TCN-attention-HAR: human activity recognition based on attention mechanism time convolutional network

DanHAR: Dual Attention Network for multimodal human activity recognition using wearable sensors

Spatio-Temporal Adaptive Network with Bidirectional Temporal Difference for Action Recognition

Unified Spatio-Temporal Attention Networks for Action Recognition in Videos.

Multihead-Res-SE Residual Network with Attention for Human Activity Recognition

A Semisupervised Recurrent Convolutional Attention Model for Human Activity Recognition

Salient Pairwise Spatio-Temporal Interest Points for Real-Time Activity Recognition.

Human Activity Recognition Based on Residual Network and BiLSTM

Enhanced Attention Tracking with Multi-Branch Network for Egocentric Activity Recognition

Innovative Dual-Decoupling CNN with Layer-wise Temporal-Spatial Attention for Sensor-Based Human Activity Recognition

A Novel Deep Learning Bi-GRU-I Model for Real-Time Human Activity Recognition Using Inertial Sensors

A Novel CNN-based Bi-LSTM parallel model with attention mechanism for human activity recognition with noisy data

AttnSense: Multi-level Attention Mechanism for Multimodal Human Activity Recognition