Abstract:Real-time Human action classification in complex scenes has applications in various domains such as visual surveillance, video retrieval and human robot interaction. While, the task is challenging due to computation efficiency, cluttered backgrounds and intro-variability among same type of actions. Spatio-temporal interest point (STIP) based methods have shown promising results to tackle human action classification in complex scenes efficiently. However, the state-of-the-art works typically utilize bag-of-visual words (BoVW) model which only focuses on the word distribution of STIPs and ignore the distinctive character of word structure. In this paper, the distribution of STIPs is organized into a salient directed graph, which reflects salient motions and can be divided into a time salient directed graph and a space salient directed graph, aiming at adding spatio-temporal discriminant to BoVW. Generally speaking, both salient directed graphs are constructed by labeled STIPs in pairs. In detail, the "directional co-occurrence" property of different labeled pairwise STIPs in same frame is utilized to represent the time saliency, and the space saliency is reflected by the "geometric relationships" between same labeled pairwise STIPs across different frames. Then, new statistical features namely the Time Salient Pairwise feature (TSP) and the Space Salient Pairwise feature (SSP) are designed to describe two salient directed graphs, respectively. Experiments are carried out with a homogeneous kernel SVM classifier, on four challenging datasets KTH, ADL and UT-Interaction. Final results confirm the complementary of TSP and SSP, and our multi-cue representation TSP + SSP + BoVW can properly describe human actions with large intro-variability in real-time. Copyright (C) 2016, Chongqing University of Technology. Production and hosting by Elsevier B.V.

On Space-Time Interest Points

Video Identification Using Spatio-temporal Salient Points

Spatiotemporal interest point detector exploiting appearance and motion-variation information

Salient Pairwise Spatio-Temporal Interest Points for Real-Time Activity Recognition.

Time-ordered Spatial-Temporal Interest Points for Human Action Classification.

Spatio-Temporal Video Segmentation of Static Scenes and Its Applications

An Empirical Study for Human Behavior Analysis

EventPoint: Self-Supervised Interest Point Detection and Description for Event-based Camera

A Low Complexity Interest Point Detector

A Novel Event-Oriented Segment-Of-Interest Discovery Method For Surveillance Video

Human Motion Recognition Based on Spatio-Temporal Interest Point

Time Perception Machine: Temporal Point Processes for the When, Where and What of Activity Prediction

Human Action Recognition Using Multi-Velocity STIPs and Motion Energy Orientation Histogram.

Event-Based Tracking Any Point with Motion-Augmented Temporal Consistency

Spatiotemporal Phase Congruency Based Invariant Features for Human Behavior Classification

Feature Point Extraction from the Local Frequency Map of an Image

3D R Transform on Spatio-temporal Interest Points for Action Recognition

Time-based Mapping of Space Using Visual Motion Invariants

Segmenting Visual Actions Based on Spatio-Temporal Motion Patterns

Temporal visitation patterns of points of interest in cities on a planetary scale: a network science and machine learning approach

Space-Time Visual Analytics of Eye-Tracking Data for Dynamic Stimuli