Abstract:Abstract: This research paper introduces innovative approaches to enhance human action analytics in computer vision through the development of spatial and temporal attention models. The first model is based on Recurrent Neural Networks (RNNs) with Long Short-Term Memory (LSTM) units, aiming to extract discriminative spatial and temporal features from skeleton data for improved human action recognition and detection. The model selectively focuses on discriminative joints within each frame, utilizing a regularized cross-entropy loss for effective training. Additionally, a joint training strategy is proposed to optimize the learning process. The second model, a spatio-temporal attention (STA) network, is designed to address the limitations of existing 3D Convolutional Neural Networks (3D CNNs) in treating all video frames equally. The STA network characterizes beneficial information at both the frame and channel levels, leveraging differences in spatial and temporal dimensions to enhance the learning capability of 3D convolutions. The proposed STA method can be seamlessly integrated into state-of-the-art 3D CNN architectures for video action detection and recognition. Evaluation of diverse datasets, including SBU Kinect Interaction, NTU RGB+D, PKU-MMD, UCF-101, HMDB-51, and THUMOS 2014, demonstrates the effectiveness of both proposed models in achieving state-of-the-art performance for action recognition and detection tasks. The spatial and temporal attention models contribute significantly to capturing discriminative features, showcasing their potential in advancing the field of human action analytics in computer vision.

HumanAttention ModelforAction MovieAnalysis

Human attention model for semantic scene analysis in movies

A novel attention model and its application in video analysis

Human Attention Based Movie Summarization: Dataset and Baseline Model

Video Abstraction via Attention Model and On-Line Clustering

Video Saliency Detection via Dynamic Consistent Spatio-Temporal Attention Modelling.

Computational Model of Primary Visual Cortex Combining Visual Attention for Action Recognition

A VISUAL ATTENTION MODEL FOR NATURAL SCENES BASED ON DYNAMIC FEATURE COMBINATION

Spatial and Temporal Visual Attention Prediction in Videos Using Eye Movement Data

A Hierarchical Framework For Movie Content Analysis: Let Computers Watch Films Like Humans

B2C-AFM: Bi-Directional Co-Temporal and Cross-Spatial Attention Fusion Model for Human Action Recognition.

Where and Why Are They Looking? Jointly Inferring Human Attention and Intentions in Complex Tasks

Human-centered attention models for video summarization

Visual Attention on Human Face

Analysis of Human Actions for Video Indexing

Fast and Accurate Action Detection in Videos With Motion-Centric Attention Model

Unified Spatio-Temporal Attention Models for Advanced Human Action Recognition & Detection

FMRI-Guided Time-Symmetric Joint Model for Visual Attention Prediction.

Predicting Human Interaction Via Relative Attention Model.

Interpretable Spatio-temporal Attention for Video Action Recognition

Human Attention Detection Using AM-FM Representations