Abstract:Spatial–temporal action detection in videos is a challenging problem that has attracted considerable attention in recent years. Most current approaches address action detection as an object detection problem, which utilizes successful object detection frameworks such as Faster R-CNN to operate action detection at every single frame first, and then generates action tubes by linking bounding boxes across the whole video in an offline fashion. However, unlike object detection in static images, temporal context information is vital for action detection in videos. Therefore, we propose an online action detection model that leverages the spatial–temporal context information existing in videos to perform action inference and localization. More specifically, we try to depict the spatial–temporal context pattern of actions via an encoder–decoder model that is based on a convolutional recurrent neural network. The model accepts a video snippet as input and encodes the dynamic information inside the snippet in the forward pass. During the backward pass, the decoder resolves the information for action detection with the current appearance or motion cue at each time stamp. In addition, we devise an incremental action-tube construction algorithm that enables our model to accomplish action prediction ahead of time and performs action detection in an online fashion. To evaluate the performance of our method, we conduct experiments on three popular public datasets UCF-101, UCF-Sports, and J-HMDB-21. The experimental results demonstrate that our method can achieve competitive or superior performance when compared to the state-of-the-art methods. To encourage further research, we release our project on “https://github.com.hjjpku.OATD.”

Online Action Detection Incorporating an Additional Action Classifier

Long Short-Term Transformer for Online Action Detection

MALT: Multi-scale Action Learning Transformer for Online Action Detection

Multi-Modality Multi-Task Recurrent Neural Network for Online Action Detection

HUMAN-AWARE COARSE-TO-FINE ONLINE ACTION DETECTION

Exploring 3d Human Action Recognition: From Offline To Online

TwinLSTM: Two-channel LSTM Network for Online Action Detection.

Annealing Temporal-Spatial Contrastive Learning for Multi-View Online Action Detection

DFAformer: A Dual Filtering Auxiliary Transformer for Efficient Online Action Detection in Streaming Videos.

Temporal Recurrent Networks for Online Action Detection

Online Activity Detection with Predicting and Classifying Transformers

Online Robust Action Recognition Based on a Hierarchical Model

Online Action Tube Detection Via Resolving The Spatio-Temporal Context Pattern

Semi-Supervised Multiple Feature Analysis for Action Recognition

Colar: Effective and Efficient Online Action Detection by Consulting Exemplars

Spatial–Temporal Context-Aware Online Action Detection and Prediction

Action Recognition from Videos with Complex Background via Transfer Learning

Discriminative Action States Discovery for Online Action Recognition

Online Action Detection and Forecast Via Multitask Deep Recurrent Neural Networks

Action Recognition by Exploring Data Distribution and Feature Correlation

An empirical study on temporal modeling for online action detection