Abstract:The aim of temporal action localization (TAL) is to determine the start and end frames of an action in a video. In recent years, TAL has attracted considerable attention because of its increasing applications in video understanding and retrieval. However, precisely estimating the duration of an action in the temporal dimension is still a challenging problem. In this paper, we propose an effective one‐stage TAL method based on a self‐defined motion data structure, called a dense joint motion matrix (DJMM), and a novel temporal detection strategy. Our method provides three main contributions. First, compared with mainstream motion images, DJMMs can preserve more pre‐processed motion features and provides more precise detail representations. Furthermore, DJMMs perfectly solve the temporal information loss problem caused by motion trajectory overlaps within a certain time period. Second, a spatial pyramid pooling (SPP) layer, which is widely used in the object detection and tracking fields, is innovatively incorporated into the proposed method for multi‐scale feature learning. Moreover, the SPP layer enables the backbone convolutional neural network (CNN) to receive DJMMs of any size in the temporal dimension. Third, a large‐scale‐first temporal detection strategy inspired by a well‐developed Chinese text segmentation algorithm is proposed to address long‐duration videos. Our method is evaluated on two benchmark data sets and one self‐collected data set: Florence‐3D, UTKinect‐Action3D and HanYue‐3D. The experimental results show that our method achieves competitive action recognition accuracy and high TAL precision, and its time efficiency and few‐shot learning capabilities enable it to be utilized for real‐time surveillance.

BRMR: TAL Based on Boundary Refinement and Multi-scale Regression.

Gated Multi-Scale Transformer for Temporal Action Localization

Mmformer: Multimodal Medical Transformer for Incomplete Multimodal Learning of Brain Tumor Segmentation

Cross Time-Frequency Transformer for Temporal Action Localization

Temporal Deformable Transformer for Action Localization

Advancing Temporal Action Localization with a Boundary Awareness Network

Boundary-Recovering Network for Temporal Action Detection

Cascaded Boundary Regression for Temporal Action Detection

Local–Global Transformer Neural Network for Temporal Action Segmentation

Multi‐scale feature learning and temporal probing strategy for one‐stage temporal action localization

Temporal Segment Transformer for Action Segmentation

TALLFormer: Temporal Action Localization with a Long-memory Transformer

Bottom-Up Temporal Action Localization with Mutual Regularization

Multi-granularity transformer fusion for temporal action localization

Double branch synergies with modal reinforcement for weakly supervised temporal action detection

TSRN: Two-Stage Refinement Network for Temporal Action Segmentation

Multi-Scale Adaptive Skeleton Transformer for action recognition

Actra: Optimized Transformer Architecture for Vision-Language-Action Models in Robot Learning

Actor-Multi-Scale Context Bidirectional Higher Order Interactive Relation Network for Spatial-Temporal Action Localization.

Localization and recognition of human action in 3D using transformers

ConvTransformer Attention Network for temporal action detection