Abstract:For RGB-based temporal action segmentation (TAS), excellent methods that capture frame-level features have achieved remarkable performance. However, for motion-centered TAS, it is still challenging for existing methods that ignore the extraction of spatial features of joints. In addition, inaccurate action boundaries caused by the frames of similar motion destroy the integrity of the action segments. To alleviate the issues, an end-to-end Involving Distinguished Temporal Graph Convolutional Networks called IDT-GCN is proposed. First, we construct an enhanced spatial graph structure that adaptively captures the similar and differential dependencies between joints in a single topology through learning two independent correlation modeling functions. Then, the proposed Involving Distinguished Graph Convolutional (ID-GC) models the spatial correlations of different actions in a video by using multiple enhanced topologies on the corresponding channels. Furthermore, we design a generic modeling temporal action regression network, termed Temporal Segment Regression (TSR), to extract segmented encoding features and action boundary representations by modeling action sequences. Combining them with label smoothing modules, we develop powerful spatial-temporal graph convolutional networks (IDT-GCN) for fine-grained TAS, which notably outperforms state-of-the-art methods on the MCFS-22 and MCFS-130 datasets. Adding TSR to TCN-based baseline methods achieves competitive performance compared with the state-of-the-art transformer-based methods on RGB-based datasets, i.e., Breakfast and 50Salads. Further experimental results on the action recognition task verify the superiority of the enhanced spatial graph structure over the previous graph convolutional networks.

Temporal Segmentation of Fine-gained Semantic Action: A Motion-Centered Figure Skating Dataset.

3D Pose-Based Temporal Action Segmentation for Figure Skating: A Fine-Grained and Jump Procedure-Aware Annotation Approach

Fine-grained Action Analysis: A Multi-modality and Multi-task Dataset of Figure Skating

Fine-Grained Unsupervised Temporal Action Segmentation and Distributed Representation for Skeleton-Based Human Motion Analysis

Decoupling Spatio-Temporal Network for Fine-Grained Temporal Action Segmentation

FSD-10: A fine-grained classification dataset for figure skating

A motion-aware and temporal-enhanced Spatial–Temporal Graph Convolutional Network for skeleton-based human action segmentation

Involving Distinguished Temporal Graph Convolutional Networks for Skeleton-Based Temporal Action Segmentation

FSD-10: A Dataset for Competitive Sports Content Analysis

Skeleton-based Tai Chi action segmentation using trajectory primitives and content

An efficient framework for few-shot skeleton-based temporal action segmentation

HFGCN-Based Action Recognition System for Figure Skating.

Learning from Temporal Spatial Cubism for Cross-Dataset Skeleton-based Action Recognition

Hierarchical Spatial-Temporal Network for Skeleton-Based Temporal Action Segmentation.

SMC-NCA: Semantic-guided Multi-level Contrast for Semi-supervised Temporal Action Segmentation

Motion Complement and Temporal Multifocusing for Skeleton-Based Action Recognition

Spatial-temporal Graph Transformer Network for Skeleton-Based Temporal Action Segmentation

Data Synthesizing for Specific Human Action Segmentation

Streaming Video Temporal Action Segmentation In Real Time

Linear Regression Motion Analysis for Unsupervised Temporal Segmentation of Human Actions

MultiSports: A Multi-Person Video Dataset of Spatio-Temporally Localized Sports Actions