Abstract:Spatiotemporal activity prediction aims to predict user activities at a particular time and location, which is applicable in city planning, activity recommendations, and other domains. The fundamental endeavor in spatiotemporal activity prediction is to model the intricate interaction patterns among users, locations, time, and activities, which is characterized by higher-order relations and heterogeneity. Recently, graph-based methods have gained popularity due to the advancements in graph neural networks. However, these methods encounter two significant challenges. Firstly, higher-order relations and heterogeneity are not adequately modeled. Secondly, the majority of established methods are designed around the static graph structures that rely solely on co-occurrence relations, which can be imprecise. To overcome these challenges, we propose Dy H2 N, a dynamic heterogeneous hypergraph network for spatiotemporal activity prediction. Specifically, to enhance the capacity for modeling higher-order relations, hypergraphs are employed in lieu of graphs. Then we propose a set representation learning-inspired heterogeneous hyperedge learning module, which models higher-order relations and heterogeneity in spatiotemporal activity prediction using a non-decomposable manner. To improve the encoding of heterogeneous spatiotemporal activity hyperedges, a knowledge representation-regularized loss is introduced. Moreover, we present a hypergraph structure learning module to update the hypergraph structures dynamically. Our proposed Dy H2 N model has been extensively tested on four real-world datasets, proving to outperform previous state-of-the-art methods by 5.98% to 27.13%. The effectiveness of all framework components is demonstrated through ablation experiments.

Heterogeneous Graph Network for Action Detection

Spatio-Temporal Action Graph Networks

A Hybrid Graph Network for Complex Activity Detection in Video

SCR-Graph: Spatial-Causal Relationships based Graph Reasoning Network for Human Action Prediction

Tackling higher-order relations and heterogeneity: Dynamic heterogeneous hypergraph network for spatiotemporal activity prediction

An Attentional Spatial Temporal Graph Convolutional Network with Co-Occurrence Feature Learning for Action Recognition

Dynamic Graph Modules for Modeling Object-Object Interactions in Activity Recognition

Contextual Heterogeneous Graph Network for Human-Object Interaction Detection

Spatial-temporal hypergraph based on dual-stage attention network for multi-view data lightweight action recognition

Human Action Recognition Based on Three-Stream Network with Frame Sequence Features

Improved Actor Relation Graph based Group Activity Recognition

Human Activity Recognition based on Dynamic Spatio-Temporal Relations

Spatial–Temporal Context-Aware Online Action Detection and Prediction

Hierarchical graph attention network with pseudo-metapath for skeleton-based action recognition

Spatial-Temporal Hypergraph Neural Network based on Attention Mechanism for Multi-view Data Action Recognition

Action Recognition by Exploring Data Distribution and Feature Correlation

Actor-Multi-Scale Context Bidirectional Higher Order Interactive Relation Network for Spatial-Temporal Action Localization.

A motion-aware and temporal-enhanced Spatial–Temporal Graph Convolutional Network for skeleton-based human action segmentation

Multi-view graph convolution network for the recognition of human action with spatial and temporal occlusion problems

Identity-aware Graph Memory Network for Action Detection

Spatiotemporal Multi-Task Network for Human Activity Understanding.