Abstract:The popularity of wearable devices has increased the demands for the research on first-person activity recognition. However, most of the current first-person activity datasets are built based on the assumption that only the human-object interaction (HOI) activities, performed by the camera-wearer, are captured in the field of view. Since humans live in complicated scenarios, in addition to the first-person activities, it is likely that third-person activities performed by other people also appear. Analyzing and recognizing these two types of activities simultaneously occurring in a scene is important for the camera-wearer to understand the surrounding environments. To facilitate the research on concurrent first- and third-person activity recognition (CFT-AR), we first created a new activity dataset, namely PolyU concurrent first- and third-person (CFT) Daily, which exhibits distinct properties and challenges, compared with previous activity datasets. Since temporal asynchronism and appearance gap usually exist between the first- and third-person activities, it is crucial to learn robust representations from all the activity-related spatio-temporal positions. Thus, we explore both holistic scene-level and local instance-level (person-level) features to provide comprehensive and discriminative patterns for recognizing both first- and third-person activities. On the one hand, the holistic scene-level features are extracted by a 3-D convolutional neural network, which is trained to mine shared and sample-unique semantics between video pairs, via two well-designed attention-based modules and a self-knowledge distillation (SKD) strategy. On the other hand, we further leverage the extracted holistic features to guide the learning of instance-level features in a disentangled fashion, which aims to discover both spatially conspicuous patterns and temporally varied, yet critical, cues. Experimental results on the PolyU CFT Daily dataset validate that our method achieves the state-of-the-art performance.

Learning Semantics-Preserving Attention and Contextual Interaction for Group Activity Recognition.

Learning Visual Context for Group Activity Recognition.

Mining Semantics-Preserving Attention for Group Activity Recognition.

Bi-View Semi-Supervised Learning Based Semantic Human Activity Recognition Using Accelerometers.

Semantics-Driven Multi-user Concurrent Activity Recognition.

Exploring Semantics in Activity Recognition Using Context Lattices.

Group Activity Recognition Based on Temporal Semantic Sub-Graph Network.

Discriminative Latent Models for Recognizing Contextual Group Activities

Progressive Relation Learning for Group Activity Recognition

Attentive pooling for Group Activity Recognition

Unveiling group activity recognition: Leveraging Local–Global Context-Aware Graph Reasoning for enhanced actor–scene interactions

Towards Context-aware Interaction Recognition.

An Attentional Spatial Temporal Graph Convolutional Network with Co-Occurrence Feature Learning for Action Recognition

Group Activity Recognition in Computer Vision: A Comprehensive Review, Challenges, and Future Perspectives

Towards Context-Aware Interaction Recognition for Visual Relationship Detection

Privacy-preserving Activity Recognition Using Multimodal Sensors in Smart Office

Latent Embeddings for Collective Activity Recognition

Holistic-Guided Disentangled Learning with Cross-Video Semantics Mining for Concurrent First-Person and Third-Person Activity Recognition.

Dynamical Attention Hypergraph Convolutional Network for Group Activity Recognition

Coherence Constrained Graph LSTM for Group Activity Recognition

Position-Aware Participation-Contributed Temporal Dynamic Model for Group Activity Recognition.