Abstract:The popularity of wearable devices has increased the demands for the research on first-person activity recognition. However, most of the current first-person activity datasets are built based on the assumption that only the human-object interaction (HOI) activities, performed by the camera-wearer, are captured in the field of view. Since humans live in complicated scenarios, in addition to the first-person activities, it is likely that third-person activities performed by other people also appear. Analyzing and recognizing these two types of activities simultaneously occurring in a scene is important for the camera-wearer to understand the surrounding environments. To facilitate the research on concurrent first- and third-person activity recognition (CFT-AR), we first created a new activity dataset, namely PolyU concurrent first- and third-person (CFT) Daily, which exhibits distinct properties and challenges, compared with previous activity datasets. Since temporal asynchronism and appearance gap usually exist between the first- and third-person activities, it is crucial to learn robust representations from all the activity-related spatio-temporal positions. Thus, we explore both holistic scene-level and local instance-level (person-level) features to provide comprehensive and discriminative patterns for recognizing both first- and third-person activities. On the one hand, the holistic scene-level features are extracted by a 3-D convolutional neural network, which is trained to mine shared and sample-unique semantics between video pairs, via two well-designed attention-based modules and a self-knowledge distillation (SKD) strategy. On the other hand, we further leverage the extracted holistic features to guide the learning of instance-level features in a disentangled fashion, which aims to discover both spatially conspicuous patterns and temporally varied, yet critical, cues. Experimental results on the PolyU CFT Daily dataset validate that our method achieves the state-of-the-art performance.

Slow Feature Analysis for Multi-Camera Activity Understanding

Slow Feature Analysis for Human Action Recognition

Detecting Group Activities with Multi-Camera Context

Multiple Person Tracking Based on Slow Feature Analysis

Video object matching across multiple non-overlapping camera views based on multi-feature fusion and incremental learning.

Foreground/Background-Masked Interaction Learning for Spatio-temporal Action Detection

A spatiotemporal multi-feature extraction framework with space and channel based squeeze-and-excitation blocks for human activity recognition

Anatomy of a Multicamera Video Surveillance System

Action Recognition by Learning Temporal Slowness Invariant Features

M&M: Recognizing Multiple Co-evolving Activities from Multi-source Videos

Camera Matching Based on Spatiotemporal Activity and Conditional Random Field Model

Slow Down to Go Better: A Survey on Slow Feature Analysis

Few-Shot Continual Learning for Activity Recognition in Classroom Surveillance Images

Automatic Object Classification Using Motion Blob Based Local Feature Fusion for Traffic Scene Surveillance

Latent Feature Learning for Activity Recognition Using Simple Sensors in Smart Homes

Human Action Recognition Using Modified Slow Feature Analysis and Multiple Kernel Learning

Action Recognition Based on A Selective Sampling Strategy for Real-Time Video Surveillance

SAInf: Stay Area Inference of Vehicles Using Surveillance Camera Records

Action-Stage Emphasized Spatiotemporal VLAD for Video Action Recognition

Multi-object Events Recognition from Video Sequences Using Extended Finite State Machine

Holistic-Guided Disentangled Learning with Cross-Video Semantics Mining for Concurrent First-Person and Third-Person Activity Recognition.