Abstract:Action recognition is a key algorithmic part of emerging on-the-edge smart video surveillance and security systems. Skeleton-based action recognition is an attractive approach which, instead of using RGB pixel data, relies on human pose information to classify appropriate actions. However, existing algorithms often assume ideal conditions that are not representative of real-world limitations, such as noisy input, latency requirements, and edge resource constraints. To address the limitations of existing approaches, this paper presents Real-World Graph Convolution Networks (RW-GCNs), an architecture-level solution for meeting the domain constraints of Real World Skeleton-based Action Recognition. Inspired by the presence of feedback connections in the human visual cortex, RW-GCNs leverage attentive feedback augmentation on existing near state-of-the-art (SotA) Spatial-Temporal Graph Convolution Networks (ST-GCNs). The ST-GCNs' design choices are derived from information theory-centric principles to address both the spatial and temporal noise typically encountered in end-to-end real-time and on-the-edge smart video systems. Our results demonstrate RW-GCNs' ability to serve these applications by achieving a new SotA accuracy on the NTU-RGB-D-120 dataset at 94.1%, and achieving 32X less latency than baseline ST-GCN applications while still achieving 90.4% accuracy on the Northwestern UCLA dataset in the presence of spatial keypoint noise. RW-GCNs further show system scalability by running on the 10X cost effective NVIDIA Jetson Nano (as opposed to NVIDIA Xavier NX), while still maintaining a respectful range of throughput (15.6 to 5.5 Actions per Second) on the resource constrained device. The code is available here: <a class="link-external link-https" href="https://github.com/TeCSAR-UNCC/RW-GCN" rel="external noopener nofollow">this https URL</a>.

Improving Action Recognition with the Graph-Neural-Network-based Interaction Reasoning

An Attentional Spatial Temporal Graph Convolutional Network with Co-Occurrence Feature Learning for Action Recognition

Human Action Recognition Based on Three-Stream Network with Frame Sequence Features

Simultaneous Detection and Interaction Reasoning for Object-Centric Action Recognition

Real-World Graph Convolution Networks (RW-GCNs) for Action Recognition in Smart Video Surveillance

DD-GCN: Directed Diffusion Graph Convolutional Network for Skeleton-based Human Action Recognition

Spatial-Temporal Graph Convolutional Networks for Action Recognition with Adjacency Matrix Generation Network

Enhanced Adjacency Matrix-Based Lightweight Graph Convolution Network for Action Recognition

Richly Activated Graph Convolutional Network for Robust Skeleton-Based Action Recognition

Spatio-Temporal Inception Graph Convolutional Networks for Skeleton-Based Action Recognition.

Multi-channel Network: Constructing Efficient GCN Baselines for Skeleton-Based Action Recognition

Feedback Graph Convolutional Network for Skeleton-Based Action Recognition

Improved Actor Relation Graph based Group Activity Recognition

Multi-view graph convolution network for the recognition of human action with spatial and temporal occlusion problems

Multi-stream P&U adaptive graph convolutional networks for skeleton-based action recognition

Forward-reverse adaptive graph convolutional networks for skeleton-based action recognition

Understanding Spatio-Temporal Relations in Human-Object Interaction using Pyramid Graph Convolutional Network

A hybrid attention-guided ConvNeXt-GRU network for action recognition

Multidimensional Refinement Graph Convolutional Network With Robust Decouple Loss for Fine-Grained Skeleton-Based Action Recognition

Multi-Dimensional Refinement Graph Convolutional Network with Robust Decouple Loss for Fine-Grained Skeleton-Based Action Recognition

Actor-Multi-Scale Context Bidirectional Higher Order Interactive Relation Network for Spatial-Temporal Action Localization.