Abstract:Action recognition is a key algorithmic part of emerging on-the-edge smart video surveillance and security systems. Skeleton-based action recognition is an attractive approach which, instead of using RGB pixel data, relies on human pose information to classify appropriate actions. However, existing algorithms often assume ideal conditions that are not representative of real-world limitations, such as noisy input, latency requirements, and edge resource constraints. To address the limitations of existing approaches, this paper presents Real-World Graph Convolution Networks (RW-GCNs), an architecture-level solution for meeting the domain constraints of Real World Skeleton-based Action Recognition. Inspired by the presence of feedback connections in the human visual cortex, RW-GCNs leverage attentive feedback augmentation on existing near state-of-the-art (SotA) Spatial-Temporal Graph Convolution Networks (ST-GCNs). The ST-GCNs' design choices are derived from information theory-centric principles to address both the spatial and temporal noise typically encountered in end-to-end real-time and on-the-edge smart video systems. Our results demonstrate RW-GCNs' ability to serve these applications by achieving a new SotA accuracy on the NTU-RGB-D-120 dataset at 94.1%, and achieving 32X less latency than baseline ST-GCN applications while still achieving 90.4% accuracy on the Northwestern UCLA dataset in the presence of spatial keypoint noise. RW-GCNs further show system scalability by running on the 10X cost effective NVIDIA Jetson Nano (as opposed to NVIDIA Xavier NX), while still maintaining a respectful range of throughput (15.6 to 5.5 Actions per Second) on the resource constrained device. The code is available here: <a class="link-external link-https" href="https://github.com/TeCSAR-UNCC/RW-GCN" rel="external noopener nofollow">this https URL</a>.

Design and Hardware Implementation of CNN-GCN Model for Skeleton-Based Human Action Recognition

Design of Convolutional Neural Network Based on FPGA

AR-C3D: Action Recognition Accelerator for Human-Computer Interaction on FPGA

Skeleton-based Human Action Recognition via Convolutional Neural Networks (CNN)

Human Skeleton Feature Optimizer and Adaptive Structure Enhancement Graph Convolution Network for Action Recognition

Temporal Attention-Augmented Graph Convolutional Network for Efficient Skeleton-Based Human Action Recognition

Multi-scale Gated Graph Convolutional Network for Skeleton-based Action Recognition

A Real-Time and Hardware-Efficient Processor for Skeleton-Based Action Recognition with Lightweight Convolutional Neural Network

Spatio-Temporal Inception Graph Convolutional Networks for Skeleton-Based Action Recognition.

HARDWARE ACCELERATOR: IMPLEMENTATION OF CNN ON FPGA FOR DIGIT RECOGNITION

Learning Graph Convolutional Network for Skeleton-Based Human Action Recognition by Neural Searching.

A Multi-Stream Graph Convolutional Networks-Hidden Conditional Random Field Model for Skeleton-Based Action Recognition

Attention adjacency matrix based graph convolutional networks for skeleton-based action recognition

A Novel Design of Adaptive and Hierarchical Convolutional Neural Networks using Partial Reconfiguration on FPGA

Real-World Graph Convolution Networks (RW-GCNs) for Action Recognition in Smart Video Surveillance

High-Performance Inference Graph Convolutional Networks for Skeleton-Based Action Recognition

VGG16 Hardware Design and Implementation for CNN in Image Recognition

Multi-Scale Adaptive Aggregate Graph Convolutional Network for Skeleton-Based Action Recognition

Face Recognition with Hybrid Efficient Convolution Algorithms on FPGAs

Fpga-based SoC design for real-time facial point detection using deep convolutional neural networks with dynamic partial reconfiguration

Dynamic Semantic-Based Spatial-Temporal Graph Convolution Network for Skeleton-Based Human Action Recognition