Abstract:The International Journal of Robotics Research, Ahead of Print. Understanding human activity is a crucial aspect of developing intelligent robots, particularly in the domain of human-robot collaboration. Nevertheless, existing systems encounter challenges such as over-segmentation, attributed to errors in the up-sampling process of the decoder. In response, we introduce a promising solution: the Temporal Fusion Graph Convolutional Network. This innovative approach aims to rectify the inadequate boundary estimation of individual actions within an activity stream and mitigate the issue of over-segmentation in the temporal dimension. Moreover, systems leveraging human activity recognition frameworks for decision-making necessitate more than just the identification of actions. They require a confidence value indicative of the certainty regarding the correspondence between observations and training examples. This is crucial to prevent overly confident responses to unforeseen scenarios that were not part of the training data and may have resulted in mismatches due to weak similarity measures within the system. To address this, we propose the incorporation of a Spectral Normalized Residual connection aimed at enhancing efficient estimation of novelty in observations. This innovative approach ensures the preservation of input distance within the feature space by imposing constraints on the maximum gradients of weight updates. By limiting these gradients, we promote a more robust handling of novel situations, thereby mitigating the risks associated with overconfidence. Our methodology involves the use of a Gaussian process to quantify the distance in feature space. The final model is evaluated on two challenging public datasets in the field of human-object interaction recognition, that is, Bimanual Actions and IKEA Assembly datasets, and outperforms popular existing methods in terms of action recognition and segmentation accuracy as well as out-of-distribution detection.

Understanding human activity with uncertainty measure for novelty in graph convolutional networks

Understanding Human Activity with Uncertainty Measure for Novelty in Graph Convolutional Networks

Versatile Graph Neural Networks Toward Intuitive Human Activity Understanding

Understanding Spatio-Temporal Relations in Human-Object Interaction using Pyramid Graph Convolutional Network

An Attentional Spatial Temporal Graph Convolutional Network with Co-Occurrence Feature Learning for Action Recognition

GANCDE: Neural networks based on graphs and attention neural control differential equations for human activity recognition

Spatio-Temporal Action Graph Networks

Human–robot interaction-oriented video understanding of human actions

Enhancing human behavior recognition with spatiotemporal graph convolutional neural networks and skeleton sequences

Real-World Graph Convolution Networks (RW-GCNs) for Action Recognition in Smart Video Surveillance

Know Thy Neighbors: A Graph Based Approach for Effective Sensor-Based Human Activity Recognition in Smart Homes

Structured Feature-Graph Model for Human Activity Recognition

Multi-view graph convolution network for the recognition of human action with spatial and temporal occlusion problems

Beyond the Gates of Euclidean Space: Temporal-Discrimination-Fusions and Attention-based Graph Neural Network for Human Activity Recognition

Leveraging spatio-temporal features using graph neural networks for human activity recognition

Efficient Modelling Across Time of Human Actions and Interactions

Human-Aware Collaborative Robots in the Wild: Coping with Uncertainty in Activity Recognition

VCHAR:Variance-Driven Complex Human Activity Recognition framework with Generative Representation

Human Behavior Recognition Based on Attention Mechanism and Bottleneck Residual Dual-Path Spatiotemporal Graph Convolutional Network

A Hybrid Graph Network for Complex Activity Detection in Video

Combining Sparse And Dense Descriptors With Temporal Semantic Structures For Robust Human Action Recognition