Abstract:Wearable sensor-based Human Action Recognition (HAR) has made significant strides in recent times. However, the accuracy performance of wearable sensor-based HAR is currently still lagging behind that of visual modalities-based systems, such as RGB video and depth data. Although diverse input modalities can provide complementary cues and improve the accuracy performance of HAR, wearable devices can only capture limited kinds of non-visual time series input, such as accelerometers and gyroscopes. This limitation hinders the deployment of multimodal simultaneously using visual and non-visual modality data in parallel on current wearable devices. To address this issue, we propose a novel Physical-aware Cross-modal Adversarial (PCA) framework that utilizes only time-series accelerometer data from four inertial sensors for the wearable sensor-based HAR problem. Specifically, we propose an effective IMU2SKELETON network to produce corresponding synthetic skeleton joints from accelerometer data. Subsequently, we imposed additional constraints on the synthetic skeleton data from a physical perspective, as accelerometer data can be regarded as the second derivative of the skeleton sequence coordinates. After that, the original accelerometer as well as the constrained skeleton sequence were fused together to make the final classification. In this way, when individuals wear wearable devices, the devices can not only capture accelerometer data, but can also generate synthetic skeleton sequences for real-time wearable sensor-based HAR applications that need to be conducted anytime and anywhere. To demonstrate the effectiveness of our proposed PCA framework, we conduct extensive experiments on Berkeley-MHAD, UTD-MHAD, and MMAct datasets. The results confirm that the proposed PCA approach has competitive performance compared to the previous methods on the mono sensor-based HAR classification problem.

Defending Black-Box Skeleton-Based Human Activity Classifiers

BASAR:Black-box Attack on Skeletal Action Recognition

Understanding the Vulnerability of Skeleton-based Human Activity Recognition via Black-box Attack

Post-train Black-box Defense via Bayesian Boundary Correction

Towards Understanding the Adversarial Vulnerability of Skeleton-based Action Recognition

Hard No-Box Adversarial Attack on Skeleton-Based Human Action Recognition with Skeleton-Motion-Informed Gradient

Boosting Adversarial Transferability for Skeleton-based Action Recognition via Exploring the Model Posterior Space

Understanding the Robustness of Skeleton-based Action Recognition under Adversarial Attack

TASAR: Transfer-based Attack on Skeletal Action Recognition

SMART: Skeletal Motion Action Recognition aTtack

Towards Physical World Backdoor Attacks against Skeleton Action Recognition

On the robustness of skeleton detection against adversarial attacks

Physical-aware Cross-modal Adversarial Network for Wearable Sensor-based Human Action Recognition

A sparse attack method on skeleton-based human action recognition for intelligent metaverse application

BadHMP: Backdoor Attack against Human Motion Prediction

Adversarial Attack on Skeleton-based Human Action Recognition

Emotion Loss Attacking: Adversarial Attack Perception for Skeleton based on Multi-dimensional Features

Adversarial Bone Length Attack on Action Recognition

Advancing Human Motion Recognition with SkeletonCLIP++: Weighted Video Feature Integration and Enhanced Contrastive Sample Discrimination

Skeleton Based Action Recognition using a Stacked Denoising Autoencoder with Constraints of Privileged Information

Ensemble-based Blackbox Attacks on Dense Prediction