Abstract:Multi-sensor based motion tracking is of great interest to the robotics community as it may lessen the need for expensive optical motion capture equipment. However, the traditional convolution algorithms have difficulty adapting to the data due to the changes of joints’ relative position during motion. The time-series networks often used in the past ignore the spatial characteristics of sensors. We tackle this challenge by combining long short-term memory (LSTM) with graph convolution network (GCN), adding the prior knowledge of sensor distribution, and integrating it into the motion law through the adjacency matrix. This article proposes a novel shortcut enhanced LSTM-GCN network (SE-LSTM-GCN). It connects LSTM and GCN in sequence and extracts temporal and spatial features of data. At the same time, the shortcut is used in the network to enhance the output of two middle layers and to restore the filtered information. Our experimental results on two different motion tracking datasets show that the proposed network is able to learn the mapping relationship with better universality, less tracking error, and without increasing much training time, and can better perform human motion tracking tasks. Note to Practitioners —Accurate and real-time multiple soft sensors motion tracking suits are more accepted for their low cost. However, the soft-sensor based motion tracking is not comparable to the traditional optical equipment in prediction error. To this end, we present a novel network shortcut enhanced LSTM-GCN (SE-LSTM-GCN), consisting of shortcuts, long short-term memory (LSTM), and graph convolution network (GCN). The LSTM solves the non-linear and hysteresis of soft strain sensors, and GCN is integrated into the network since the knowledge of sensor location can be put into the adjacency matrix generated by the k-nearest neighbor (KNN). While shortcuts are used to enhance the output of middle layers to form combined features. Experimental results on two public datasets show that the proposed network is superior to competing algorithms in terms of prediction error. The network can be deployed in embedded devices, such as VR gloves to provide a better gaming experience. The current algorithm is based on the relationship between sensor data and distance. In future research, we will focus on adding other human kinematics laws to the network.

Continuous Motion Recognition Using Multiple Time Constant Recurrent Neural Network With A Deep Network Model

Multiple Timescale Recurrent Neural Network with Slow Feature Analysis for Efficient Motion Recognition.

Supervised Multiple Timescale Recurrent Neuron Network Model for Human Action Classification.

Real-time human action classification using a dynamic neural model.

Continuous Timescale Long-Short Term Memory Neural Network for Human Intent Understanding

Human motion based intent recognition using a deep dynamic neural model

A Fast Training Algorithm of Multiple-Timescale Recurrent Neural Network for Agent Motion Generation

Motion Sequence Recognition with Multi-sensors Using Deep Convolutional Neural Network.

Motion-Pose Recurrent Neural Network With Instantaneous Kinematic Descriptor For Skeleton Based Gesture Detection And Recognition

Continuous Action Segmentation and Recognition Using Hybrid Convolutional Neural Network-Hidden Markov Model Model

Recognition of Visually Perceived Compositional Human Actions by Multiple Spatio-Temporal Scales Recurrent Neural Networks

Continuous Human Motion Recognition Using Micro-Doppler Signatures in the Scenario with Micro Motion Interference

Continuous Motion Recognition In Depth Camera Based On Recurrent Neural Networks And Grid-Based Average Depth

A Shortcut Enhanced LSTM-GCN Network for Multi-Sensor Based Human Motion Tracking

Deep Dilation on Multimodality Time Series for Human Activity Recognition.

DMS-GCN: Dynamic Mutiscale Spatiotemporal Graph Convolutional Networks for Human Motion Prediction

Continuous Estimation of Human Joint Angles From sEMG Using a Multi-Feature Temporal Convolutional Attention-Based Network

Predictive Coding for Dynamic Vision : Development of Functional Hierarchy in a Multiple Spatio-Temporal Scales RNN Model

Tracking Human-like Natural Motion Using Deep Recurrent Neural Networks

Continuous Arm Motion Recognition Using Two-stream Spatial-Temporal Neural Network Based on Millimeter Wave Sensor

Motion sensitive network for action recognition in control and decision-making of autonomous systems