Abstract:Robots have become increasingly prevalent in various aspects of daily life. In an increasing number of scenarios, particularly in social contexts, such as those involving delivery robots and self-driving buses, human-robot interaction (HRI) is essential for conveying decision commands to the robot's control algorithm. The growing convergence of human activity spaces and robot workspaces presents increasingly emergent and challenging interaction scenarios. This transformation necessitates convenient measurement sensor and gesture recognition algorithms with shorter recognition delays and more reliable recognition results. In pursuit of this objective, we propose a novel pipeline for continuous and rapid gesture recognition employing a portable inertial measurement unit (IMU) device. First, we design a lightweight neural network (NN) to distinguish the gesture from daily hand movement (gesture spotting). Then, one lookback window evaluation is inserted between gesture spotting and recognition, aiming to reduce the false positive rate of gesture spotting. Finally, a novel transformer-based model is used to recognize expected hand gestures (gesture recognition), which can consider both cross-time and cross-sensor dependencies. In this study, the IMU data of six gestures are collected from 20 participants for training and validation. The experimental results show that the proposed gesture spotting model detects all gesture movements in the test data with a 0.3% false positive rate. The gesture recognition model achieves better accuracy than multiple state-of-the-art models. Overall, this study proposes an efficient and reliable pipeline for single IMU-based continuous gesture recognition for practical applications.

WristCam: A Wearable Sensor for Hand Trajectory Gesture Recognition and Intelligent Human–Robot Interaction

Finger Angle-Based Hand Gesture Recognition for Smart Infrastructure Using Wearable Wrist-Worn Camera

A Novel Gesture Recognition System for Intelligent Interaction with a Nursing-Care Assistant Robot

Real-time Dynamic Gesture Recognition and Hand Servo Tracking Using PTZ Camera

Hand Posture Recognition and Tracking Based on Bag-of-Words for Human Robot Interaction

A Sensor Glove Based on Inertial Measurement Unit for Robot Teleoperetion.

Gesture recognition using combination of acceleration sensor and images for casual communication between robots and humans

Static and Dynamic Human Arm/Hand Gesture Capturing and Recognition Via Multiinformation Fusion of Flexible Strain Sensors

Recognizing Hand Gestures with Pressure-Sensor-Based Motion Sensing.

Wrist-Worn Hand Gesture Recognition While Walking Via Transfer Learning.

Development of a Real-Time Hand Gesture Recognition Wristband Based on Semg and IMU Sensing.

Hand Gesture Recognition and Real-time Game Control Based on A Wearable Band with 6-Axis Sensors.

Hand Gesture Recognition using Deep Feature Fusion Network based on Wearable Sensors

Hand Gesture Recognition Framework Using a Lie Group Based Spatio-Temporal Recurrent Network with Multiple Hand-Worn Motion Sensors

A Novel PPG-FMG-ACC Wristband for Hand Gesture Recognition

Feasibility of Wrist-Worn, Real-Time Hand, and Surface Gesture Recognition Via Semg and IMU Sensing

A pilot study on hand posture recognition from wrist-worn camera for human machine interaction

Soft wrist-worn multi-functional sensor array for real-time hand gesture recognition

Dynamic Hand Gesture-Featured Human Motor Adaptation in Tool Delivery using Voice Recognition

Continuous Hand Gestures Detection and Recognition in Emergency Human-Robot Interaction Based on the Inertial Measurement Unit

A Real-Time Approach to the Spotting, Representation, and Recognition of Hand Gestures for Human–Computer Interaction