Abstract:Wearable ultrasound (US) is a novel sensing approach that shows promise in multiple application domains, and specifically in hand gesture recognition (HGR). In fact, US enables to collect information from deep musculoskeletal structures at high spatiotemporal resolution and high signal-to-noise ratio, making it a perfect candidate to complement surface electromyography for improved accuracy performance and on-the-edge classification. However, existing wearable solutions for US-based gesture recognition are not sufficiently low power for continuous, long-term operation. On top of that, practical hardware limitations of wearable US devices (limited power budget, reduced wireless throughput, and restricted computational power) set the need for the compressed size of models for feature extraction and classification. To overcome these limitations, this article presents a novel end-to-end approach for feature extraction from raw musculoskeletal US data suited for edge computing, coupled with an armband for HGR based on a truly wearable (12 cm2, 9 g), ultralow-power (ULP) (16 mW) US probe. The proposed approach uses a 1-D convolutional autoencoder (CAE) to compress raw US data by 20× while preserving the main amplitude features of the envelope signal. The latent features of the autoencoder are used to train an XGBoost classifier for HGR on datasets collected with a custom US armband, considering armband removal/repositioning in between sessions. Our approach achieves a classification accuracy of 96%. Furthermore, the proposed unsupervised feature extraction approach offers generalization capabilities for intersubject use, as demonstrated by testing the pretrained encoder on a different subject and conducting posttraining analysis, revealing that the operations performed by the encoder are subject-independent. The autoencoder is also quantized to 8-bit integers and deployed on a ULP wearable US probe along with the XGBoost classifier, allowing for a gesture recognition rate ≥ 25 Hz and leading to 21% lower power consumption [at 30 frames/s (FPS)] compared to the conventional approach (raw data transmission and remote processing).

Bayesian Filter-Based Ultrasonic Continuous Hand Gesture Recognition for Low-Resource Platform

Research on Dynamic Gesture Recognition with Low-Pixel ToF-Sensors

A real-time hand gesture recognition method

Efficient High Cross-User Recognition Rate Ultrasonic Hand Gesture Recognition System

Low-Latency Gesture Recognition From Spatial Filtering of Single-Element Ultrasound Signals

Multi-Angle Fusion for Low-Cost Near-Field Ultrasonic in-Air Gesture Recognition

Micro Hand Gesture Recognition System Using Ultrasonic Active Sensing

Wireless Hand Gesture Recognition Based on Continuous-Wave Doppler Radar Sensors

A Lightweight Hand-Gesture Recognition Network With Feature Fusion Prefiltering and FMCW Radar Spatial Angle Estimation

24Ghz FMCW Radar Based Lightweight Real-time Hand Gesture Recognition System

A Lightweight Hand-Gesture Recognition Network with Feature Fusion Pre-filtering and FMCW Radar Spatial Angle Estimation

Efficiently User-Independent Ultrasonic-Based Gesture Recognition Algorithm

UltraGesture: Fine-Grained Gesture Sensing and Recognition

A Robust Hand Gesture Sensing and Recognition Based on Dual-Flow Fusion with FMCW Radar.

Hand Gesture Recognition Using IR-UWB Radar with Spiking Neural Networks

Unsupervised Feature Extraction From Raw Data for Gesture Recognition With Wearable Ultralow-Power Ultrasound

Angle-of-Arrival-Based Gesture Recognition Using Ultrasonic Multi-Frequency Signals

Hand Gesture Recognition Using Ultrasonic Array with Machine Learning

Gesture Recognition System Using 24 GHz FMCW Radar Sensor Realized on Real-Time Edge Computing Platform

Agile gesture recognition for capacitive sensing devices: adapting on-the-job

Joint Bayes Filter: A Hybrid Tracker For Non-Rigid Hand Motion Recognition