Abstract:High-density electromyography (HD-EMG) can provide a natural interface to enhance human-computer interaction (HCI). This study aims to demonstrate the capability of a novel HD-EMG forearm sleeve equipped with up to 150 electrodes to capture high-resolution muscle activity, decode complex hand gestures, and estimate continuous hand position via joint angle predictions. Ten able-bodied participants performed 37 hand movements and grasps while EMG was recorded using the HD-EMG sleeve. Simultaneously, an 18-sensor motion capture glove calculated 23 joint angles from the hand and fingers across all movements for training regression models. For classifying across the 37 gestures, our decoding algorithm was able to differentiate between sequential movements with 97.3 ± 0.3 % accuracy calculated on a 100 ms bin-by-bin basis. In a separate mixed dataset consisting of 19 movements randomly interspersed, decoding performance achieved an average bin-wise accuracy of 92.8 ± 0.8 % . When evaluating decoders for use in real-time scenarios, we found that decoders can reliably decode both movements and movement transitions, achieving an average accuracy of 93.3 ± 0.9 % on the sequential set and 88.5 ± 0.9 % on the mixed set. Furthermore, we estimated continuous joint angles from the EMG sleeve data, achieving a R 2 of 0.884 ± 0.003 in the sequential set and 0.750 ± 0.008 in the mixed set. Median absolute error (MAE) was kept below 10° across all joints, with a grand average MAE of 1.8 ± 0 . 04 ∘ and 3.4 ± 0 . 07 ∘ for the sequential and mixed datasets, respectively. We also assessed two algorithm modifications to address specific challenges for EMG-driven HCI applications. To minimize decoder latency, we used a method that accounts for reaction time by dynamically shifting cue labels in time. To reduce training requirements, we show that pretraining models with historical data provided an increase in decoding performance compared with models that were not pretrained when reducing the in-session training data to only one attempt of each movement. The HD-EMG sleeve, combined with sophisticated machine learning algorithms, can be a powerful tool for hand gesture recognition and joint angle estimation. This technology holds significant promise for applications in HCI, such as prosthetics, assistive technology, rehabilitation, and human-robot collaboration.

Muscle Modeling between Upper Arm and Lower Arm for Wearable Haptic Sleeve for Force Feedback of Virtual Reality

Wearable teleoperation controller with 2-DoF robotic arm and haptic feedback for enhanced interaction in virtual reality

Soft Robotic Glove with Sensing and Force Feedback for Rehabilitation in Virtual Reality

Realistic Virtual Hand Modeling with Applications for Virtual Grasping

Haptic Sketches on the Arm for Manipulation in Virtual Reality

A Wearable Haptic Device for the Hand With Interchangeable End-Effectors

A 4-Layer Flexible Virtual Hand Model for Haptic Interaction

Force feedback haptic interface for bilateral teleoperation of robot manipulation

Haptic Feedback Relocation from the Fingertips to the Wrist for Two-Finger Manipulation in Virtual Reality

Skin-Integrated Haptic Interfaces Enabled by Scalable Mechanical Actuators for Virtual Reality

3D Hand Motion Generation for VR Interactions Using a Haptic Data Glove

Force-Aware Interface via Electromyography for Natural VR/AR Interaction

Modelling Contact Forces For 3d Interactive Peg-In-Hole Virtual Reality Operations

A Framework for Virtual Hand Haptic Interaction

Design of EMG-driven Musculoskeletal Model for Volitional Control of a Robotic Ankle Prosthesis

Virtual Reality Based Tactile Sensing Enhancements for Bilateral Teleoperation System with In-Hand Manipulation

Designing a robot-assisted rehabilitation system for hand function recovery using virtual reality and haptic robot

Wearable Haptic Device to Render 360-degree Torque Feedback on the Wrist

All-Fabric Bi-directional Actuators for Multi-joint Assistance of Upper Limb

Characterizing the Sensing Response of Carbon Nanocomposite-Based Wearable Sensors on Elbow Joint Using an End Point Robot and Virtual Reality

Wearable high-density EMG sleeve for complex hand gesture classification and continuous joint angle estimation