Abstract:Background: Cervical spinal cord injury severely affects grasping ability of its survivors. Fortunately, many individuals with tetraplegia retain residual arm movements that allow them to reach for objects. We propose a wearable technology that utilizes arm movement trajectory information and deep learning methods to determine grasp selection. Furthermore, we combined this approach with neuromuscular stimulation to determine if self-driven functional hand movement could be enabled in spinal cord injury participants. Methods: Two cervical SCI participants performed arbitrary and natural reaching movements toward target objects in three-dimensional space, which were recorded using an inertial sensor worn on their wrist. Time series classifiers were trained to recognize the trajectories using either a Dynamic Time Warping (DTW) algorithm or a Long Short-Term Memory (LSTM) recurrent neural network. As an initial proof-of-concept, we demonstrate real-time classification of the arbitrary movements using DTW only (due to its implementation simplicity), which when used in combination with a high density neuromuscular stimulation sleeve with textile electrodes, enabled participants to perform functional grasping. Results: Participants were able to consistently perform arbitrary two-dimensional and three-dimensional arm movements which could be classified with high accuracy. Furthermore, it was found that natural reaching trajectories for two different target objects (requiring two different grasp types) were distinct and also discriminable with high accuracy. In offline comparisons, LSTM (mean accuracies 99%) performed significantly better than DTW (mean accuracies 86 and 83%) for both arbitrary and natural reaching movements, respectively. Type I and II errors occurred more frequently for DTW (up to 60 and 15%, respectively), whereas it stayed under 5% for LSTM. Also, DTW achieved online accuracy of 79%. Conclusions: We demonstrate the feasibility of utilizing arm trajectory information to determine grasp selection using a wearable inertial sensor along with DTW and deep learning methods. Importantly, this technology can be successfully used to control neuromuscular stimulation and restore functional independence to individuals living with paralysis. Trial registration: NCT, NCT03385005. Registered September 26, 2017.

Reach and grasp by people with tetraplegia using a neurally controlled robotic arm

Assisted Grasping in Individuals with Tetraplegia: Improving Control through Residual Muscle Contraction and Movement

Feel Your Reach: An EEG-Based Framework to Continuously Detect Goal-Directed Movements and Error Processing to Gate Kinesthetic Feedback Informed Artificial Arm Control

Noninvasive Electroencephalogram Based Control of a Robotic Arm for Reach and Grasp Tasks

On the feasibility of using motor imagery EEG-based brain–computer interface in chronic tetraplegics for assistive robotic arm control: a clinical test and long-term post-trial follow-up

Shared control of a robotic arm using non-invasive brain–computer interface and computer vision guidance

Determining grasp selection from arm trajectories via deep learning to enable functional hand movement in tetraplegia

Restoration of Finger and Arm Movements Using Hybrid Brain/Neural Assistive Technology in Everyday Life Environments

Non-invasive Cognitive-level Human Interfacing for the Robotic Restoration of Reaching & Grasping

Use of an arm weight-bearing combined with upper-limb reaching apparatus to facilitate motor paralysis recovery in an incomplete spinal cord injury patient: a single case report

Decoding hand movements from human EEG to control a robotic arm in a simulation environment

Decoding of unimanual and bimanual reach-and-grasp actions from EMG and IMU signals in persons with cervical spinal cord injury

An Exploratory Multi-Session Study of Learning High-Dimensional Body-Machine Interfacing for Assistive Robot Control

Upper Limb Rehabilitation Robot Powered by PAMs Cooperates with FES Arrays to Realize Reach-to-Grasp Trainings

User-Driven Functional Movement Training With a Wearable Hand Robot After Stroke

Robotic Aid for Upper Limb Rehabilitation

Movement-Based Prosthesis Control with Angular Trajectory Is Getting Closer to Natural Arm Coordination

Control of a 7-DOF Robotic Arm System with an SSVEP-Based BCI.

In-home and remote use of robotic body surrogates by people with profound motor deficits

Reconnecting the Hand and Arm to the Brain: Efficacy of Neural Interfaces for Sensorimotor Restoration After Tetraplegia

Decoding unconstrained arm movements in primates using high-density electrocorticography signals for brain-machine interface use