Abstract:Wearable robotic exoskeletons have emerged as an exciting new treatment tool for disorders affecting mobility; however, the human–machine interface, used by the patient for device control, requires further improvement before robotic assistance and rehabilitation can be widely adopted. One method, made possible through advancements in machine learning technology, is the use of bioelectrical signals, such as electroencephalography (EEG) and electromyography (EMG), to classify the user's actions and intentions. While classification using these signals has been demonstrated for many relevant control tasks, such as motion intention detection and gesture recognition, challenges in decoding the bioelectrical signals have caused researchers to seek methods for improving the accuracy of these models. One such method is the use of EEG–EMG fusion, creating a classification model that decodes information from both EEG and EMG signals simultaneously to increase the amount of available information. So far, EEG–EMG fusion has been implemented using traditional machine learning methods that rely on manual feature extraction; however, new machine learning methods have emerged that can automatically extract relevant information from a dataset, which may prove beneficial during EEG–EMG fusion. In this study, Convolutional Neural Network (CNN) models were developed using combined EEG–EMG inputs to determine if they have potential as a method of EEG–EMG fusion that automatically extracts relevant information from both signals simultaneously. EEG and EMG signals were recorded during elbow flexion–extension and used to develop CNN models based on time–frequency (spectrogram) and time (filtered signal) domain image inputs. The results show a mean accuracy of 80.51 ± 8.07% for a three-class output (33.33% chance level), with an F-score of 80.74%, using time–frequency domain-based models. This work demonstrates the viability of CNNs as a new method of EEG–EMG fusion and evaluates different signal representations to determine the best implementation of a combined EEG–EMG CNN. It leverages modern machine learning methods to advance EEG–EMG fusion, which will ultimately lead to improvements in the usability of wearable robotic exoskeletons.

Bio-signal based motion control system using deep learning models: a deep learning approach for motion classification using EEG and EMG signal fusion

Enhanced Recognition of Amputated Wrist and Hand Movements by Deep Learning Method Using Multimodal Fusion of Electromyography and Electroencephalography

Sensor Fusion for Myoelectric Control Based on Deep Learning With Recurrent Convolutional Neural Networks

EEG-Based Hand Motion Pattern Recognition Using Deep Learning Network Algorithms.

Deep learning approach to control of prosthetic hands with electromyography signals

Upper Limb Movement Recognition utilising EEG and EMG Signals for Rehabilitative Robotics

Gesture Classification in Electromyography Signals for Real-Time Prosthetic Hand Control Using a Convolutional Neural Network-Enhanced Channel Attention Model

Multi-motion Robots Control Based on Bioelectric Signals from Single-Channel Dry Electrode

Deep learning-based framework for real-time upper limb motion intention classification using combined bio-signals

Deep Neural Network for Electromyography Signal Classification Via Wearable Sensors.

EMG-Based Feature Extraction and Classification for Prosthetic Hand Control

Enhancing orthopedics and sports medicine with lower limb exoskeleton control in rehabilitation using deep learning based electromyography signal classification

A novel deep learning approach for classification of EEG motor imagery signals

EMG-Based Estimation of Limb Movement Using Deep Learning With Recurrent Convolutional Neural Networks.

An EMG-Based Deep Learning Approach for Multi-DOF Wrist Movement Decoding

Evaluating Convolutional Neural Networks as a Method of EEG–EMG Fusion

Deep Learning with Convolutional Neural Networks Applied to Electromyography Data: A Resource for the Classification of Movements for Prosthetic Hands

Dynamic Fusion of Electromyographic and Electroencephalographic Data towards Use in Robotic Prosthesis Control

A Systematic Study on Electromyography-Based Hand Gesture Recognition for Assistive Robots Using Deep Learning and Machine Learning Models

Deep learning classification of EEG-based BCI monitoring of the attempted arm and hand movements

Wireless Collaboration Technology Based on Electroencephalograph and Electromyography