Abstract:Wearable robotic exoskeletons have emerged as an exciting new treatment tool for disorders affecting mobility; however, the human–machine interface, used by the patient for device control, requires further improvement before robotic assistance and rehabilitation can be widely adopted. One method, made possible through advancements in machine learning technology, is the use of bioelectrical signals, such as electroencephalography (EEG) and electromyography (EMG), to classify the user's actions and intentions. While classification using these signals has been demonstrated for many relevant control tasks, such as motion intention detection and gesture recognition, challenges in decoding the bioelectrical signals have caused researchers to seek methods for improving the accuracy of these models. One such method is the use of EEG–EMG fusion, creating a classification model that decodes information from both EEG and EMG signals simultaneously to increase the amount of available information. So far, EEG–EMG fusion has been implemented using traditional machine learning methods that rely on manual feature extraction; however, new machine learning methods have emerged that can automatically extract relevant information from a dataset, which may prove beneficial during EEG–EMG fusion. In this study, Convolutional Neural Network (CNN) models were developed using combined EEG–EMG inputs to determine if they have potential as a method of EEG–EMG fusion that automatically extracts relevant information from both signals simultaneously. EEG and EMG signals were recorded during elbow flexion–extension and used to develop CNN models based on time–frequency (spectrogram) and time (filtered signal) domain image inputs. The results show a mean accuracy of 80.51 ± 8.07% for a three-class output (33.33% chance level), with an F-score of 80.74%, using time–frequency domain-based models. This work demonstrates the viability of CNNs as a new method of EEG–EMG fusion and evaluates different signal representations to determine the best implementation of a combined EEG–EMG CNN. It leverages modern machine learning methods to advance EEG–EMG fusion, which will ultimately lead to improvements in the usability of wearable robotic exoskeletons.

Enhancing EEG and Semg Fusion Decoding Using a Multi-Scale Parallel Convolutional Network with Attention Mechanism.

A Multiscale Feature Extraction Network Based on Channel-Spatial Attention for Electromyographic Signal Classification

PMF-CNN: Parallel multi-band fusion convolutional neural network for SSVEP-EEG decoding

A Multi-Scale Fusion Convolutional Neural Network Based on Attention Mechanism for the Visualization Analysis of EEG Signals Decoding

A Multiscale Feature Fusion Network Based on Attention Mechanism for Motor Imagery EEG Decoding

EEG-EMG FAConformer: Frequency Aware Conv-Transformer for the fusion of EEG and EMG

Multi-View Fusion Network-Based Gesture Recognition Using sEMG Data

MSFF-Net: Multi-Stream Feature Fusion Network for surface electromyography gesture recognition

EEG Decoding Method Based on Multi-Feature Information Fusion for Spinal Cord Injury

Evaluating Convolutional Neural Networks as a Method of EEG–EMG Fusion

Study on exercise muscle fatigue based on sEMG and ECG data fusion and temporal convolutional network

Fusion Learning for Semg Recognition of Multiple Upper-Limb Rehabilitation Movements

MFFNet: Multi-dimensional Feature Fusion Network based on attention mechanism for sEMG analysis to detect muscle fatigue

A Temporal-Spectral-Based Squeeze-and- Excitation Feature Fusion Network for Motor Imagery EEG Decoding

Temporal Fusion Dynamically Separable Graph Convolutional Network for EEG Motion Intention Decoding Based on Source Information Extraction

Pattern recognition of surface electromyography based on multi-scale convolutional neural network with attention mechanism.

Attention-Based Parallel Multiscale Convolutional Neural Network for Visual Evoked Potentials EEG Classification.

Attention-based convolutional neural network with multi-modal temporal information fusion for motor imagery EEG decoding

MI-MBFT: Superior Motor Imagery Decoding of Raw EEG Data Based on a Multi-Branch and Fusion Transformer Framework

MCFHNet: Multi-Channel Fusion Hybrid Network for Efficient EEG-fNIRS Multi-modal Motor Imagery Decoding

EEG-FMCNN: A fusion multi-branch 1D convolutional neural network for EEG-based motor imagery classification