Abstract:This article describes a multimodal fusion data acquisition and processing system about electromyography for dynamic movement recognition and bioelectrical impedance for key posture recognition. In addition, a new dynamic–static fusion algorithm strategy is designed. Experimental results demonstrate that proposed algorithm significantly improves the accuracy of movement recognition, and has been successfully applied to the remote control of Universal Robot. Accurate recognition of human dynamic movement is essential for seamless human–machine interaction (HMI) across various domains. However, most of the existing methods are single‐modal movement recognition, which has inherent limitations, such as limited feature representation and instability to noise, which will affect its practical performance. To address these limitations, this article proposes a novel fusion approach that can integrate two biological signals, including electromyography (EMG) and bioelectrical impedance (BI). The fusion method combines EMG for capturing dynamic movement features and BI for discerning key postures representing discrete points within dynamic movements. In this method, the identification of key postures and their temporal sequences provide a guiding framework for the selection and weighted correction of probability prediction matrices in EMG‐based dynamic recognition. To verify the effectiveness of the method, six dynamic upper limb movements and nine key postures are defined, and a Universal Robot that can follow movements is employed for experimental validation. Experimental results demonstrate that the recognition accuracy of the dynamic movement reaches 96.2%, representing an improvement of nearly 10% compared with single‐modal signal. This study illustrates the potential of multimodal fusion of EMG and BI in movement recognition, with broad prospects for application in HMI fields.

Mechanical Acoustic Signal Assisted Translational Model for Industrial Human-Machine Interaction

Human-machine Interface for a Master-Slave Hydraulic Manipulator with Vision Enhancement and Auditory Feedback

Real-time FDM Machine Condition Monitoring and Diagnosis Based on Acoustic Emission and Hidden Semi-Markov Model

Intention recognition-based human–machine interaction for mixed flow assembly

A Digital Twin-Driven Guidance Method for Human-Machine Collaborative Assembly Operations Based on Machine Learning and Computer Vision

Toward a Human-Machine Interface Based on Electrical Impedance Tomography for Robotic Manipulator Control.

Design of Human-Machine Integration System to Meet Diverse Interactive Tasks

Human-Machine Interaction Systems for Training Industrial Robots

Intelligent Malfunction Identification Method in Mechanical Manufacturing Process Based on Multisensor Data

Face-to-Machine Proximity Estimation for Mobile Industrial Human Machine Interaction

Exploring the transformation of user interactions to Adaptive Human-Machine Interfaces

A High‐Precision Dynamic Movement Recognition Algorithm Using Multimodal Biological Signals for Human–Machine Interaction

Hidden Markov Model Deep Learning Architecture for Virtual Reality Assessment to Compute Human–Machine Interaction-Based Optimization Model

Human-Manipulator Interface Using Hybrid Sensors Via CMAC for Dual Robots.

Latency Mitigation Using Applied HMMs for Mixed Reality-Enhanced Intuitive Teleoperation in Intelligent Robotic Welding

A Learning-Based Multiscale Modelling Approach to Real-Time Serial Manipulator Kinematics Simulation.

A Multi‐Module Sensing and Bi‐Directional HMI Integrating Interaction, Recognition, and Feedback for Intelligent Robots

Human motion intention recognition method with visual, audio, and surface electromyography modalities for a mechanical hand in different environments

A Survey of Multimodal Human-Machine Interface.

Application of DHMM to mechanical equipment audio recognition

Mechanomyography signals processing method using multivariate variational mode decomposition