Abstract:A flexible production model in which humans and robots collaborate has become an urgent need for industrial production. Therefore, research on human-machine integration systems has received more and more attention in recent years. Human-computer interaction is widely used in industrial production and entertainment life, such as industrial control, VR games, etc. In this article, we designed and implemented a human-machine fusion system that meets different interactive tasks. Human movement estimation and robot interactive control are the key technologies. First, we propose a human posture solution method based on information fusion of multiple cameras and inertial measurement units. This method constructs an optimization problem, fuses the 2D joint detection information under multiple cameras and the measurement information of the worn inertial measurement unit, and optimizes the estimation of human kinematic posture, which improves the problems of incomplete posture information and sensitivity to noise under a single sensor. Therefore, this method improves the accuracy of pose estimation. Secondly, combining the kinematic characteristics of the robot and the characteristics of human-computer interaction, we designed a robot control strategy based on target point tracking and model predictive control, so that the robot can adapt to dynamic environments and different interaction needs by adjusting control parameters. At the same time, our design ensures the safety of robots and operators. Finally, we conducted human-computer interaction experiments such as action following, item delivery, and active obstacle avoidance. The experimental results show the effectiveness and reliability of the designed robot interaction system in a human-machine fusion environment.

ACP theory based plant human machine interaction evaluation

Simulative Evaluation on Human Machine Interface in DCS Main Control Room of Power Plant

Simulation based multi-experts evaluation method for human-machine interface design of main control room

Human-machine Interface for a Master-Slave Hydraulic Manipulator with Vision Enhancement and Auditory Feedback

A Computer-Aided Layout Method for Human Machine Interface of Main Control Room

Intelligent ACP Based Experimental Approach

Human machine interface (HMI) based on a multi-agent system in a water purification plant

Design for human-machine interface in the main control rooms of nuclear power plants

Human Performance Measures for the Evaluation of Process Control Human-System Interfaces in High-Fidelity Simulations

The development and design of human machine interface of nuclear power plant operator support system

The Development and Design of Human Machine Interface of the Monitor& Control System for Nuclear Power Plants

Research on Fuzzy Comprehensive Evaluation of Human-Machine Interface Layout of Driller Control Room

Design of Human-Machine Integration System to Meet Diverse Interactive Tasks

Establishment Of Evaluation System For Human-Machine Interface In Agricultural Machinery Cab

Study on HRA-based Method for Assessing Digital Man-machine Interface

Evaluation of Human-Machine Interface Based on Human Factor Indicator

Improving Human-System Interface Design Through Human Behavior Assessment in the Control Room of Nuclear Power Plants

A FBS Approach to Designing Monitoring System in Nuclear Power Plant

A Multi-disciplinary Assessments Tool for Human-Machine Interaction

Human-Machine Interaction In Future Nuclear Power Plant Control Rooms - A Review

Influence of Nuclear Power Plant Interface Complexity on Operator Performance: A Modeling Study