Abstract:Purpose of research . Development of a tool for debugging intelligent control system algorithms, including the development of a vision system and planning a software trajectory for an industrial robot. Methods . To achieve this goal, a review of existing simulation tools was carried out. A protocol of contactless humanrobot interaction is presented. An algorithm for the recognition of gesture commands based on the difference of three-dimensional binary microblocks and the construction of the skeleton of the human body has been developed. An example of using the ROBOGuid software tool for imitating the motion of an industrial robot in the development and debugging of its own control methods focused on real objects is presented. Results . The use of digital twins of technological equipment to simulate and display real technological processes in a virtual environment, in the context of the formation of a new concept of Industry 4.0 and the sixth technological order, allows improving the main and auxiliary production processes, as well as analyzing, researching and evaluating the economic efficiency of new technological and technical solutions. Simulation allows the development of ergonomic ways of human interaction with mechatronic objects. The solution proposed in the work was tested on the example of working out a complex spatial contour that simulates the milling of a part. Experimental studies of the gesture command recognition algorithm proposed in the work were carried out on the publicly available UCF101 dataset, the results are compared with known approaches to recognizing human actions. Conclusion . The developed interface module was used on the example of working out a complex spatial contour that simulates the milling of a part, and the method of a contactless robot control system has shown its effectiveness and the need to develop this direction.

Development and Application of Integrated Human Machine Interaction Simulation System

An Integrated Simulation System for Human Factors Study

Development of a Synchronized Human-Robot-Virtuality Interaction System Using Cooperative Robot and Motion Capture Device

Virtual Factory Integrated Manufacturing System for Process Simulation and Monitoring

Virtual-Physical Interaction System

Human-machine Interface for a Master-Slave Hydraulic Manipulator with Vision Enhancement and Auditory Feedback

An interaction system using mixed hand gestures.

Multi-modal Sensing-Based Interactive Glove System for Teleoperation and VR/AR

Work Design Based on Motion Tracking and Interactive Simulation

An Integrated Virtual Design And Virtual Assembly Environment

Design and Development of a Gesture-Based Interaction System for Multi-Projector Tiled Display Wall

Design and Implementation of Virtual Reality Simulation System for Petrochemical Enterprise

Integrating digital human modeling into virtual environment for ergonomic oriented design

3D Intuitive Gesture Interaction via Motion Sensing

A Multi-modal Virtual-Real Fusion System for Multi-task Human-Computer Interaction.

Research on Virtual Human in Ergonomic Simulation

Design of Human-Machine Integration System to Meet Diverse Interactive Tasks

A Multi‐Module Sensing and Bi‐Directional HMI Integrating Interaction, Recognition, and Feedback for Intelligent Robots

A Visual Human-Computer Interaction System Based on Hybrid Visual Model

Design and exploration of a virtual reality-based human-computer interaction system

Virtual Interface Technology in the Process of Simulation of Complex Functional Modules of Control Systems for Industrial Robots and Multi-Axis Mechatronic Systems