Abstract:The article explores the possibilities of using hand gestures as a control interface for robotic systems in a collaborative workspace. The development of hand gesture control interfaces has become increasingly important in everyday life as well as professional contexts such as manufacturing processes. We present a system designed to facilitate collaboration between humans and robots in manufacturing processes that require frequent revisions of the robot path and that allows direct definition of the waypoints, which differentiates our system from the existing ones. We introduce a novel and intuitive approach to human-robot cooperation through the use of simple gestures. As part of a robotic workspace, a proposed interface was developed and implemented utilising three RGB-D sensors for monitoring the operator's hand movements within the workspace. The system employs distributed data processing through multiple Jetson Nano units, with each unit processing data from a single camera. MediaPipe solution is utilised to localise the hand landmarks in the RGB image, enabling gesture recognition. We compare the conventional methods of defining robot trajectories with their developed gesture-based system through an experiment with 20 volunteers. The experiment involved verification of the system under realistic conditions in a real workspace closely resembling the intended industrial application. Data collected during the experiment included both objective and subjective parameters. The results indicate that the gesture-based interface enables users to define a given path objectively faster than conventional methods. We critically analyse the features and limitations of the developed system and suggest directions for future research. Overall, the experimental results indicate the usefulness of the developed system as it can speed up the definition of the robot's path.

Research of the Gesture Control System for a Industrial Robot

A Sensor Glove Based on Inertial Measurement Unit for Robot Teleoperetion.

Intuitive Control System for Industrial Robot Application Using Smartphone

Entertainment Robot Hand Gesture Recognition

Design of Kinect-Based Human Robot Interaction Systems for A Robocup Middle Size League Soccer Robot

Gesture Recognition Based on Localist Attractor Networks with Application to Robot Control [Application Notes]

Interaction and control with the auxiliary of hand gesture

Gesture and Voice Controlled Robot for Industrial Applications

Using Deep Learning Technology to Realize the Automatic Control Program of Robot Arm Based on Hand Gesture Recognition

High-level programming and control for industrial robotics: using a hand-held accelerometer-based input device for gesture and posture recognition

Hand Gesture Interface for Robot Path Definition in Collaborative Applications: Implementation and Comparative Study

A Novel Gesture Recognition System for Intelligent Interaction with a Nursing-Care Assistant Robot

Hands-On Robotics: Enabling Communication Through Direct Gesture Control

Novel Gesture-Based Robot Programming Approach with the Ability of Code Reuse

Real time robotic arm control using hand gestures

WristCam: A Wearable Sensor for Hand Trajectory Gesture Recognition and Intelligent Human–Robot Interaction

The integration of contactless static pose recognition and dynamic hand motion tracking control system for industrial human and robot collaboration

Design of Three-Dimensional Gesture Recognition and Motion Tracking Human-Computer Intelligent Interaction System based on PAJ7620

Visual gesture recognition for human robot interaction using dynamic movement primitives

Research on Dynamic Gesture Recognition and Control System based on Machine Vision

A Multi-sensor Gesture Interaction System for Human-robot Cooperation