Abstract:The article explores the possibilities of using hand gestures as a control interface for robotic systems in a collaborative workspace. The development of hand gesture control interfaces has become increasingly important in everyday life as well as professional contexts such as manufacturing processes. We present a system designed to facilitate collaboration between humans and robots in manufacturing processes that require frequent revisions of the robot path and that allows direct definition of the waypoints, which differentiates our system from the existing ones. We introduce a novel and intuitive approach to human-robot cooperation through the use of simple gestures. As part of a robotic workspace, a proposed interface was developed and implemented utilising three RGB-D sensors for monitoring the operator's hand movements within the workspace. The system employs distributed data processing through multiple Jetson Nano units, with each unit processing data from a single camera. MediaPipe solution is utilised to localise the hand landmarks in the RGB image, enabling gesture recognition. We compare the conventional methods of defining robot trajectories with their developed gesture-based system through an experiment with 20 volunteers. The experiment involved verification of the system under realistic conditions in a real workspace closely resembling the intended industrial application. Data collected during the experiment included both objective and subjective parameters. The results indicate that the gesture-based interface enables users to define a given path objectively faster than conventional methods. We critically analyse the features and limitations of the developed system and suggest directions for future research. Overall, the experimental results indicate the usefulness of the developed system as it can speed up the definition of the robot's path.

A system for autonomous execution of hand-eye tasks-an approach to cooperative hand-eye programming and active visions sensing

AN AUTONOMOUS HAND-EYE CALIBRATION METHOD

Perception of Demonstration for Automatic Programing of Robotic Assembly: Framework, Algorithm, and Validation

Automated tracking and grasping of a moving object with a robotic hand-eye system

EyeSight Hand: Design of a Fully-Actuated Dexterous Robot Hand with Integrated Vision-Based Tactile Sensors and Compliant Actuation

Space Robot Teleoperation Based on Active Vision

Autonomous Robot Calibration for Hand-Eye Coordination

Human-Machine Interface for Remote Training of Robot Tasks

Persistent Human–Machine Interfaces for Robotic Arm Control Via Gaze and Eye Direction Tracking

A Modular Software Framework for Eye–Hand Coordination in Humanoid Robots

Biologically inspired automatic construction of cross-modal mapping in robotic eye/hand systems

3D stereo estimation and fully automated learning of eye-hand coordination in humanoid robots

A Vision-Based Fully-Automatic Calibration Method for Hand-Eye Serial Robot

Interactive Task Encoding System for Learning-from-Observation

Eye-Hand Relations for Sensor Placement and for Objects Location Determination

Hand Gesture Interface for Robot Path Definition in Collaborative Applications: Implementation and Comparative Study

An active vision system for navigation in a natural environment

Hold My Hand: Development of a Force Controller and System Architecture for Joint Walking with a Companion Robot

Object-Independent Human-to-Robot Handovers Using Real Time Robotic Vision

A Solution of Human-Computer Remote Interaction with Tactile Feedback

Research of Hand–Eye System with 3D Vision towards Flexible Assembly Application