Abstract:In industrial scenarios, effective human-robot collaboration relies on multi-camera systems to robustly monitor human operators despite the occlusions that typically show up in a robotic workcell. In this scenario, precise localization of the person in the robot coordinate system is essential, making the hand-eye calibration of the camera network critical. This process presents significant challenges when high calibration accuracy should be achieved in short time to minimize production downtime, and when dealing with extensive camera networks used for monitoring wide areas, such as industrial robotic workcells. Our paper introduces an innovative and robust multi-camera hand-eye calibration method, designed to optimize each camera's pose relative to both the robot's base and to each other camera. This optimization integrates two types of key constraints: i) a single board-to-end-effector transformation, and ii) the relative camera-to-camera transformations. We demonstrate the superior performance of our method through comprehensive experiments employing the METRIC dataset and real-world data collected on industrial scenarios, showing notable advancements over state-of-the-art techniques even using less than 10 images. Additionally, we release an open-source version of our multi-camera hand-eye calibration algorithm at <a class="link-external link-https" href="https://github.com/davidea97/Multi-Camera-Hand-Eye-Calibration.git" rel="external noopener nofollow">this https URL</a>.

What problem does this paper attempt to address?

This paper attempts to solve the multi - camera hand - eye calibration problem encountered when achieving efficient human - robot collaboration in industrial robot workcells. Specifically, the paper focuses on how to accurately locate the position of the operator through a multi - camera system in a large - scale industrial environment, so that the robot can work in coordination with the human operator and robustly monitor the human operator even in the presence of occlusion. To achieve this goal, the following challenges need to be overcome: 1. **High - precision and fast calibration**: Complete high - precision hand - eye calibration in a short time to minimize production downtime. 2. **Handling large - scale camera networks**: For the monitoring requirements of a wide area, handle large - scale networks containing multiple cameras, and the distances between these cameras may be large. 3. **Calibration under limited image conditions**: In an industrial environment, due to space congestion, it is difficult to move the robotic arm, so the number of available images may be small, and accurate calibration needs to be achieved under such conditions. For this reason, the paper proposes an innovative and robust multi - camera hand - eye calibration method, aiming to optimize the pose of each camera relative to the robot base and other cameras. This method integrates two key constraints: - **Single - plate - to - end - effector transformation**: Through a single - plate - to - end - effector transformation, the calibration process is simplified, and the redundancy of determining the transformation for each individual hand - eye calibration is avoided. - **Relative inter - camera transformation**: By using the information of multiple cameras detecting the calibration pattern simultaneously, the relative transformation between cameras is optimized to ensure the consistency of all camera poses, thereby improving the calibration performance and preventing error propagation. The paper demonstrates the superior performance of the proposed method through comprehensive experiments using the METRIC dataset and real - world industrial scenario data, which is significantly better than the existing techniques even when using less than 10 images. In addition, the authors also open - sourced their multi - camera hand - eye calibration algorithm.

Multi-Camera Hand-Eye Calibration for Human-Robot Collaboration in Industrial Robotic Workcells

AN AUTONOMOUS HAND-EYE CALIBRATION METHOD

Robust Kinematic Calibration for Improving Collaboration Accuracy of Dual-Arm Manipulators with Experimental Validation

A Graph-based Optimization Framework for Hand-Eye Calibration for Multi-Camera Setups

EasyHeC: Accurate and Automatic Hand-eye Calibration via Differentiable Rendering and Space Exploration

MEMROC: Multi-Eye to Mobile RObot Calibration

EasyHeC++: Fully Automatic Hand-Eye Calibration with Pretrained Image Models

Automatic Robot Hand-Eye Calibration Enabled by Learning-Based 3D Vision

Automatic self-contained calibration of an industrial dual-arm robot with cameras using self-contact, planar constraints, and self-observation

A Simultaneous Hand-Eye Calibration Method for Hybrid Eye-in-Hand/Eye-to-Hand System

3D stereo estimation and fully automated learning of eye-hand coordination in humanoid robots

Autonomous Robot Calibration for Hand-Eye Coordination

Simultaneous Calibration of the Hand-Eye, Flange-Tool and Robot-Robot Relationship in Dual-Robot Collaboration Systems

Multi-sensor Calibration System for Mobile Robots

Camera calibration issues in robot calibration with eye-on-hand configuration

Towards Robotic Eye Surgery: Marker-free, Online Hand-eye Calibration using Optical Coherence Tomography Images

Hand-Eye Calibration Technology of Intelligent Picking Robot Using Multiple Photoelectric Sensor Information Fusion

Hand-Eye Camera Calibration with an Optical Tracking System

Economical Precise Manipulation and Auto Eye-Hand Coordination with Binocular Visual Reinforcement Learning

Hand-eye Calibration of Dual Mechanical Arms of Explosive Ordnance Disposal Robot

Simultaneously Calibration of Multi Hand-Eye Robot System Based on Graph