Abstract:Visual servoing is a well-known task in robotics. However, there are still challenges when multiple visual sources are combined to accurately guide the robot or occlusions appear. In this paper we present a novel visual servoing approach using hybrid multi-camera input data to lead a robot arm accurately to dynamically moving target points in the presence of partial occlusions. The approach uses four RGBD sensors as Eye-to-Hand (EtoH) visual input, and an arm-mounted stereo camera as Eye-in-Hand (EinH). A Master supervisor task selects between using the EtoH or the EinH, depending on the distance between the robot and target. The Master also selects the subset of EtoH cameras that best perceive the target. When the EinH sensor is used, if the target becomes occluded or goes out of the sensor's view-frustum, the Master switches back to the EtoH sensors to re-track the object. Using this adaptive visual input data, the robot is then controlled using an iterative planner that uses position, orientation and joint configuration to estimate the trajectory. Since the target is dynamic, this trajectory is updated every time-step. Experiments show good performance in four different situations: tracking a ball, targeting a bulls-eye, guiding a straw to a mouth and delivering an item to a moving hand. The experiments cover both simple situations such as a ball that is mostly visible from all cameras, and more complex situations such as the mouth which is partially occluded from some of the sensors.

Active Cooperative Observation of A 3D Moving Target Using Two Dynamical Monocular Vision Sensors

Dynamical vision sensors based active cooperative observation in three dimensional environment

An Active Cooperative Observation Method for Multi-robots in Three Dimensional Environments

Accurate and Real-Time 3-D Tracking for the Following Robots by Fusing Vision and Ultrasonar Information

Multi-view-based Cooperative Observation Method for Space Non-Cooperative Target's Attitude Determination

Vision-based multi-agent cooperative target search

Double-Robot Dynamic Stereo Vision Algorithm Based on UKF and Its Application in Moving Target Tracking

Vision-based Pose and Motion Estimation of Non-cooperative Target for Space Robotic Manipulators

ESMF based multiple UAVs active cooperative observation method in relative velocity coordinates

Directional Monitoring of Multiple Moving Targets by Multiple Unmanned Aerial Vehicles.

Autonomous Robotic Capture of Non-Cooperative Target by Adaptive Extended Kalman Filter Based Visual Servo

A Method of Target Tracking Based on Monocular Vision for Mobile Robot in Unknown Environment

Incremental Visual Servo Control of Robotic Manipulator for Autonomous Capture of Non-Cooperative Target

Multi-Agent Cooperative Target Search

TDOA-based adaptive sensing in multi-agent cooperative target tracking

On-Line Cooperative Observation Based on ESMF in Three-Dimensional Enviroments

Predictive visual servo kinematic control for autonomous robotic capture of non-cooperative space target

Multi-UAV cooperative observation based on ESMF in 3D environments

Fair Observation of Multiple Moving Targets in Cooperative Multi-UAV Systems

Hybrid Multi-camera Visual Servoing to Moving Target

Vision-based Relative Detection and Tracking for Teams of Micro Aerial Vehicles