Abstract:Assembly positioning by visual servoing (VS) is a basis for autonomous robotic assembly. In practice, VS control suffers potential stability and convergence problems due to image and physical constraints, e.g., field of view constraints, image local minima, obstacle collisions, and occlusion. Therefore, this article proposes a novel deep reinforcement learning-based hybrid visual servoing (DRL-HVS) controller for motion planning of VS tasks. DRL-HVS controller takes current observed image features and camera pose as inputs, and the core parameters of hybrid VS are dynamically optimized using a deep deterministic policy gradient (DDPG) algorithm to obtain an optimal motion scheme, considering image/physical constraints and robot motion performance. In addition, an adaptive exploration strategy is proposed to further improve the training efficiency by adaptively tuning the exploration noise parameters. In this way, the offline pretrained DRL-HVS controller in the virtual environment, where the DDPG actor–critic network is continuously optimized, can be quickly deployed to a real robot system for real-time control. Experiments based on an eye-in-hand VS system are conducted with a calibrated HIKVISION RGB camera mounted on the end-effector of a GSK-RB03A1 six degree-of-freedom (6-DoF) robot. Basic VS task experiments show that the proposed controller achieves better performance than the existing methods: the servoing time is 24% smaller than that of the five-dimensional VS method, a 100% success rate with the perturbed ranges of the initial position within 25 mm for translation and 20° for rotation, and a 48% efficiency improvement. Moreover, a planetary gear component assembly process case study, where the robot aims to automatically put the gears on the gear shafts, is conducted to demonstrate the applicability of the proposed method in practice.

Hierarchical Reinforcement Learning-Based End-to-End Visual Servoing with Smooth Subgoals

A Motion Planning Method for Visual Servoing Using Deep Reinforcement Learning in Autonomous Robotic Assembly

Toward Scalable Visual Servoing Using Deep Reinforcement Learning and Optimal Control

Policy-Based Deep Reinforcement Learning for Visual Servoing Control of Mobile Robots With Visibility Constraints

Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Control

A Closed-Loop Multi-perspective Visual Servoing Approach with Reinforcement Learning

End-to-end UAV Intelligent Training via Deep Reinforcement Learning

Hierarchical reinforcement learning for handling sparse rewards in multi-goal navigation

Learning Quadrotor Control From Visual Features Using Differentiable Simulation

Bootstrapping Reinforcement Learning with Imitation for Vision-Based Agile Flight

Robust Policies via Mid-Level Visual Representations: An Experimental Study in Manipulation and Navigation

Learning Sparse Control Tasks from Pixels by Latent Nearest-Neighbor-Guided Explorations

Monocular vision guided deep reinforcement learning UAV systems with representation learning perception

A data-efficient goal-directed deep reinforcement learning method for robot visuomotor skill

Reinforcement-Learning-Based Visual Servoing of Underwater Vehicle Dual-Manipulator System

Reinforcement Learning Meets Visual Odometry

Learning to Regrasp Using Visual–Tactile Representation-Based Reinforcement Learning

A survey Of learning-Based control of robotic visual servoing systems

Learning of Long-Horizon Sparse-Reward Robotic Manipulator Tasks With Base Controllers

Robot Subgoal-guided Navigation in Dynamic Crowded Environments with Hierarchical Deep Reinforcement Learning

Towards Generalization and Data Efficient Learning of Deep Robotic Grasping