Abstract:Assembly positioning by visual servoing (VS) is a basis for autonomous robotic assembly. In practice, VS control suffers potential stability and convergence problems due to image and physical constraints, e.g., field of view constraints, image local minima, obstacle collisions, and occlusion. Therefore, this article proposes a novel deep reinforcement learning-based hybrid visual servoing (DRL-HVS) controller for motion planning of VS tasks. DRL-HVS controller takes current observed image features and camera pose as inputs, and the core parameters of hybrid VS are dynamically optimized using a deep deterministic policy gradient (DDPG) algorithm to obtain an optimal motion scheme, considering image/physical constraints and robot motion performance. In addition, an adaptive exploration strategy is proposed to further improve the training efficiency by adaptively tuning the exploration noise parameters. In this way, the offline pretrained DRL-HVS controller in the virtual environment, where the DDPG actor–critic network is continuously optimized, can be quickly deployed to a real robot system for real-time control. Experiments based on an eye-in-hand VS system are conducted with a calibrated HIKVISION RGB camera mounted on the end-effector of a GSK-RB03A1 six degree-of-freedom (6-DoF) robot. Basic VS task experiments show that the proposed controller achieves better performance than the existing methods: the servoing time is 24% smaller than that of the five-dimensional VS method, a 100% success rate with the perturbed ranges of the initial position within 25 mm for translation and 20° for rotation, and a 48% efficiency improvement. Moreover, a planetary gear component assembly process case study, where the robot aims to automatically put the gears on the gear shafts, is conducted to demonstrate the applicability of the proposed method in practice.

A Learning-Based Two-Stage Method for Submillimeter Insertion Tasks with Only Visual Inputs

Learning Robot Manipulation Skills from Human Demonstration Videos Using Two-Stream 2-D/3-D Residual Networks with Self-Attention

PEPL: A Two-Stage Method for Submillimeter Insertion Tasks with Pose Estimation and Primitives Learning

Ensemble Bootstrapped Deep Deterministic Policy Gradient For Vision-Based Robotic Grasping

Safe Self-Supervised Learning in Real of Visuo-Tactile Feedback Policies for Industrial Insertion

A Practical Approach to Insertion with Variable Socket Position Using Deep Reinforcement Learning

Vision-Based Robotic Object Grasping—A Deep Reinforcement Learning Approach

InsertionNet 2.0: Minimal Contact Multi-Step Insertion Using Multimodal Multiview Sensory Input

Reinforcement Learning Strategy Based on Multimodal Representations for High-Precision Assembly Tasks

Leveraging Multi-modal Sensing for Robotic Insertion Tasks in R&D Laboratories

A Motion Planning Method for Visual Servoing Using Deep Reinforcement Learning in Autonomous Robotic Assembly

Vision-based robotic peg-in-hole research: integrating object recognition, positioning, and reinforcement learning

A Task-Learning Strategy for Robotic Assembly Tasks from Human Demonstrations

Skill Learning for Robotic Insertion Based on One-shot Demonstration and Reinforcement Learning

Learning to Fill the Seam by Vision: Sub-millimeter Peg-in-hole on Unseen Shapes in Real World

InsertionNet -- A Scalable Solution for Insertion

Learning Tactile Insertion in the Real World

Stage-Wise Learning of Reaching Using Little Prior Knowledge

Skill Learning in Robot-Assisted Micro-Manipulation Through Human Demonstrations with Attention Guidance

Integrating Vision Localization and Deep Reinforcement Learning for High-Precision, Low-Cost Peg-in-Hole Assembly

Iterative Visual Recognition for Learning Based Randomized Bin-Picking