Abstract:Assembly positioning by visual servoing (VS) is a basis for autonomous robotic assembly. In practice, VS control suffers potential stability and convergence problems due to image and physical constraints, e.g., field of view constraints, image local minima, obstacle collisions, and occlusion. Therefore, this article proposes a novel deep reinforcement learning-based hybrid visual servoing (DRL-HVS) controller for motion planning of VS tasks. DRL-HVS controller takes current observed image features and camera pose as inputs, and the core parameters of hybrid VS are dynamically optimized using a deep deterministic policy gradient (DDPG) algorithm to obtain an optimal motion scheme, considering image/physical constraints and robot motion performance. In addition, an adaptive exploration strategy is proposed to further improve the training efficiency by adaptively tuning the exploration noise parameters. In this way, the offline pretrained DRL-HVS controller in the virtual environment, where the DDPG actor–critic network is continuously optimized, can be quickly deployed to a real robot system for real-time control. Experiments based on an eye-in-hand VS system are conducted with a calibrated HIKVISION RGB camera mounted on the end-effector of a GSK-RB03A1 six degree-of-freedom (6-DoF) robot. Basic VS task experiments show that the proposed controller achieves better performance than the existing methods: the servoing time is 24% smaller than that of the five-dimensional VS method, a 100% success rate with the perturbed ranges of the initial position within 25 mm for translation and 20° for rotation, and a 48% efficiency improvement. Moreover, a planetary gear component assembly process case study, where the robot aims to automatically put the gears on the gear shafts, is conducted to demonstrate the applicability of the proposed method in practice.

End-to-End Deep Reinforcement Learning and Control with Multimodal Perception for Planetary Robotic Dual Peg-in-Hole Assembly

Mapless Collaborative Navigation for a Multi-Robot System Based on the Deep Reinforcement Learning

A Motion Planning Method for Visual Servoing Using Deep Reinforcement Learning in Autonomous Robotic Assembly

Research on Dual-Arm Control of Lunar Assisted Robot Based on Hierarchical Reinforcement Learning under Unstructured Environment

Leveraging the Efficiency of Multi-Task Robot Manipulation Via Task-Evoked Planner and Reinforcement Learning

Coordinated Control Based on Reinforcement Learning for Dual-Arm Continuum Manipulators in Space Capture Missions

Path Planning of 6-DOF Free-Floating Space Robotic Manipulators Using Reinforcement Learning

Multimodality Driven Impedance-Based Sim2Real Transfer Learning for Robotic Multiple Peg-in-Hole Assembly

Active compliance control of robot peg-in-hole assembly based on combined reinforcement learning

Knowledge-Driven Deep Deterministic Policy Gradient for Robotic Multiple Peg-in-Hole Assembly Tasks

Deep Reinforcement Learning for Multi-contact Motion Planning of Hexapod Robots

Adaptive Cooperative Detection Method for Unmanned Planetary Vehicles Based on Deep Reinforcement Learning

Multi-Stage Joint Level Motion Planning for Robot Assisted Peg-in-Hole Task

Vision-based robotic peg-in-hole research: integrating object recognition, positioning, and reinforcement learning

Model Predictive Control of a Novel Wheeled–Legged Planetary Rover for Trajectory Tracking

Feedback Deep Deterministic Policy Gradient With Fuzzy Reward for Robotic Multiple Peg-in-Hole Assembly Tasks

Model-Based Reinforcement Learning Inspired by Augmented PD for Robotic Control

Lunar Rover Collaborated Path Planning with Artificial Potential Field-Based Heuristic on Deep Reinforcement Learning

Multi-target Approaching Control of Hyper-redundant Manipulators Using Reinforcement Learning

Multiple peg-in-hole compliant assembly based on a learning-accelerated deep deterministic policy gradient strategy