Abstract:Assembly positioning by visual servoing (VS) is a basis for autonomous robotic assembly. In practice, VS control suffers potential stability and convergence problems due to image and physical constraints, e.g., field of view constraints, image local minima, obstacle collisions, and occlusion. Therefore, this article proposes a novel deep reinforcement learning-based hybrid visual servoing (DRL-HVS) controller for motion planning of VS tasks. DRL-HVS controller takes current observed image features and camera pose as inputs, and the core parameters of hybrid VS are dynamically optimized using a deep deterministic policy gradient (DDPG) algorithm to obtain an optimal motion scheme, considering image/physical constraints and robot motion performance. In addition, an adaptive exploration strategy is proposed to further improve the training efficiency by adaptively tuning the exploration noise parameters. In this way, the offline pretrained DRL-HVS controller in the virtual environment, where the DDPG actor–critic network is continuously optimized, can be quickly deployed to a real robot system for real-time control. Experiments based on an eye-in-hand VS system are conducted with a calibrated HIKVISION RGB camera mounted on the end-effector of a GSK-RB03A1 six degree-of-freedom (6-DoF) robot. Basic VS task experiments show that the proposed controller achieves better performance than the existing methods: the servoing time is 24% smaller than that of the five-dimensional VS method, a 100% success rate with the perturbed ranges of the initial position within 25 mm for translation and 20° for rotation, and a 48% efficiency improvement. Moreover, a planetary gear component assembly process case study, where the robot aims to automatically put the gears on the gear shafts, is conducted to demonstrate the applicability of the proposed method in practice.

Space Non-cooperative Object Active Tracking with Deep Reinforcement Learning

Space Noncooperative Object Active Tracking with Deep Reinforcement Learning.

On Deep Recurrent Reinforcement Learning for Active Visual Tracking of Space Noncooperative Objects

Robust Active Visual Tracking of Space Non-Cooperative Objects

On Deep Recurrent Reinforcement Learning for Active Visual Tracking of Space Noncooperative Objects

End-to-End Active Object Tracking and Its Real-World Deployment Via Reinforcement Learning

Enhancing Continuous Control of Mobile Robots for End-to-End Visual Active Tracking

End-to-end Active Object Tracking Via Reinforcement Learning

UAV Dynamic Object Tracking with Lightweight Deep Vision Reinforcement Learning

A Motion Planning Method for Visual Servoing Using Deep Reinforcement Learning in Autonomous Robotic Assembly

Deep Reinforcement Learning Multi-UAV Trajectory Control for Target Tracking

UAV Target Tracking Method Based on Deep Reinforcement Learning

Deep Reinforcement Learning-Based DQN Agent Algorithm for Visual Object Tracking in a Virtual Environmental Simulation

Visual Tracking Via Hierarchical Deep Reinforcement Learning

End-to-End Binocular Active Visual Tracking Based on Reinforcement Learning

A Learning System for Motion Planning of Free-Float Dual-Arm Space Manipulator towards Non-Cooperative Object

A General Visual-Servoing Control Strategy for Active Non-Cooperative Target Tracking of Space Manipulators

An Improved Method Based on Deep Reinforcement Learning for Target Searching

Long-Term Tracking of Evasive Urban Target Based on Intention Inference and Deep Reinforcement Learning

Standoff Target Tracking for Networked UAVs with Specified Performance Via Deep Reinforcement Learning