Abstract:Background: As an important part of robot operation, peg-in-hole assembly has problems such as a low degree of automation, a large amount of tasks and low efficiency. It is still a huge challenge for robots to automatically complete assembly tasks because the traditional assembly control policy requires complex analysis of the contact model and it is difficult to build the contact model. The deep reinforcement learning method does not require the establishment of complex contact models, but the long training time and low data utilization efficiency make the training costs very high. Methods: With the aim of addressing the problem of how to accurately obtain the assembly policy and improve the data utilization rate of the robot in the peg-in-hole assembly, we propose the Experience Fusion Proximal Policy Optimization algorithm (EFPPO) based on the Proximal Policy Optimization algorithm (PPO). The algorithm improves the assembly speed and the utilization efficiency of training data by combining force control policy and adding a memory buffer, respectively. Results: We build a single-axis hole assembly system based on the UR5e robotic arm and six-dimensional force sensor in the CoppeliaSim simulation environment to effectively realize the prediction of the assembly environment. Compared with the traditional Deep Deterministic Policy Gradient algorithm (DDPG) and PPO algorithm, the peg-in-hole assembly success rate reaches 100% and the data utilization rate is 125% higher than that of the PPO algorithm. Conclusions: The EFPPO algorithm has a high exploration efficiency. While improving the assembly speed and training speed, the EFPPO algorithm achieves smooth assembly and accurate prediction of the assembly environment.

Learning A Simulation-based Visual Policy for Real-world Peg in Unseen Holes.

Learning A Simulation-based Visual Policy for Real-world Peg In Unseen Holes

Learning to Fill the Seam by Vision: Sub-millimeter Peg-in-hole on Unseen Shapes in Real World

Learning Active Force–Torque Based Policy for Sub-mm Localization of Unseen Holes

Learning-based Optimization Algorithms Combining Force Control Strategies for Peg-in-Hole Assembly.

Transferable Trial-Minimizing Progressive Peg-in-hole Model

Quickly Inserting Pegs into Uncertain Holes using Multi-view Images and Deep Network Trained on Synthetic Data

Vision-based robotic peg-in-hole research: integrating object recognition, positioning, and reinforcement learning

Modular Deep Q Networks for Sim-to-real Transfer of Visuo-motor Policies

Learning-Based Visual Servoing for High-Precision Peg-in-Hole Assembly

Multimodality Driven Impedance-Based Sim2Real Transfer Learning for Robotic Multiple Peg-in-Hole Assembly

A Containerized Simulation Platform for Robot Learning Peg-in-hole Task

Safe Self-Supervised Learning in Real of Visuo-Tactile Feedback Policies for Industrial Insertion

PolyFit: A Peg-in-hole Assembly Framework for Unseen Polygon Shapes via Sim-to-real Adaptation

Peg-in-hole assembly in live-line maintenance based on generative mapping and searching network

Visual Spatial Attention and Proprioceptive Data-Driven Reinforcement Learning for Robust Peg-in-Hole Task Under Variable Conditions

Active Extrinsic Contact Sensing: Application to General Peg-in-Hole Insertion

Evaluating Robustness of Visual Representations for Object Assembly Task Requiring Spatio-Geometrical Reasoning

Fast Peg-in-hole Assembly Policy for Robots Based on Experience Fusion Proximal Optimization

Knowledge-Driven Deep Deterministic Policy Gradient for Robotic Multiple Peg-in-Hole Assembly Tasks

Real2Sim or Sim2Real: Robotics Visual Insertion using Deep Reinforcement Learning and Real2Sim Policy Adaptation