Uncertainty-driven Exploration Strategies for Online Grasp Learning

Yitian Shi,Philipp Schillinger,Miroslav Gabriel,Alexander Qualmann,Zohar Feldman,Hanna Ziesche,Ngo Anh Vien

2024-04-24

Abstract:Existing grasp prediction approaches are mostly based on offline learning, while, ignoring the exploratory grasp learning during online adaptation to new picking scenarios, i.e., objects that are unseen or out-of-domain (OOD), camera and bin settings, etc. In this paper, we present an uncertainty-based approach for online learning of grasp predictions for robotic bin picking. Specifically, the online learning algorithm with an effective exploration strategy can significantly improve its adaptation performance to unseen environment settings. To this end, we first propose to formulate online grasp learning as an RL problem that will allow us to adapt both grasp reward prediction and grasp poses. We propose various uncertainty estimation schemes based on Bayesian uncertainty quantification and distributional ensembles. We carry out evaluations on real-world bin picking scenes of varying difficulty. The objects in the bin have various challenging physical and perceptual characteristics that can be characterized by semi- or total transparency, and irregular or curved surfaces. The results of our experiments demonstrate a notable improvement of grasp performance in comparison to conventional online learning methods which incorporate only naive exploration strategies. Video: <a class="link-external link-https" href="https://youtu.be/fPKOrjC2QrU" rel="external noopener nofollow">this https URL</a>

Robotics,Artificial Intelligence

What problem does this paper attempt to address?

This paper aims to address the issue of online learning in robotic bin picking. Specifically, most existing grasp prediction methods are based on offline learning, neglecting exploratory learning when faced with new scenarios, such as unseen objects, camera setups, and bin arrangements. Therefore, this paper proposes an uncertainty-based approach to achieve online learning for robotic grasp prediction. This method significantly improves adaptability to unknown environmental settings through effective exploration strategies. To achieve this goal, the paper first models online grasp learning as a reinforcement learning problem and proposes several uncertainty estimation schemes, including Bayesian uncertainty quantification and distributional ensemble. These schemes allow the robot to quickly adjust based on pre-trained networks to adapt to new objects or environmental settings. Experimental evaluations show that, compared to traditional methods that only use simple exploration strategies, this method significantly improves the grasp success rate.

Uncertainty-driven Exploration Strategies for Online Grasp Learning

Ensemble Bootstrapped Deep Deterministic Policy Gradient For Vision-Based Robotic Grasping

Pseudo-Labeling and Contextual Curriculum Learning for Online Grasp Learning in Robotic Bin Picking

Online Tool Selection with Learned Grasp Prediction Models

Online augmentation of learned grasp sequence policies for more adaptable and data-efficient in-hand manipulation

A learning framework for semantic reach-to-grasp tasks integrating machine learning and optimization.

Accelerating Grasp Exploration by Leveraging Learned Priors

vMF-Contact: Uncertainty-aware Evidential Learning for Probabilistic Contact-grasp in Noisy Clutter

Learning Visuotactile Estimation and Control for Non-prehensile Manipulation under Occlusions

Vision-Based Robotic Object Grasping—A Deep Reinforcement Learning Approach

Probabilistic Closed-Loop Active Grasping

Dealing with Ambiguity in Robotic Grasping via Multiple Predictions

Exploratory Grasping: Asymptotically Optimal Algorithms for Grasping Challenging Polyhedral Objects

Leveraging Contact Forces for Learning to Grasp

Unscented Bayesian Optimization for Safe Robot Grasping

Offline-to-online Reinforcement Learning for Image-based Grasping with Scarce Demonstrations

Efficient End-to-End Detection of 6-DoF Grasps for Robotic Bin Picking

UPG: 3D Vision-Based Prediction Framework for Robotic Grasping in Multi-Object Scenes.

Global Search with Bernoulli Alternation Kernel for Task-oriented Grasping Informed by Simulation

A Novel Robotic Grasp Detection Framework Using Low-Cost RGB-D Camera for Industrial Bin Picking

Online Evaluation for Learning Feasible Robotic Grasps with Physical Constraints