Abstract:Robotic grasping in the open world is a critical component of manufacturing and automation processes. While numerous existing approaches depend on 2D segmentation output to facilitate the grasping procedure, accurately determining depth from 2D imagery remains a challenge, often leading to limited performance in complex stacking scenarios. In contrast, techniques utilizing 3D point cloud data inherently capture depth information, thus enabling adeptly navigating and manipulating a diverse range of complex stacking scenes. However, such efforts are considerably hindered by the variance in data capture devices and the unstructured nature of the data, which limits their generalizability. Consequently, much research is narrowly concentrated on managing designated objects within specific settings, which confines their real-world applicability. This paper presents a novel pipeline capable of executing object grasping tasks in open-world scenarios even on previously unseen objects without the necessity for training. Additionally, our pipeline supports the flexible use of different 3D point cloud segmentation models across a variety of scenes. Leveraging the segmentation results, we propose to engage a training-free binary clustering algorithm that not only improves segmentation precision but also possesses the capability to cluster and localize unseen objects for executing grasping operations. In our experiments, we investigate a range of open-world scenarios, and the outcomes underscore the remarkable robustness and generalizability of our pipeline, consistent across various environments, robots, cameras, and objects. The code will be made available upon acceptance of the paper.

Picking from Clutter: An Object Segmentation Method for Robot Grasping.

A Robotic Semantic Grasping Method for Pick-and-place Tasks

Bin-picking of novel objects through category-agnostic-segmentation: RGB matters

3D Object Segmentation Using Cross-Window Point Transformer with Latent Semantic Boundary Guidance

Efficient and Robust Robotic Grasping in Cluttered Scenes: A Point Cloud-Based Approach with Heuristic Evaluation.

Unseen Object Few-Shot Semantic Segmentation for Robotic Grasping

Domain-Independent Disperse and Pick method for Robotic Grasping

SG-Grasp: Semantic Segmentation Guided Robotic Grasp Oriented to Weakly Textured Objects Based on Visual Perception Sensors

Multi-scale deep learning and clustering-based tabletop object instance segmentation for robot manipulation

Towards Cross-device and Training-free Robotic Grasping in 3D Open World

Graspability-Aware Object Pose Estimation in Cluttered Scenes

Efficient point cloud segmentation approach using energy optimization with geometric features for 3D scene understanding

Application of Image Segmentation Algorithm Based on Entropy Clustering in Apple Harvesting Robot

A Novel Geometry-based Algorithm for Robust Grasping in Extreme Clutter Environment

Self-Supervised Instance Segmentation by Grasping

Fast and Robust Bin-picking System for Densely Piled Industrial Objects

RGB-D Object Detection and Semantic Segmentation for Autonomous Manipulation in Clutter

GPR: Grasp Pose Refinement Network for Cluttered Scenes

Learning Tri-mode Grasping for Ambidextrous Robot Picking

SegGrasp: Zero-Shot Task-Oriented Grasping via Semantic and Geometric Guided Segmentation

RGBGrasp: Image-based Object Grasping by Capturing Multiple Views during Robot Arm Movement with Neural Radiance Fields