Abstract:Purpose This paper aims to use fully convolutional network (FCN) to predict pixel-wise antipodal grasp affordances for unknown objects and improve the grasp detection performance through multi-scale feature fusion. Design/methodology/approach A modified FCN network is used as the backbone to extract pixel-wise features from the input image, which are further fused with multi-scale context information gathered by a three-level pyramid pooling module to make more robust predictions. Based on the proposed unify feature embedding framework, two head networks are designed to implement different grasp rotation prediction strategies (regression and classification), and their performances are evaluated and compared with a defined point metric. The regression network is further extended to predict the grasp rectangles for comparisons with previous methods and real-world robotic grasping of unknown objects. Findings The ablation study of the pyramid pooling module shows that the multi-scale information fusion significantly improves the model performance. The regression approach outperforms the classification approach based on same feature embedding framework on two data sets. The regression network achieves a state-of-the-art accuracy (up to 98.9%) and speed (4 ms per image) and high success rate (97% for household objects, 94.4% for adversarial objects and 95.3% for objects in clutter) in the unknown object grasping experiment. Originality/value A novel pixel-wise grasp affordance prediction network based on multi-scale feature fusion is proposed to improve the grasp detection performance. Two prediction approaches are formulated and compared based on the proposed framework. The proposed method achieves excellent performances on three benchmark data sets and real-world robotic grasping experiment.

Hierarchical Multi-modal Fusion for Language-conditioned Robotic Grasping Detection in Clutter

Multi-Object Grasping Detection with Hierarchical Feature Fusion

LAC-Net: Linear-Fusion Attention-Guided Convolutional Network for Accurate Robotic Grasping Under the Occlusion

Bilateral Cross-Modal Fusion Network for Robot Grasp Detection

Joint Segmentation and Grasp Pose Detection with Multi-Modal Feature Fusion Network.

A robot grasping detection network based on flexible selection of multi-modal feature fusion structure

GraspFusionNet: a Two-Stage Multi-Parameter Grasp Detection Network Based on RGB–XYZ Fusion in Dense Clutter

A Joint Modeling of Vision-Language-Action for Target-oriented Grasping in Clutter

Visual-tactile Fusion for Transparent Object Grasping in Complex Backgrounds

A novel integrated method of detection-grasping for specific object based on the box coordinate matching

Two-stage Grasp Detection Method for Robotics Using Point Clouds and Deep Hierarchical Feature Learning Network

A Parameter-Efficient Tuning Framework for Language-guided Object Grounding and Robot Grasping

Language Guided Robotic Grasping with Fine-Grained Instructions

Robotic Grasp Detection Method Based on Lightweight Feature Fusion Convolutional Neural Network

Fine Grained Dual Level Attention Mechanisms with Spacial Context Information Fusion for Object Detection

Robotic Grasp Detection Using Structure Prior Attention and Multiscale Features

Visual-and-Language Multimodal Fusion for Sweeping Robot Navigation Based on CNN and GRU

Multi-Stage Image-Language Cross-Generative Fusion Network for Video-Based Referring Expression Comprehension

GraspMamba: A Mamba-based Language-driven Grasp Detection Framework with Hierarchical Feature Learning

Robotic Grasping in Multi-Object Stacking Scenes Based on Visual Reasoning

Real-time Pixel-Wise Grasp Affordance Prediction Based on Multi-Scale Context Information Fusion