Abstract:Purpose This paper aims to use fully convolutional network (FCN) to predict pixel-wise antipodal grasp affordances for unknown objects and improve the grasp detection performance through multi-scale feature fusion. Design/methodology/approach A modified FCN network is used as the backbone to extract pixel-wise features from the input image, which are further fused with multi-scale context information gathered by a three-level pyramid pooling module to make more robust predictions. Based on the proposed unify feature embedding framework, two head networks are designed to implement different grasp rotation prediction strategies (regression and classification), and their performances are evaluated and compared with a defined point metric. The regression network is further extended to predict the grasp rectangles for comparisons with previous methods and real-world robotic grasping of unknown objects. Findings The ablation study of the pyramid pooling module shows that the multi-scale information fusion significantly improves the model performance. The regression approach outperforms the classification approach based on same feature embedding framework on two data sets. The regression network achieves a state-of-the-art accuracy (up to 98.9%) and speed (4 ms per image) and high success rate (97% for household objects, 94.4% for adversarial objects and 95.3% for objects in clutter) in the unknown object grasping experiment. Originality/value A novel pixel-wise grasp affordance prediction network based on multi-scale feature fusion is proposed to improve the grasp detection performance. Two prediction approaches are formulated and compared based on the proposed framework. The proposed method achieves excellent performances on three benchmark data sets and real-world robotic grasping experiment.

Grasp Stability Assessment Through Spatio-Temporal Attention Mechanism and Adaptive Gate Fusion

Grasp Stability Assessment Through Attention-Guided Cross-Modality Fusion and Transfer Learning

A Cascaded Deep Learning Framework for Real-time and Robust Grasp Planning

Grasp Stability Prediction Using Tactile Information.

Grasp Stability Assessment Through the Fusion of Visual and Tactile Signals Using Deep Bilinear Network.

Visual-Tactile Fusion for Robotic Stable Grasping

Robot Grasping Stability Prediction Network based on Feature-fusion and Feature-reconstruction of Tactile Information

Detecting Transitions from Stability to Instability in Robotic Grasping Based on Tactile Perception

Tactile Grasp Stability Classification Based on Graph Convolutional Networks

Efficient Grasp Detection Network with Gaussian-Based Grasp Representation for Robotic Manipulation

Predict Robot Grasp Outcomes Based On Multi-Modal Information

Robot Grasping System and Grasp Stability Prediction Based on Flexible Tactile Sensor Array

Integrating High-Resolution Tactile Sensing into Grasp Stability Prediction

High-performance Pixel-level Grasp Detection Based on Adaptive Grasping and Grasp-aware Network

Gated Self Attention Network for Efficient Grasping of Target Objects in Stacked Scenarios

Slip Detection through Fusion of Visual-Tactile Data using Swin Transformer V2

Visuo-Tactile-Based Slip Detection Using A Multi-Scale Temporal Convolution Network

Grasp stability prediction with time series data based on STFT and LSTM

Real-time Pixel-Wise Grasp Affordance Prediction Based on Multi-Scale Context Information Fusion

Vahagn: VisuAl Haptic Attention Gate Net for slip detection

Robotic Slip Detection with Visual-Tactile Fusion Based on 3D Attention Mechanism