Abstract:Object affordance detection aims to identify, locate and segment the functional regions of objects, so that robots can understand and manipulate objects like humans. The affordance detection task has two main challenges: (1) Due to the need to provide accurate positioning information for the robot to manipulate objects, the affordance segmentation results are required to have high boundary quality. (2) Different kinds of objects have significant differences in appearances, but may have the same affordance. Correspondingly, parts with the same appearance may have different affordances. The existing methods regard affordance detection as an image segmentation problem, without focusing on the boundary quality of detection results. In addition, most of the existing methods do not consider the potential relationship between object categories and object affordances. Aiming at the above problems, we propose a boundary-preserving network (BPN) to provide affordance masks with better boundary quality for robots to manipulate objects. Our framework contains three new components: the IoU (Intersection-over-Union) branch, the affordance boundary branch and the relationship attention module. The IoU branch is used to predict the IoU score of each object bounding box. The affordance boundary branch is used to guide the network to learn the boundary features of objects. The relationship attention module is used to enhance the feature representation capability of the network by exploring the potential relationship between object categories and object affordances. Experiments show that our method is helpful to improve the boundary quality of the predicted affordance masks. On the IIT-AFF dataset, the performance of the proposed BPN is 2.32% (F-score) and 2.89% (F-score) higher than that of the strong baseline in terms of affordance masks and the boundaries of affordance masks, respectively. Furthermore, the real-world robot manipulation experiments show that the proposed BPN can provide accurate affordance information for robots to manipulate objects.

Affordance segmentation using tiny networks for sensing systems in wearable robotic devices

Affordance segmentation using tiny networks for sensing systems in wearable robotic devices

A Scalable Real-time Semantic Segmentation Network for Autonomous Driving

High-level Object Affordance Recognition.

Affordance detection for task-specific grasping using deep learning

Object affordance detection with boundary-preserving network for robotic manipulation tasks

Deep Affordance-Grounded Sensorimotor Object Recognition

Bayesian Deep Learning for Affordance Segmentation in images

Real-time Pixel-Wise Grasp Affordance Prediction Based on Multi-Scale Context Information Fusion

Information-driven Affordance Discovery for Efficient Robotic Manipulation

Learning Visual Affordance Grounding from Demonstration Videos

Object affordance detection with relationship-aware network

LAC-Net: Linear-Fusion Attention-Guided Convolutional Network for Accurate Robotic Grasping Under the Occlusion

Exploring Artificial Neural Networks Efficiency in Tiny Wearable Devices for Human Activity Recognition

AttoNets: Compact and Efficient Deep Neural Networks for the Edge via Human-Machine Collaborative Design

Modular Anti-noise Deep Learning Network for Robotic Grasp Detection Based on RGB Images

Lightweight robotic grasping detection network based on dual attention and inverted residual

Bimodal SegNet: Instance Segmentation Fusing Events and RGB Frames for Robotic Grasping

Point Cloud Hand–Object Segmentation Using Multimodal Imaging with Thermal and Color Data for Safe Robotic Object Handover

Learning Precise Affordances from Egocentric Videos for Robotic Manipulation

Hierarchical Affordance Discovery using Intrinsic Motivation