Abstract:Understanding what objects could furnish for humans-namely, learning object affordance-is the crux to bridge perception and action. In the vision community, prior work primarily focuses on learning object affordance with dense (e.g., at a per-pixel level) supervision. In stark contrast, we humans learn the object affordance without dense labels. As such, the fundamental question to devise a computational model is: What is the natural way to learn the object affordance from visual appearance and geometry with humanlike sparse supervision? In this work, we present a new task of part-level affordance discovery (PartAfford): Given only the affordance labels per object, the machine is tasked to (i) decompose 3D shapes into parts and (ii) discover how each part of the object corresponds to a certain affordance category. We propose a novel learning framework for PartAfford, which discovers part-level representations by leveraging only the affordance set supervision and geometric primitive regularization, without dense supervision. The proposed approach consists of two main components: (i) an abstraction encoder with slot attention for unsupervised clustering and abstraction, and (ii) an affordance decoder with branches for part reconstruction, affordance prediction, and cuboidal primitive regularization. To learn and evaluate PartAfford, we construct a part-level, cross-category 3D object affordance dataset, annotated with 24 affordance categories shared among >25, 000 objects. We demonstrate that our method enables both the abstraction of 3D objects and part-level affordance discovery, with generalizability to difficult and cross-category examples. Further ablations reveal the contribution of each component.

Segmenting Object Affordances: Reproducibility and Sensitivity to Scale

Are standard Object Segmentation models sufficient for Learning Affordance Segmentation?

Affordance segmentation of hand-occluded containers from exocentric images

LASO: Language-guided Affordance Segmentation on 3D Object

3D AffordanceNet: A Benchmark for Visual Object Affordance Understanding

3D-TAFS: A Training-free Framework for 3D Affordance Segmentation

Rethinking Interactive Image Segmentation: Feature Space Annotation

Benchmarking Segmentation Models with Mask-Preserved Attribute Editing

Chair Segments: A Compact Benchmark for the Study of Object Segmentation

Fpha-afford: a domain-specific benchmark dataset for occluded object affordance estimation in human-object-robot interaction

Transparency Distortion Robustness for SOTA Image Segmentation Tasks

Segment anything, from space?

OccuSeg: Occupancy-Aware 3D Instance Segmentation

Robot Instance Segmentation with Few Annotations for Grasping

O2O-Afford: Annotation-Free Large-Scale Object-Object Affordance Learning

Occlusion-Aware Object Localization, Segmentation and Pose Estimation

Reliability in Semantic Segmentation: Are We on the Right Track?

PartAfford: Part-level Affordance Discovery from 3D Objects

RESSCAL3D++: Joint Acquisition and Semantic Segmentation of 3D Point Clouds

Affordance segmentation using tiny networks for sensing systems in wearable robotic devices