Abstract:Open containers, i.e., containers without covers, are an important and ubiquitous class of objects in human life. In this letter, we propose a novel method for robots to "imagine" the open containability affordance of a previously unseen object via physical simulations. We implement our imagination method on a UR5 manipulator. The robot autonomously scans the object with an RGB-D camera. The scanned 3D model is used for open containability imagination which quantifies the open containability affordance by physically simulating dropping particles onto the object and counting how many particles are retained in it. This quantification is used for open-container vs. non-open-container binary classification (hereafter referred to as open container classification). If the object is classified as an open container, the robot further imagines pouring into the object, again using physical simulations, to obtain the pouring position and orientation for real robot autonomous pouring. We evaluate our method on open container classification and autonomous pouring of granular material on a dataset containing 130 previously unseen objects with 57 object categories. Although our proposed method uses only 11 objects for simulation calibration (training), its open container classification aligns well with human judgements. In addition, our method endows the robot with the capability to autonomously pour into the 55 containers in the dataset with a very high success rate. We also compare to a deep learning method. Results show that our method achieves the same performance as the deep learning method on open container classification and outperforms it on autonomous pouring. Moreover, our method is fully explainable.

Active Auditory Exploration for Identifying Object Contents

In-Hand Manipulation For Active Object Recognition

Open-Environment Robotic Acoustic Perception for Object Recognition.

Vision-Based Robotic Object Grasping—A Deep Reinforcement Learning Approach

Deep Neural Object Analysis by Interactive Auditory Exploration with a Humanoid Robot

Active Object Perceiver: Recognition-Guided Policy Learning for Object Searching on Mobile Robots

Can I Pour into It? Robot Imagining Open Containability Affordance of Previously Unseen Objects via Physical Simulations

Active Affordance Exploration For Robot Grasping

AcTExplore: Active Tactile Exploration of Unknown Objects

Elder-oriented Active Learning for Adaptation of Perception Intelligence in Home Service Robots

Language-guided Active Sensing of Confined, Cluttered Environments via Object Rearrangement Planning

Active Object Detection Based on a Novel Deep Q-Learning Network and Long-Term Learning Strategy for the Service Robot

Active Object Search

That Sounds Right: Auditory Self-Supervision for Dynamic Robot Manipulation

Active Acoustic Sensing for Robot Manipulation

Active Object Discovery and Localization Using Sound-Induced Attention

Learning Visual-Audio Representations for Voice-Controlled Robots

Sound-Indicated Visual Object Detection for Robotic Exploration

SonicSense: Object Perception from In-Hand Acoustic Vibration

Position Identification by Actively Localizing Spacial Sound Beacons.

Latent Object Characteristics Recognition with Visual to Haptic-Audio Cross-modal Transfer Learning