Abstract:Recent progress in robotic manipulation has dealt with the case of previously unknown objects in the context of relatively simple tasks, such as bin-picking. Existing methods for more constrained problems, however, such as deliberate placement in a tight region, depend more critically on shape information to achieve safe execution. This work deals with pick-and-constrained placement of objects without access to geometric models. The objective is to pick an object and place it safely inside a desired goal region without any collisions, while minimizing the time and the sensing operations required to complete the task. An algorithmic framework is proposed for this purpose, which performs manipulation planning simultaneously over a conservative and an optimistic estimate of the object's volume. The conservative estimate ensures that the manipulation is safe while the optimistic estimate guides the sensor-based manipulation process when no solution can be found for the conservative estimate. To maintain these estimates and dynamically update them during manipulation, objects are represented by a simple volumetric representation, which stores sets of occupied and unseen voxels. The effectiveness of the proposed approach is demonstrated by developing a robotic system that picks a previously unseen object from a table-top and places it in a constrained space. The system comprises of a dual-arm manipulator with heterogeneous end-effectors and leverages hand-offs as a re-grasping strategy. Real-world experiments show that straightforward pick-sense-and-place alternatives frequently fail to solve pick-and-constrained placement problems. The proposed pipeline, however, achieves more than 95% success rate and faster execution times as evaluated over multiple physical experiments.

Unknown object modeling on the basis of vision and pushing manipulation.

Unknown Objects Grasp Planning Using Force/Torque Measurements

Dexterous Manoeuvre Through Touch in a Cluttered Scene

Vision-Based Categorical Object Pose Estimation and Manipulation.

Sliding Touch-based Exploration for Modeling Unknown Object Shape with Multi-fingered Hands

3D Foundation Models Enable Simultaneous Geometry and Pose Estimation of Grasped Objects

Grasping Unknown Objects Based on 3d Model Reconstruction

Design And Use Of An Apparatus For Presenting Graspable Objects In 3d Workspace

Unified Motion/Force/Impedance Control for Manipulators in Unknown Contact Environments Based on Robust Model-Reaching Approach

UNO Push: Unified Nonprehensile Object Pushing via Non-Parametric Estimation and Model Predictive Control

Safe and Efficient Robot Manipulation: Task-Oriented Environment Modeling and Object Pose Estimation

Multi-object recognition on the basis of vision and manipulation

Push to know! -- Visuo-Tactile based Active Object Parameter Inference with Dual Differentiable Filtering

Task-driven Perception and Manipulation for Constrained Placement of Unknown Objects

Uncertainty-aware Active Learning of NeRF-based Object Models for Robot Manipulators using Visual and Re-orientation Actions

Vision-based Manipulation of Deformable and Rigid Objects Using Subspace Projections of 2D Contours

Realtime State Estimation with Tactile and Visual Sensing. Application to Planar Manipulation

Autonomous Robotic Manipulation: Real-Time, Deep-Learning Approach for Grasping of Unknown Objects

A Comprehensive Study of 3-D Vision-Based Robot Manipulation

Object Property Identification Using Uncertain Robot Manipulator.

Tac-Man: Tactile-Informed Prior-Free Manipulation of Articulated Objects