Abstract:Accurately estimating the shape of objects in dense clutters makes important contribution to robotic packing, because the optimal object arrangement requires the robot planner to acquire shape information of all existed objects. However, the objects for packing are usually piled in dense clutters with severe occlusion, and the object shape varies significantly across different instances for the same category. They respectively cause large object segmentation errors and inaccurate shape recovery on unseen instances, which both degrade the performance of shape estimation during deployment. In this paper, we propose a category-level shape estimation method for densely cluttered objects. Our framework partitions each object in the clutter via the multi-view visual information fusion to achieve high segmentation accuracy, and the instance shape is recovered by deforming the category templates with diverse geometric transformations to obtain strengthened generalization ability. Specifically, we first collect the multi-view RGB-D images of the object clutters for point cloud reconstruction. Then we fuse the feature maps representing the visual information of multi-view RGB images and the pixel affinity learned from the clutter point cloud, where the acquired instance segmentation masks of multi-view RGB images are projected to partition the clutter point cloud. Finally, the instance geometry information is obtained from the partially observed instance point cloud and the corresponding category template, and the deformation parameters regarding the template are predicted for shape estimation. Experiments in the simulated environment and real world show that our method achieves high shape estimation accuracy for densely cluttered everyday objects with various shapes.

Estimating Spatial Layout of Cluttered Rooms by Using Object Prior and Spatial Constraints

Estimating spatial layout of rooms from RGB-D videos.

The Polygonal 3D Layout Reconstruction of an Indoor Environment via Voxel-Based Room Segmentation and Space Partition

Fast 3D Indoor Scene Synthesis with Discrete and Exact Layout Pattern Extraction

Object-aware Semantic Mapping of Indoor Scenes Using Octomap

3D Room Layout Estimation from a Single RGB Image.

Fast 3D Indoor Scene Synthesis by Learning Spatial Relation Priors of Objects

Fast 3D Indoor Scene Synthesis by LearningSpatial Relation Priors of Objects

Physics Inspired Optimization on Semantic Transfer Features: an Alternative Method for Room Layout Estimation.

Total3DUnderstanding: Joint Layout, Object Pose and Mesh Reconstruction for Indoor Scenes from a Single Image

Category-level Shape Estimation for Densely Cluttered Objects

Three-Dimensional Structure Measurement And Optimization Method Of Indoor Scene Based On Single Image

Disentangling Orthogonal Planes for Indoor Panoramic Room Layout Estimation with Cross-Scale Distortion Awareness

Semantic-aware Room-Level Indoor Modeling from Point Clouds

Volumetric Representation and Sphere Packing of Indoor Space for Three-Dimensional Room Segmentation

Learning a Room with the Occ-SDF Hybrid: Signed Distance Function Mingled with Occupancy Aids Scene Representation

Understanding of Indoor Scenes Based on Projection of Spatial Rectangles.

Clutter Detection and Removal in 3D Scenes with View-Consistent Inpainting

Efficient 3D Room Shape Recovery from a Single Panorama.

Edge-Semantic Learning Strategy for Layout Estimation in Indoor Environment

Spatial Topological Relation Analysis for Cluttered Scenes