Abstract:Instance segmentation needs to locate all instances in an image correctly and segment each instance precisely. Currently, the most dominant methods for instance segmentation take object detection as a pre-task. However, they rely on the accuracy of object detection incredibly. If the pre-task cannot predict an accurate bounding box, the performance of instance segmentation will degenerate. In this paper, we present a novel method for instance segmentation to solve this problem, which is called S egmenting B eyond the B ounding B ox ( S3B-Net ). Our S3B-Net designs a sub-network to help instance segmentation methods based on object detection to segment the part of an instance beyond the bounding box. Specifically, the sub-network first predicts a two-dimensional pixel embedding for each pixel. Then, the Gaussian function is employed to calculate a pixel’s probability belongs to a corresponding instance according to the two-dimensional pixel embedding. Finally, the output of the sub-network combines with the output of instance segmentation based on object detection to generate a more precise instance mask. Our sub-network can easily extend on the existing instance segmentation method based on object detection to segment instance beyond the bounding box. We do our experiments on dominant instance segmentation datasets, such as the COCO dataset and Cityscapes dataset. The results show that our method can achieve 6.8 points gain compared with the baseline Mask R-CNN with ResNet-50-FPN in Cityscapes datasets, and 1.7 points gain with ResNet-101-FPN-DCN in COCO datasets. Our S3B-Net outperforms the previous state-of-the-art instance segmentation method, which proves our method is competitive. The source code of our method will be made available.

Object Bounding Box-Aware Embedding for Point Cloud Instance Segmentation

Dive Deeper into Box for Object Detection

Instance-Aware Embedding for Point Cloud Instance Segmentation

Learning Object Bounding Boxes for 3D Instance Segmentation on Point Clouds

Boundary-Aware Geometric Encoding for Semantic Segmentation of Point Clouds

3D Object Segmentation Using Cross-Window Point Transformer with Latent Semantic Boundary Guidance

Boundary-Aware Instance Segmentation

An Efficient Instance Segmentation Framework Using Segmentation Foundation Models with Oriented Bounding Box Prompts

Point Cloud Instance Segmentation with Semi-supervised Bounding-Box Mining

Box2Seg: Learning Semantics of 3D Point Clouds with Box-Level Supervision

ISBNet: a 3D Point Cloud Instance Segmentation Network with Instance-aware Sampling and Box-aware Dynamic Convolution

BEB-Net: Boundary Extraction Based Semantic Segmentation Network for Indoor Scenes in Smart Power Plants

When 3D Bounding-Box Meets SAM: Point Cloud Instance Segmentation with Weak-and-Noisy Supervision

Deep Level Set for Box-supervised Instance Segmentation in Aerial Images

Divide and Conquer: 3D Point Cloud Instance Segmentation With Point-Wise Binarization

Multi-fineness Boundaries and the Shifted Ensemble-aware Encoding for Point Cloud Semantic Segmentation

Object-based Building Instance Segmentation from Airborne LiDAR Point Clouds

SPBA-Net point cloud object detection with sparse attention and box aligning

End-to-End 3D Point Cloud Instance Segmentation Without Detection

Occluded Object Segmentation Based on Bilayer Decoupling Strategy and Attention Mechanism br

Segmenting Beyond the Bounding Box for Instance Segmentation.