Abstract:In contrast to fully supervised methods using pixel-wise mask labels, box-supervised instance segmentation takes advantage of simple box annotations, which has recently attracted increasing research attention. This paper presents a novel single-shot instance segmentation approach, namely Box2Mask, which integrates the classical level-set evolution model into deep neural network learning to achieve accurate mask prediction with only bounding box supervision. Specifically, both the input image and its deep features are employed to evolve the level-set curves implicitly, and a local consistency module based on a pixel affinity kernel is used to mine the local context and spatial relations. Two types of single-stage frameworks, i.e., CNN-based and transformer-based frameworks, are developed to empower the level-set evolution for box-supervised instance segmentation, and each framework consists of three essential components: instance-aware decoder, box-level matching assignment and level-set evolution. By minimizing the level-set energy function, the mask map of each instance can be iteratively optimized within its bounding box annotation. The experimental results on five challenging testbeds, covering general scenes, remote sensing, medical and scene text images, demonstrate the outstanding performance of our proposed Box2Mask approach for box-supervised instance segmentation. In particular, with the Swin-Transformer large backbone, our Box2Mask obtains 42.4% mask AP on COCO, which is on par with the recently developed fully mask-supervised methods. The code is available at: https://github.com/LiWentomng/boxlevelset.

BorderPointsMask: One-stage Instance Segmentation with Boundary Points Representation.

A Simultaneous Object Detection and Component Segmentation Approach Based on Mask R-CNN

Boundary-Area Enhanced Module for Instance Segmentation

Boundary-Aware Instance Segmentation

BAProto: Boundary-Aware Prototype for High-quality Instance Segmentation.

BoxInst: High-Performance Instance Segmentation with Box Annotations.

Box2Mask: Box-supervised Instance Segmentation via Level-set Evolution

BlendMask: Top-Down Meets Bottom-Up for Instance Segmentation

Boundary-preserving Mask R-CNN

Mask SSD: an Effective Single-Stage Approach to Object Instance Segmentation

CenterMask: Real-Time Anchor-Free Instance Segmentation

A Coarse-to-Fine Instance Segmentation Network with Learning Boundary Representation

RefineMask: Towards High-Quality Instance Segmentation with Fine-Grained Features

MaskGroup: Hierarchical Point Grouping and Masking for 3D Instance Segmentation

Look Closer to Segment Better: Boundary Patch Refinement for Instance Segmentation

EmbedMask: Embedding Coupling for One-stage Instance Segmentation

PolarMask: Single Shot Instance Segmentation With Polar Representation

MaskLab: Instance Segmentation by Refining Object Detection with Semantic and Direction Features

Mask Encoding: A General Instance Mask Representation for Object Segmentation

DOBNET - Dynamic Object Boundary-Refinement Network for Real-Time Instance Segmentation.

PointINS: Point-based instance segmentation