Abstract:We present a novel explicit shape representation for instance segmentation. Based on how to model the object shape, current instance segmentation systems can be divided into two categories, implicit and explicit models. The implicit methods, which represent the object mask/contour by intractable network parameters, and produce it through pixel-wise classification, are predominant. However, the explicit methods, which parameterize the shape with simple and explainable models, are less explored. Since the operations to generate the final shape are light-weighted, the explicit methods have a clear speed advantage over implicit methods, which is crucial for real-world applications. The proposed USD-Seg adopts a linear model, sparse coding with dictionary, for object shapes. First, it learns a dictionary from a large collection of shape datasets, making any shape being able to be decomposed into a linear combination through the dictionary. Hence the name "Universal Shape Dictionary". Then it adds a simple shape vector regression head to ordinary object detector, giving the detector segmentation ability with minimal overhead. For quantitative evaluation, we use both average precision (AP) and the proposed Efficiency of AP (AP$_E$) metric, which intends to also measure the computational consumption of the framework to cater to the requirements of real-world applications. We report experimental results on the challenging COCO dataset, in which our single model on a single Titan Xp GPU achieves 35.8 AP and 27.8 AP$_E$ at 65 fps with YOLOv4 as base detector, 34.1 AP and 28.6 AP$_E$ at 12 fps with FCOS as base detector.

Shape Sparse Representation for Joint Object Classification and Segmentation

Simultaneous variational image segmentation and object recognition via shape sparse representation

LEARNING SHAPE PRIORS BY PAIRWISE COMPARISON FOR ROBUST SEMANTIC SEGMENTATION

Shape Representation and Recognition in High Dimensional Feature Space

Implicit Kernel Sparse Shape Representation: A Sparse-Neighbors-Based Objection Segmentation Framework

A New Sparse Representation-Based Object Segmentation Framework

Learning Universal Shape Dictionary for Realtime Instance Segmentation

A Unified Feature Representation and Learning Framework for 3D Shape

Deep Learning Shape Priors for Object Segmentation

Object Segmentation Framework Based on Dictionary-Group and Sparse Shape Representation

Reduced Set Density Estimator For Object Segmentation Based On Shape Probabilistic Representation

Implicit Kernel Presentation Aware Object Segmentation Framework

Unify 3D Shape Retrieval and Classification in One Framework

A Geometric Approach to Joint 2D Region-Based Segmentation and 3D Pose Estimation Using a 3D Shape Prior

Optimal Multi-Object Segmentation with Novel Gradient Vector Flow Based Shape Priors

Multi-class Object Segmentation Based on Jointly Integrating Segment-Level and Image-Level Object Priors

Learning Coarse-To-Fine Sparselets For Efficient Object Detection And Scene Classification

Visual Classification with Multi-Task Joint Sparse Representation

Shared Latent Membership Enables Joint Shape Abstraction and Segmentation With Deformable Superquadrics

Deep Convolutional Neural Networks Meet Variational Shape Compactness Priors for Image Segmentation

Sparse Representation Shape Models