Abstract:In this paper, we address the semantic segmentation task with a new context aggregation scheme named \emph{object context}, which focuses on enhancing the role of object information. Motivated by the fact that the category of each pixel is inherited from the object it belongs to, we define the object context for each pixel as the set of pixels that belong to the same category as the given pixel in the image. We use a binary relation matrix to represent the relationship between all pixels, where the value one indicates the two selected pixels belong to the same category and zero otherwise. We propose to use a dense relation matrix to serve as a surrogate for the binary relation matrix. The dense relation matrix is capable to emphasize the contribution of object information as the relation scores tend to be larger on the object pixels than the other pixels. Considering that the dense relation matrix estimation requires quadratic computation overhead and memory consumption w.r.t. the input size, we propose an efficient interlaced sparse self-attention scheme to model the dense relations between any two of all pixels via the combination of two sparse relation matrices. To capture richer context information, we further combine our interlaced sparse self-attention scheme with the conventional multi-scale context schemes including pyramid pooling~\citep{zhao2017pyramid} and atrous spatial pyramid pooling~\citep{chen2018deeplab}. We empirically show the advantages of our approach with competitive performances on five challenging benchmarks including: Cityscapes, ADE20K, LIP, PASCAL-Context and COCO-Stuff

Sparse Coding of Visual Context

Spatial Relationship Representation for Visual Object Searching

Spatially-Aware Context Neural Networks.

Sparse Concept Coding for Visual Analysis

Learning and Memory of Spatial Relationship by a Neural Network with Sparse Features

Latent Visual Context Learning for Web Image Applications

An Object-Oriented Visual Saliency Detection Framework Based on Sparse Coding Representations

Spatial Neighborhood-Constrained Linear Coding for Visual Object Tracking

Weakly-Supervised Spatial Context Networks.

Adjacent Coding for Image Classification.

Unsupervised Visual Representation Learning by Context Prediction

Object-Based Image Retrieval Using Spatial Context

OCNet: Object Context Network for Scene Parsing

Sparse coding for image feature extraction and image recognition

Emergence of simple-cell receptive field properties by learning a sparse code for natural images

Orthogonal Projective Sparse Coding for Image Representation

Exploring Dense Context for Salient Object Detection

Topic Level Sampling Towards Optimized Locality Sensitive Vocabulary Coding

Contextual Hashing for Large-Scale Image Search.

Learning Coarse-To-Fine Sparselets For Efficient Object Detection And Scene Classification

Low-Rank Sparse Coding for Image Classification