Abstract:Combining Convolutional Neural Networks (CNNs) with Conditional Random Fields (CRFs) achieves great success among recent object segmentation methods. There are two advantages by such usage. First, CNNs can extract low-level features, which are very similar to the extracted features in primates’ primary visual cortex (V1). Second, CRFs can set up the relationship between input features and output labels in a direct way. In this paper, we extend the first advantage by using CNNs for low-level feature extraction and a Structured Random Forest (SRF)-based border ownership detector for high-level feature extraction, which are similar to the outputs of primates secondary visual cortex (V2). Compared to the CRF model, an improved Conditional Boltzmann Machine (CBM), which has a multi-channel visible layer, is proposed to model the relationship between predicted labels, local and global contexts of objects with multi-scale and multilevel features. Besides, our proposed CBM model is extended for object parsing by using multivisible branches instead of a single visible layer of CBM, which cannot only segment the whole body but also the parts of the body under. These visible branches use each branch for the segmentation of the whole body or one of the body parts. All branches share the same hidden layers of CBM and train the branches under an iterative way. By exploiting object parsing, the whole body segmentation performance of object is improved. To refine the segmentation output, two kinds of optimization algorithms are proposed. The superpixel-based algorithm can re-label the overlapped regions of multiple kinds of objects. The other curve correction algorithm corrects the edges of segmented object parts by using smooth edges under a curve similarity criterion. Experiments demonstrate that our models yield competitive results for object segmentation on the PASCAL VOC 2012 dataset and for object parsing on the PennFudan Pedestrian Parsing dataset, Pedestrian Parsing Surveillance Scenes dataset, Horse-Cow parsing dataset, and PASCAL Quadrupeds dataset.

Integrating Pixels and Segments: A Deep-Learning Method Inspired by the Informational Diversity of the Visual Pathways

Deep Dual-Stream Network with Scale Context Selection Attention Module for Semantic Segmentation

Using Line Segments to Train Multi-Stream Stacked Autoencoders for Image Classification

ISDNet: Integrating Shallow and Deep Networks for Efficient Ultra-high Resolution Segmentation

SegNet Network Architecture for Deep Learning Image Segmentation and Its Integrated Applications and Prospects

Learning Cross-Channel Representations for Semantic Segmentation

Learning Common and Specific Features for RGB-D Semantic Segmentation with Deconvolutional Networks

Superpixel Semantics Representation and Pre-training for Vision-Language Task

Semantic Image Segmentation with Improved Position Attention and Feature Fusion

Using a Vertical-Stream Variational Auto-Encoder to Generate Segment-Based Images and Its Biological Plausibility for Modelling the Visual Pathways.

Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation

SegNet: A Deep Convolutional Encoder-Decoder Architecture for Robust Semantic Pixel-Wise Labelling

IIE-SegNet: Deep Semantic Segmentation Network With Enhanced Boundary Based on Image Information Entropy

Learning Deep Conditional Neural Network for Image Segmentation

Fully Convolutional Networks for Semantic Segmentation

Recognizing Image Objects by Relational Analysis Using Heterogeneous Superpixels and Deep Convolutional Features

Bi-directional Cross-Modality Feature Propagation with Separation-and-Aggregation Gate for RGB-D Semantic Segmentation

A Novel Approach to Optimizing Convolutional Neural Networks for Improved Digital Image Segmentation

On the Influence of Shape, Texture and Color for Learning Semantic Segmentation

A Holistically-Guided Decoder for Deep Representation Learning with Applications to Semantic Segmentation and Object Detection.

A Cyclic Information–Interaction Model for Remote Sensing Image Segmentation