Abstract:Recently, progress on semantic image segmentation is substantial, benefiting from the rapid development of Convolutional Neural Networks. Semantic image segmentation approaches proposed lately have been mostly based on Fully convolutional Networks (FCNs). However, these FCN-based methods use large receptive fields and too many pooling layers to depict the discriminative semantic information of the images. Specifically, on one hand, convolutional kernel with large receptive field smooth the detailed edges, since too much contexture information is used to depict the “center pixel.” However, the pooling layer increases the receptive field through zooming out the latest feature maps, which loses many detailed information of the image, especially in the deeper layers of the network. These operations often cause low spatial resolution inside deep layers, which leads to spatially fragmented prediction. To address this problem, we exploit the inherent multi-scale and pyramidal hierarchy of deep convolutional networks to extract the feature maps with different resolutions and take full advantages of these feature maps via a gradually stacked fusing way. Specifically, for two adjacent convolutional layers, we upsample the features from deeper layer with stride of 2 and then stack them on the features from shallower layer. Then, a convolutional layer with kernels of 1× 1 is followed to fuse these stacked features. The fused feature preserves the spatial structure information of the image; meanwhile, it owns strong discriminative capability for pixel classification. Additionally, to further preserve the spatial structure information and regional connectivity of the predicted category label map, we propose a novel loss term for the network. In detail, two graph model-based spatial affinity matrixes are proposed, which are used to depict the pixel-level relationships in the input image and predicted category label map respectively, and then their cosine distance is backward propagated to the network. The proposed architecture, called spatial structure preserving feature pyramid network, significantly improves the spatial resolution of the predicted category label map for semantic image segmentation. The proposed method achieves state-of-the-art results on three public and challenging datasets for semantic image segmentation.

Fractal Pyramid Networks

Pyramid Feature Attention Network for Monocular Depth Prediction

FlatteNet: A Simple Versatile Framework for Dense Pixelwise Prediction

FPANet: Feature Pyramid Aggregation Network for Real-Time Semantic Segmentation

Flattenet: A Simple And Versatile Framework For Dense Pixelwise Prediction

PNEN: Pyramid Non-Local Enhanced Networks

DCPNet: A Densely Connected Pyramid Network for Monocular Depth Estimation

EPRNet: Efficient Pyramid Representation Network for Real-Time Street Scene Segmentation

Pyramid Frequency Network with Spatial Attention Residual Refinement Module for Monocular Depth Estimation

Tripartite Feature Enhanced Pyramid Network for Dense Prediction

Fully Convolutional Pyramidal Networks for Semantic Segmentation.

Spatial Structure Preserving Feature Pyramid Network for Semantic Image Segmentation

Ppednet: Pyramid Pooling Encoder-Decoder Network For Real-Time Semantic Segmentation

Efficient pyramid context encoding and feature embedding for semantic segmentation

Pyramidal Predictive Network: A Model for Visual-Frame Prediction Based on Predictive Coding Theory

DFPNet:Dislocation Double Feature Pyramid Real-time Semantic Segmentation Network

Fractal Pyramid Low-Light Image Enhancement Network with Illumination Information

Depthwise and Spatial Factorized Network: a Light-weight Network for Real-time Semantic Segmentation

SPFNet:Subspace Pyramid Fusion Network for Semantic Segmentation

Fast Monocular Depth Estimation via Side Prediction Aggregation with Continuous Spatial Refinement

Pixel-Level Dense Prediction without Decoder