Abstract:Although fully convolution networks (FCNs) have dominated dense prediction tasks (e.g., semantic seg-mentation, depth estimation and object detection) for decades, they are inherently limited in captur-ing long-range structured relationship with the layers of local kernels. While recent Transformer-based models have proven extremely successful in computer vision tasks by capturing global representation, they would deteriorate dense prediction results by over-smoothing the regions containing fine details (e.g., boundaries and small objects). To this end, we aim to provide an alternative perspective by re-thinking local and global feature representation for the dense prediction task. Specifically, we deploy a Dual-Stream Convolution-Transformer architecture, called DSCT, by taking advantage of both the convolu-tion and Transformer to learn a rich feature representation, combining with a task decoder to provide a powerful dense prediction model. DSCT extracts high resolution local feature representation from convo-lution layers and global feature representation from Transformer layers. With the local and global context modeled explicitly in every layer, the two streams can be combined with a decoder to perform task of se-mantic segmentation, monocular depth estimation or object detection. Extensive experiments show that DSCT can achieve superior performance on the three tasks above. For semantic segmentation, DSCT builds a new state of the art on Cityscapes validation set (83.31% mIoU) with only 80,0 0 0 training iterations and appealing performance (49.27% mIoU) on ADE20K validation set, outperforming most of the alternatives. For monocular depth estimation, our model achieves 2.423 RMSE on KITTI Eigen split, superior to most of the convolution or Transformer counterparts. For object detection, without using FPN, we can achieve 44.5% APb on COCO dataset when using Faster R-CNN, which is higher than Conformer.(c) 2022 Elsevier Ltd. All rights reserved.

Dense Convolution for Semantic Segmentation.

Dense Convolutional Networks for Semantic Segmentation.

Dense-U-Net: Densely Connected Convolutional Network for Semantic Segmentation with a Small Number of Samples

Densely Based Multi-Scale and Multi-Modal Fully Convolutional Networks for High-Resolution Remote-Sensing Image Semantic Segmentation

Rethinking Local and Global Feature Representation for Dense Prediction

Dense-scale Dynamic Network with Filter-Varying Atrous Convolution for Semantic Segmentation

Fully Convolutional Networks for Semantic Segmentation

Better Image Segmentation by Exploiting Dense Semantic Predictions

Enlarging Effective Receptive Field of Convolutional Neural Networks for Better Semantic Segmentation

Dense CNN Learning with Equivalent Mappings

DENSELY CONNECTED LARGE KERNEL CONVOLUTIONAL NETWORK FOR SEMANTIC MEMBRANE SEGMENTATION IN MICROSCOPY IMAGES

Superdense-scale Network for Semantic Segmentation

Hybrid Dilated Convolution Network Using Attentive Kernels for Real-Time Semantic Segmentation

Semantic Segmentation Algorithm Based on Dense Connection and Feature Enhancement

Semantic Segmentation of Remote Sensing Images Using Transfer Learning and Deep Convolutional Neural Network With Dense Connection

Dense Relation Network: Learning Consistent and Context-Aware Representation for Semantic Image Segmentation

Densely connected multidilated convolutional networks for dense prediction tasks

An Enhancement Model Based on Dense Atrous and Inception Convolution for Image Semantic Segmentation

A Lightweight and Dynamic Convolutional Network for Real-time Semantic Segmentation

Dense Transformer Networks

Convolutional Networks with Dense Connectivity