An Open-Source Project for Real-Time Image Semantic Segmentation

Quan Zhou,Yu Wang,Jia Liu,Xin Jin,Longin Jan Latecki

DOI: https://doi.org/10.1007/s11432-019-2685-1

2019-01-01

Science China Information Sciences

Abstract:>Recent years have witnessed great progress of deep convolutional neural networks (DCNNs) for solving scene understanding tasks [1–3]. These advances prefer to construct deeper and larger net-

What problem does this paper attempt to address?

See Clearer at Night: Towards Robust Nighttime Semantic Segmentation Through Day-Night Image Conversion

Lei Sun,Kaiwei Wang,Kailun Yang,Kaite Xiang

DOI: https://doi.org/10.1117/12.2532477

2019-01-01

Abstract:In recent years, intelligent driving navigation and security monitoring have made considerable progress with the help of deep Convolutional Neural Networks (CNNs). As one of the state-of-the-art perception approaches, semantic segmentation unifies distinct detection tasks widely desired by both autonomous driving and security monitoring. Currently, semantic segmentation shows remarkable efficiency and reliability in standard scenarios such as daytime scenes with favorable illumination conditions. However, in face of adverse conditions such as the nighttime, semantic segmentation loses its accuracy significantly. One of the main causes of the problem is the lack of sufficient annotated segmentation datasets of nighttime scenes. In this paper, we propose a framework to alleviate the accuracy decline when semantic segmentation is taken to adverse conditions by using Generative Adversarial Networks (GANs). To bridge the daytime and nighttime image domains, we made key observation that compared to datasets in adverse conditions, there are considerable amount of segmentation datasets in standard conditions such as BDD and our collected ZJU datasets. Our GAN-based nighttime semantic segmentation framework includes two methods. In the first method, GANs were used to translate nighttime images to the daytime, thus semantic segmentation can be performed using robust models already trained on daytime datasets. In another method, we use GANs to translate different ratio of daytime images in the dataset to the nighttime but still with their labels. In this sense, synthetic nighttime segmentation datasets can be generated to yield models prepared to operate at nighttime conditions robustly. In our experiment, the later method significantly boosts the performance at the nighttime evidenced by quantitative results using Intersection over Union (IoU) and Pixel Accuracy (Acc). We show that the performance varies with respect to the proportion of synthetic nighttime images in the dataset, where the sweet spot corresponds to most robust performance across the day and night. The proposed framework not only makes contribution to the optimization of visual perception in intelligent vehicles, but also can be applied to diverse navigational assistance systems.
Real-time Semantic Segmentation with Weighted Factorized-Depthwise Convolution

Xiaochen Hao,Xingjun Hao,Yaru Zhang,Yuanyuan Li,Chao Wu

DOI: https://doi.org/10.1016/j.imavis.2021.104269

IF: 3.86

2021-01-01

Image and Vision Computing

Abstract:Semantic segmentation has achieved great success with the popularity of convolutional neural networks (CNNs). However, the huge computational burden restricts the application of most existing networks on edge devices with strict inference time constraints. To solve this problem, a weighted factorized-depthwise convolution network (WFDCNet) is presented in this paper, which contains full- dimensional continuous separation convolution (FCS) modules and a lateral asymmetric pyramid fusion (LAPF) module, aiming to obtain high accuracy without damaging inference speed. Specifically, the FCS module enables the calculation of each dimension to be completed independently in a continuous separation process and uses simplified SE (SSE) attention layer to adjust the channels, achieving the extensive extraction of feature information. The LAPF module is able to eliminate semantic divergence and fuse feature maps of three different scales to realize the combination of multiple information from the front-end and the back-end network. WFDCNet shows superior performance on Cityscapes, Camvid, Mapillary Vistas and COCO-Stuff datasets. Especially, the experimental results demonstrate that our network achieves 73.7% mIoU on Cityscapes dataset, with the inference speed of 102.6FPS on a single RTX 2080 Ti GPU, and 17.2FPS on Jetson TX2.
Omnisupervised Omnidirectional Semantic Segmentation

Kailun Yang,Xinxin Hu,Yicheng Fang,Kaiwei Wang,Rainer Stiefelhagen

DOI: https://doi.org/10.1109/tits.2020.3023331

IF: 8.5

2020-01-01

IEEE Transactions on Intelligent Transportation Systems

Abstract:Modern efficient Convolutional Neural Networks (CNNs) are able to perform semantic segmentation both swiftly and accurately, which covers typically separate detection tasks desired by Intelligent Vehicles (IV) in a unified way. Most of the current semantic perception frameworks are designed to work with pinhole cameras and benchmarked against public datasets with narrow Field-of-View (FoV) images. However, there is a large accuracy downgrade when a pinhole-yielded CNN is taken to omnidirectional imagery, causing it unreliable for surrounding perception. In this paper, we propose an omnisupervised learning framework for efficient CNNs, which bridges multiple heterogeneous data sources that are already available in the community, bypassing the labor-intensive process to have manually annotated panoramas, while improving their reliability in unseen omnidirectional domains. Being omnisupervised, the efficient CNN exploits both labeled pinhole images and unlabeled panoramas. The framework is based on our specialized ensemble method that considers the wide-angle and wrap-around features of omnidirectional images, to automatically generate panoramic labels for data distillation. A comprehensive variety of experiments demonstrates that the proposed solution helps to attain significant generalizability gains in panoramic imagery domains. Our approach outperforms state-of-the-art efficient segmenters on highly unconstrained IDD20K and PASS datasets.
A Top-Down Manner-Based DCNN Architecture for Semantic Image Segmentation.

Kai Qiao,Jian Chen,Linyuan Wang,Lei Zeng,Bin Yan

DOI: https://doi.org/10.1371/journal.pone.0174508

IF: 3.7

2017-01-01

PLoS ONE

Abstract:Given their powerful feature representation for recognition, deep convolutional neural networks (DCNNs) have been driving rapid advances in high-level computer vision tasks. However, their performance in semantic image segmentation is still not satisfactory. Based on the analysis of visual mechanism, we conclude that DCNNs in a bottom-up manner are not enough, because semantic image segmentation task requires not only recognition but also visual attention capability. In the study, superpixels containing visual attention information are introduced in a top-down manner, and an extensible architecture is proposed to improve the segmentation results of current DCNN-based methods. We employ the current state-of-the-art fully convolutional network (FCN) and FCN with conditional random field (DeepLab-CRF) as baselines to validate our architecture. Experimental results of the PASCAL VOC segmentation task qualitatively show that coarse edges and error segmentation results are well improved. We also quantitatively obtain about 2%-3% intersection over union (IOU) accuracy improvement on the PASCAL VOC 2011 and 2012 test sets.
Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs

Liang-Chieh Chen,George Papandreou,Iasonas Kokkinos,Kevin Murphy,Alan L. Yuille

DOI: https://doi.org/10.48550/arXiv.1412.7062

2014-12-22

Computer Vision and Pattern Recognition

Abstract:Deep Convolutional Neural Networks (DCNNs) have recently shown state of the art performance in high level vision tasks, such as image classification and object detection. This work brings together methods from DCNNs and probabilistic graphical models for addressing the task of pixel-level classification (also called "semantic image segmentation"). We show that responses at the final layer of DCNNs are not sufficiently localized for accurate object segmentation. This is due to the very invariance properties that make DCNNs good for high level tasks. We overcome this poor localization property of deep networks by combining the responses at the final DCNN layer with a fully connected Conditional Random Field (CRF). Qualitatively, our "DeepLab" system is able to localize segment boundaries at a level of accuracy which is beyond previous methods. Quantitatively, our method sets the new state-of-art at the PASCAL VOC-2012 semantic image segmentation task, reaching 71.6% IOU accuracy in the test set. We show how these results can be obtained efficiently: Careful network re-purposing and a novel application of the 'hole' algorithm from the wavelet community allow dense computation of neural net responses at 8 frames per second on a modern GPU.
Real-Time High-Performance Semantic Image Segmentation of Urban Street Scenes

Genshun Dong,Yan Yan,Chunhua Shen,Hanzi Wang

DOI: https://doi.org/10.48550/arXiv.2003.08736

2020-04-03

Abstract:Deep Convolutional Neural Networks (DCNNs) have recently shown outstanding performance in semantic image segmentation. However, state-of-the-art DCNN-based semantic segmentation methods usually suffer from high computational complexity due to the use of complex network architectures. This greatly limits their applications in the real-world scenarios that require real-time processing. In this paper, we propose a real-time high-performance DCNN-based method for robust semantic segmentation of urban street scenes, which achieves a good trade-off between accuracy and speed. Specifically, a Lightweight Baseline Network with Atrous convolution and Attention (LBN-AA) is firstly used as our baseline network to efficiently obtain dense feature maps. Then, the Distinctive Atrous Spatial Pyramid Pooling (DASPP), which exploits the different sizes of pooling operations to encode the rich and distinctive semantic information, is developed to detect objects at multiple scales. Meanwhile, a Spatial detail-Preserving Network (SPN) with shallow convolutional layers is designed to generate high-resolution feature maps preserving the detailed spatial information. Finally, a simple but practical Feature Fusion Network (FFN) is used to effectively combine both shallow and deep features from the semantic branch (DASPP) and the spatial branch (SPN), respectively. Extensive experimental results show that the proposed method respectively achieves the accuracy of 73.6% and 68.0% mean Intersection over Union (mIoU) with the inference speed of 51.0 fps and 39.3 fps on the challenging Cityscapes and CamVid test datasets (by only using a single NVIDIA TITAN X card). This demonstrates that the proposed method offers excellent performance at the real-time speed for semantic segmentation of urban street scenes.

Computer Vision and Pattern Recognition
Recent Progress in Semantic Image Segmentation

Xiaolong Liu,Zhidong Deng,Yuhan Yang

DOI: https://doi.org/10.1007/s10462-018-9641-3

IF: 9.588

2018-01-01

Artificial Intelligence Review

Abstract:Semantic image segmentation, which becomes one of the key applications in image processing and computer vision domain, has been used in multiple domains such as medical area and intelligent transportation. Lots of benchmark datasets are released for researchers to verify their algorithms. Semantic segmentation has been studied for many years. Since the emergence of Deep Neural Network (DNN), segmentation has made a tremendous progress. In this paper, we divide semantic image segmentation methods into two categories: traditional and recent DNN method. Firstly, we briefly summarize the traditional method as well as datasets released for segmentation, then we comprehensively investigate recent methods based on DNN which are described in the eight aspects: fully convolutional network, up-sample ways, FCN joint with CRF methods, dilated convolution approaches, progresses in backbone network, pyramid methods, Multi-level feature and multi-stage method, supervised, weakly-supervised and unsupervised methods. Finally, a conclusion in this area is drawn.
Semantic Segmentation of Aerial Imagery Via Split-Attention Networks with Disentangled Nonlocal and Edge Supervision

Cheng Zhang,Wanshou Jiang,Qing Zhao

DOI: https://doi.org/10.3390/rs13061176

IF: 5

2021-01-01

Remote Sensing

Abstract:In this work, we propose a new deep convolution neural network (DCNN) architecture for semantic segmentation of aerial imagery. Taking advantage of recent research, we use split-attention networks (ResNeSt) as the backbone for high-quality feature expression. Additionally, a disentangled nonlocal (DNL) block is integrated into our pipeline to express the inter-pixel long-distance dependence and highlight the edge pixels simultaneously. Moreover, the depth-wise separable convolution and atrous spatial pyramid pooling (ASPP) modules are combined to extract and fuse multiscale contextual features. Finally, an auxiliary edge detection task is designed to provide edge constraints for semantic segmentation. Evaluation of algorithms is conducted on two benchmarks provided by the International Society for Photogrammetry and Remote Sensing (ISPRS). Extensive experiments demonstrate the effectiveness of each module of our architecture. Precision evaluation based on the Potsdam benchmark shows that the proposed DCNN achieves competitive performance over the state-of-the-art methods.
DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs

Liang-Chieh Chen,George Papandreou,Iasonas Kokkinos,Kevin Murphy,Alan L. Yuille

DOI: https://doi.org/10.1109/tpami.2017.2699184

IF: 23.6

2018-04-01

IEEE Transactions on Pattern Analysis and Machine Intelligence

Abstract:In this work we address the task of semantic image segmentation with Deep Learning and make three main contributions that are experimentally shown to have substantial practical merit. First, we highlight convolution with upsampled filters, or 'atrous convolution', as a powerful tool in dense prediction tasks. Atrous convolution allows us to explicitly control the resolution at which feature responses are computed within Deep Convolutional Neural Networks. It also allows us to effectively enlarge the field of view of filters to incorporate larger context without increasing the number of parameters or the amount of computation. Second, we propose atrous spatial pyramid pooling (ASPP) to robustly segment objects at multiple scales. ASPP probes an incoming convolutional feature layer with filters at multiple sampling rates and effective fields-of-views, thus capturing objects as well as image context at multiple scales. Third, we improve the localization of object boundaries by combining methods from DCNNs and probabilistic graphical models. The commonly deployed combination of max-pooling and downsampling in DCNNs achieves invariance but has a toll on localization accuracy. We overcome this by combining the responses at the final DCNN layer with a fully connected Conditional Random Field (CRF), which is shown both qualitatively and quantitatively to improve localization performance. Our proposed "DeepLab" system sets the new state-of-art at the PASCAL VOC-2012 semantic image segmentation task, reaching 79.7 percent mIOU in the test set, and advances the results on three other datasets: PASCAL-Context, PASCAL-Person-Part, and Cityscapes. All of our code is made publicly available online.

computer science, artificial intelligence,engineering, electrical & electronic
A Survey of Image Semantic Segmentation Algorithm Based on Deep Learning

Jian Chen,Fen Luo

DOI: https://doi.org/10.54097/ajst.v5i1.5248

2023-01-01

Academic Journal of Science and Technology

Abstract:Image semantic segmentation technology is one of the core research contents in the field of computer vision, and has a wide range of applications in production and life. With the improvement of computer performance and the continuous development of deep learning technology, researchers have increasingly high research enthusiasm for the performance of image semantic segmentation. This paper summarizes the research status of image semantic segmentation based on deep learning and introduces the common datasets used in the field of semantic segmentation. Finally, we point out the existing problems and future development trend of image semantic segmentation algorithms.
Semantic Segmentation With Light Field Imaging and Convolutional Neural Networks

Chen Jia,Fan Shi,Meng Zhao,Yao Zhang,Xu Cheng,Mianzhao Wang,Shengyong Chen

DOI: https://doi.org/10.1109/tim.2021.3115204

IF: 5.6

2021-01-01

IEEE Transactions on Instrumentation and Measurement

Abstract:Semantic segmentation is of great importance and a challenge in computer vision. One of its main problems is how to efficiently obtain rich information (geometric structure) and identify useful features from higher dimensions. A light field camera, due to its special microlens array structure, can completely record the angular-spatial information of scenes, which is attractive and has great potential to improve the performance of semantic segmentation tasks. Inspired by this, we propose an end-to-end semantic segmentation network that can process light field macropixel images robustly and extract their features. In addition, this network can flexibly and efficiently load the different popular deep learning backbones. Furthermore, we propose an efficient angular model, which, to learn the angular features between the different viewpoints of the macropixel image, improves the nonlinearity of angular-spatial features and enhances multichannel semantic correlations. To evaluate the network, we construct a new real scene light field dataset comprising 800 high-quality samples. The quantitative and qualitative results show that the highest mean intersection over union (mIoU) based on our algorithm is greater than 57%. Our algorithm achieves a 10.30% increase compared with state-of-the-art semantic segmentation algorithms. In combination with different backbones or multiscale light field macropixel images, the network can also achieve comparable results. This preliminary work demonstrates that the combination of light field imaging and deep learning technology has potential applications in the future study of semantic segmentation.

engineering, electrical & electronic,instruments & instrumentation
SegNet Network Architecture for Deep Learning Image Segmentation and Its Integrated Applications and Prospects

Chenwei Zhang,Wenran Lu,Jiang Wu,Chunhe Ni,Hongbo Wang

DOI: https://doi.org/10.54097/rfa5x119

2024-02-26

Academic Journal of Science and Technology

Abstract:Semantic image segmentation is a crucial task in computer vision, with applications ranging from autonomous driving to medical image analysis. In recent years, deep learning has revolutionized this field, leading to the development of various neural network models aimed at improving segmentation accuracy. One such architecture is SegNet, which we explore in this article.SegNet's architecture consists of an encoder network, a corresponding decoder network, and a pixel-wise classification layer. The encoder network, resembling VGG16 with 13 convolutional layers, extracts high-level features from input images. The innovation lies in the decoder network's approach to upsampling, utilizing pooled indices from the encoder's maximum pooling step to perform non-linear up sampling. This eliminates the need for additional learning during up sampling, making SegNet efficient in both storage and computation.SegNet represents an exciting advancement in deep learning image segmentation. Its efficient architecture, memory-conscious design, and potential for real-time applications make it a valuable tool in the field of computer vision with promising integrated applications and prospects.
Fast Semantic Segmentation for Scene Perception

Xuetao Zhang,Zhenxue Chen,Q. M. Jonathan Wu,Lei Cai,Dan Lu,Xianming Li

DOI: https://doi.org/10.1109/tii.2018.2849348

IF: 12.3

2019-01-01

IEEE Transactions on Industrial Informatics

Abstract:Semantic segmentation is a challenging problem in computer vision. Many applications, such as autonomous driving and robot navigation with urban road scene, need accurate and efficient segmentation. Most state-of-the-art methods focus on accuracy, rather than efficiency. In this paper, we propose a more efficient neural network architecture, which has fewer parameters, for semantic segmentation in the urban road scene. An asymmetric encoder-decoder structure based on ResNet is used in our model. In the first stage of encoder, we use continuous factorized block to extract low-level features. Continuous dilated block is applied in the second stage, which ensures that the model has a larger view field, while keeping the model small-scale and shallow. The down sampled features from encoder are up sampled with decoder to the same-size output as the input image and the details refined. Our model can achieve end-to-end and pixel-to-pixel training without pretraining from scratch. The parameters of our model are only 0.2M, 100x less than those of others such as SegNet, etc. Experiments are conducted on five public road scene datasets (CamVid, CityScapes, Gatech, KITTI Road Detection, and KITTI Semantic Segmentation), and the results demonstrate that our model can achieve better performance.
Real time backbone for semantic segmentation

Zhengeng Yang,Hongshan Yu,Qiang Fu,Wei Sun,Wenyan Jia,Mingui Sun,Zhi-Hong Mao

DOI: https://doi.org/10.48550/arXiv.1903.06922

2019-03-16

Abstract:The rapid development of autonomous driving in recent years presents lots of challenges for scene understanding. As an essential step towards scene understanding, semantic segmentation thus received lots of attention in past few years. Although deep learning based state-of-the-arts have achieved great success in improving the segmentation accuracy, most of them suffer from an inefficiency problem and can hardly applied to practical applications. In this paper, we systematically analyze the computation cost of Convolutional Neural Network(CNN) and found that the inefficiency of CNN is mainly caused by its wide structure rather than the deep structure. In addition, the success of pruning based model compression methods proved that there are many redundant channels in CNN. Thus, we designed a very narrow while deep backbone network to improve the efficiency of semantic segmentation. By casting our network to FCN32 segmentation architecture, the basic structure of most segmentation methods, we achieved 60.6\% mIoU on Cityscape val dataset with 54 frame per seconds(FPS) on $1024\times2048$ inputs, which already outperforms one of the earliest real time deep learning based segmentation methods: ENet.

Computer Vision and Pattern Recognition
Very Fast Semantic Image Segmentation Using Hierarchical Dilation and Feature Refining

Qingqun Ning,Jianke Zhu,Chun Chen

DOI: https://doi.org/10.1007/s12559-017-9530-0

IF: 4.89

2017-01-01

Cognitive Computation

Abstract:With the rapid development of deep learning techniques, semantic image segmentation has been considerably improved recently, which is viewed as the key problem of scene understanding in computer vision. These advances are built upon the capability of complex architectures for deep neural network. In this paper, we present a novel deep neural network architecture designed for semantic image segmentation. In order to improve the segmentation accuracy, we introduce a novel hierarchical dilation block to effectively enlarge the size of receptive field and enable multi-scale processing in fully convolutional neural network. Moreover, we exploit the technique of bypass and intermediate supervision to capture the context information during upsampling and refining coarse features. We have conducted extensive experiments on several popular semantic segmentation testbeds, including Cityscapes, CamVid, Kitti, and Helen facial datasets. The experimental results demonstrate that our proposed approach runs two times faster than the state-of-the-art method. Our full system is able to obtain realtime inference performance on 1080P images using a PC with single GPU. It executes a network forwarding at 200fps in our experiment while retaining high accuracy. Our proposed approach not only runs faster than the existing realtime methods but also performs on par with them.
DARSegNet: A Real-Time Semantic Segmentation Method Based on Dual Attention Fusion Module and Encoder-Decoder Network

Yongfeng Xing,Luo Zhong,Xian Zhong

DOI: https://doi.org/10.1155/2022/6195148

IF: 1.43

2022-06-08

Mathematical Problems in Engineering

Abstract:The convolutional neural network achieves excellent semantic segmentation results in artificially annotated datasets with complex scenes. However, semantic segmentation methods still suffer from several problems such as low use rate of the features, high computational complexity, and being far from practical real-time application, which bring about challenges for the image semantic segmentation. Two factors are very critical to semantic segmentation task: global context and multilevel semantics. However, generating these two factors will always lead to high complexity. In order to solve this, we propose a novel structure, dual attention fusion module (DAFM), by eliminating structural redundancy. Unlike most of the existing algorithms, we combine the attention mechanism with the depth pyramid pool module (DPPM) to extract accurate dense features for pixel labeling rather than complex expansion convolution. Specifically, we introduce a DPPM to execute the spatial pyramid structure in output and combine the global pool method. The DAFM is introduced in each decoder layer. Finally, the low-level features and high-level features are fused to obtain semantic segmentation result. The experiments and visualization results on Cityscapes and CamVid datasets show that, in real-time semantic segmentation, we have achieved a satisfactory balance between accuracy and speed, which proves the effectiveness of the proposed algorithm. In particular, on a single 1080ti GPU computer, ResNet-18 produces 75.53% MIoU at 70 FPS on Cityscapes and 73.96% MIoU at 109 FPS on CamVid.

engineering, multidisciplinary,mathematics, interdisciplinary applications
MDNet: A Multi-image Input Real-Time Semantic Segmentation Model with Decoupled Supervision

Yunze Wu

DOI: https://doi.org/10.1145/3471287.3471301

2021-05-27

Abstract:Current state-of-the-art Real-Time Semantic Segmentation Model is still not fast enough. They spend too much time on processing images in a deep CNN to grab the spatial and context information. Somehow, this information may not be so deterministic. In this work, we come up with a multi-image input real-time semantic segmentation model with decoupled label supervision. It can decrease the computational time and keep a relatively high precision of semantic segmentation meanwhile. The novelty of our model lies is picking up the decoupled label supervision to be our loss function and combining it with a multi-branch image processing framework. The edge detection module can not only improve the recognition of the differences between object body and edge but also guarantee the processing procedure of our network to be faster enough. Apart from this, the multi-branch image processing framework is not a burden of running time. Our network is trained on difficult datasets like CamVid and has favourable quality in real-time testing. The mean class IoU of our network is 66.6. It is the highest one among all of the other comparisons.
Real-time Semantic Segmentation for Road Scene

Xuetao Zhang,Zhenxue Chen,Dan Lu,Xianming Li

DOI: https://doi.org/10.1109/ICARM.2018.8610868

2018-01-01

Abstract:Semantic segmentation is challenging for diverse scenes. In this paper, we propose a new model for road complex scene. Our model use encoder-decoder structure with auxiliary loss. Based on ResNet bottleneck block, we proposed dilated bottleneck block and tiny block. These block applied in the encoder and decoder. The dilated bottleneck block enlarges the field-of-view and the tiny block maintains the model as small as possible. We train our model end-to-end from scartch, and the image and segmentation map in network is pixel-to-pixel. With the help of auxiliary loss, our model yields 56.9% mean IoU on CamVid dataset, it is smaller and faster than ENet and SegNet.
Medical Image Semantic Segmentation Based on Deep Learning

Feng Jiang,Aleksei Grigorev,Seungmin Rho,Zhihong Tian,YunSheng Fu,Worku Jifara,Khan Adil,Shaohui Liu

DOI: https://doi.org/10.1007/s00521-017-3158-6

2017-01-01

Neural Computing and Applications

Abstract:The image semantic segmentation has been extensively studying. The modern methods rely on the deep convolutional neural networks, which can be trained to address this problem. A few years ago networks require the huge dataset to be trained. However, the recent advances in deep learning allow training networks on the small datasets, which is a critical issue for medical images, since the hospitals and research organizations usually do not provide the huge amount of data. In this paper, we address medical image semantic segmentation problem by applying the modern CNN model. Moreover, the recent achievements in deep learning allow processing the whole image per time by applying concepts of the fully convolutional neural network. Our qualitative and quantitate experiment results demonstrated that modern CNN can successfully tackle the medical image semantic segmentation problem.
Fully Convolutional Networks for Semantic Segmentation

Jonathan Long,Evan Shelhamer,Trevor Darrell

DOI: https://doi.org/10.48550/arXiv.1411.4038

2015-03-09

Abstract:Convolutional networks are powerful visual models that yield hierarchies of features. We show that convolutional networks by themselves, trained end-to-end, pixels-to-pixels, exceed the state-of-the-art in semantic segmentation. Our key insight is to build "fully convolutional" networks that take input of arbitrary size and produce correspondingly-sized output with efficient inference and learning. We define and detail the space of fully convolutional networks, explain their application to spatially dense prediction tasks, and draw connections to prior models. We adapt contemporary classification networks (AlexNet, the VGG net, and GoogLeNet) into fully convolutional networks and transfer their learned representations by fine-tuning to the segmentation task. We then define a novel architecture that combines semantic information from a deep, coarse layer with appearance information from a shallow, fine layer to produce accurate and detailed segmentations. Our fully convolutional network achieves state-of-the-art segmentation of PASCAL VOC (20% relative improvement to 62.2% mean IU on 2012), NYUDv2, and SIFT Flow, while inference takes one third of a second for a typical image.

Computer Vision and Pattern Recognition

An Open-Source Project for Real-Time Image Semantic Segmentation

See Clearer at Night: Towards Robust Nighttime Semantic Segmentation Through Day-Night Image Conversion

Real-time Semantic Segmentation with Weighted Factorized-Depthwise Convolution

Omnisupervised Omnidirectional Semantic Segmentation

A Top-Down Manner-Based DCNN Architecture for Semantic Image Segmentation.

Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs

Real-Time High-Performance Semantic Image Segmentation of Urban Street Scenes

Recent Progress in Semantic Image Segmentation

Semantic Segmentation of Aerial Imagery Via Split-Attention Networks with Disentangled Nonlocal and Edge Supervision

DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs

A Survey of Image Semantic Segmentation Algorithm Based on Deep Learning

Semantic Segmentation With Light Field Imaging and Convolutional Neural Networks

SegNet Network Architecture for Deep Learning Image Segmentation and Its Integrated Applications and Prospects

Fast Semantic Segmentation for Scene Perception

Real time backbone for semantic segmentation

Very Fast Semantic Image Segmentation Using Hierarchical Dilation and Feature Refining

DARSegNet: A Real-Time Semantic Segmentation Method Based on Dual Attention Fusion Module and Encoder-Decoder Network

MDNet: A Multi-image Input Real-Time Semantic Segmentation Model with Decoupled Supervision

Real-time Semantic Segmentation for Road Scene

Medical Image Semantic Segmentation Based on Deep Learning

Fully Convolutional Networks for Semantic Segmentation