Abstract:In recent years, intelligent driving navigation and security monitoring have made considerable progress with the help of deep Convolutional Neural Networks (CNNs). As one of the state-of-the-art perception approaches, semantic segmentation unifies distinct detection tasks widely desired by both autonomous driving and security monitoring. Currently, semantic segmentation shows remarkable efficiency and reliability in standard scenarios such as daytime scenes with favorable illumination conditions. However, in face of adverse conditions such as the nighttime, semantic segmentation loses its accuracy significantly. One of the main causes of the problem is the lack of sufficient annotated segmentation datasets of nighttime scenes. In this paper, we propose a framework to alleviate the accuracy decline when semantic segmentation is taken to adverse conditions by using Generative Adversarial Networks (GANs). To bridge the daytime and nighttime image domains, we made key observation that compared to datasets in adverse conditions, there are considerable amount of segmentation datasets in standard conditions such as BDD and our collected ZJU datasets. Our GAN-based nighttime semantic segmentation framework includes two methods. In the first method, GANs were used to translate nighttime images to the daytime, thus semantic segmentation can be performed using robust models already trained on daytime datasets. In another method, we use GANs to translate different ratio of daytime images in the dataset to the nighttime but still with their labels. In this sense, synthetic nighttime segmentation datasets can be generated to yield models prepared to operate at nighttime conditions robustly. In our experiment, the later method significantly boosts the performance at the nighttime evidenced by quantitative results using Intersection over Union (IoU) and Pixel Accuracy (Acc). We show that the performance varies with respect to the proportion of synthetic nighttime images in the dataset, where the sweet spot corresponds to most robust performance across the day and night. The proposed framework not only makes contribution to the optimization of visual perception in intelligent vehicles, but also can be applied to diverse navigational assistance systems.

Research on Transmission Line Ice-Cover Segmentation Based on Improved U-Net and GAN

Real-time Water Area Segmentation for USV using Enhanced U-Net

See Clearer at Night: Towards Robust Nighttime Semantic Segmentation Through Day-Night Image Conversion

A Multi-Task Network with Dynamic Segmentation for Sea Ice Classification in Arctic Shipping Route Optimization

ISDNet: Integrating Shallow and Deep Networks for Efficient Ultra-high Resolution Segmentation

TL-PVCNN: A Point Cloud Semantic Segmentation Framework on Transmission Line Scene

S Seg: A Three-Stage Unsupervised Foreground and Background Segmentation Network

EDUNet++: An Enhanced Denoising Unet++ for Ice-Covered Transmission Line Images

CSEU-Net: ConvNeXt-SE-U-Net for river ice floe segmentation using unmanned aerial vehicle grayscale remote sensing images

A Deeply Supervised Semantic Segmentation Method Based on GAN

GMSA-Net: A Transmission Line Ice Thickness Identification Network Based on Global Micro Strip Awareness

CM-Unet: A Novel Remote Sensing Image Segmentation Method Based on Improved U-Net

Deep Attention Unet: A Network Model with Global Feature Perception Ability

A Semantic Segmentation Method Based on AS-Unet++ for Power Remote Sensing of Images

DSGA-Net: Deeply Separable Gated Transformer and Attention Strategy for Medical Image Segmentation Network

CloudU-Netv2: A Cloud Segmentation Method for Ground-Based Cloud Images Based on Deep Learning

Research on semantic segmentation algorithm of high latitude urban river ice based on deep transfer learning

GCT-UNET: U-Net Image Segmentation Model for a Small Sample of Adherent Bone Marrow Cells Based on a Gated Channel Transform Module

Class-aware edge-assisted lightweight semantic segmentation network for power transmission line inspection

UNet#: A UNet-like Redesigning Skip Connections for Medical Image Segmentation

Enhancing Efficient Global Understanding Network with CSWin Transformer for Urban Scene Images Segmentation