Abstract:Semantic segmentation is important for scene understanding. To address the scenes of adverse illumination conditions of natural images, thermal infrared (TIR) images are introduced. Most existing RGB-T semantic segmentation methods follow three cross-modal fusion paradigms, i.e. encoder fusion, decoder fusion, and feature fusion. Some methods, unfortunately, ignore the properties of RGB and TIR features or the properties of features at different levels. In this paper, we propose a novel feature fusion-based network for RGB-T semantic segmentation, named \emph{LASNet}, which follows three steps of location, activation, and sharpening. The highlight of LASNet is that we fully consider the characteristics of cross-modal features at different levels, and accordingly propose three specific modules for better segmentation. Concretely, we propose a Collaborative Location Module (CLM) for high-level semantic features, aiming to locate all potential objects. We propose a Complementary Activation Module for middle-level features, aiming to activate exact regions of different objects. We propose an Edge Sharpening Module (ESM) for low-level texture features, aiming to sharpen the edges of objects. Furthermore, in the training phase, we attach a location supervision and an edge supervision after CLM and ESM, respectively, and impose two semantic supervisions in the decoder part to facilitate network convergence. Experimental results on two public datasets demonstrate that the superiority of our LASNet over relevant state-of-the-art methods. The code and results of our method are available at <a class="link-external link-https" href="https://github.com/MathLee/LASNet" rel="external noopener nofollow">this https URL</a>.

Rgb-t semantic segmentation based on cross-operational fusion attention in autonomous driving scenario

NLFNet: Non-Local Fusion Towards Generalized Multimodal Semantic Segmentation Across RGB-Depth, Polarization, and Thermal Images

Multimodal Image Semantic Segmentation Based on Attention Mechanism

Real-Time Fusion Network for RGB-D Semantic Segmentation Incorporating Unexpected Obstacle Detection for Road-Driving Images

CCAFFMNet: Dual-spectral semantic segmentation network with channel-coordinate attention feature fusion module

Multispectral Fusion Transformer Network for RGB-Thermal Urban Scene Semantic Segmentation

Residual Spatial Fusion Network for RGB-Thermal Semantic Segmentation

A RGB-Thermal Image Segmentation Method Based on Parameter Sharing and Attention Fusion for Safe Autonomous Driving

RGB and LiDAR Fusion-based 3D Semantic Segmentation for Autonomous Driving

An RGB-D Fusion Based Semantic Segmentation Algorithm Based on Neighborhood Metric Relations

Real-time Semantic Segmentation in Traffic Scene Using Cross Stage Partial-based Encoder–decoder Network

Cross-modal Attention Fusion Network for RGB-D Semantic Segmentation

MLFNet: Multi-Level Fusion Network for Real-Time Semantic Segmentation of Autonomous Driving

FuseSeg: Semantic Segmentation of Urban Scenes Based on RGB and Thermal Data Fusion

Dual-branch deep cross-modal interaction network for semantic segmentation with thermal images

Bimodal Feature Propagation and Fusion for Real-time Semantic Segmentation on RGB-D Images

DARSegNet: A Real-Time Semantic Segmentation Method Based on Dual Attention Fusion Module and Encoder-Decoder Network

FEANet: Feature-Enhanced Attention Network for RGB-Thermal Real-time Semantic Segmentation

Robust Semantic Segmentation Based on RGB-thermal in Variable Lighting Scenes

A Real-Time Semantic Segmentation Approach for Autonomous Driving Scenes

RGB-T Semantic Segmentation with Location, Activation, and Sharpening