Abstract:RGB thermal semantic segmentation facilitates unmanned platforms to perceive and characterize their surrounding environment, which is critical for autonomous driving tasks. Deep-learning-based algorithms have achieved dominance in terms of accuracy and robustness. However, their large parameter sizes and significant computational demands impede their application in terminal devices. To address this challenge, we propose a novel strategy for achieving a balance between effectiveness and compactness. It includes a robust teacher network, CLNet-T, and a streamlined student network, CLNet-S. Using knowledge distillation (KD), we obtained an optimized model called CLNet-S*. Specifically, CLNet-T and CLNet-S were identical in all aspects except for the feature extraction component. They included a multi-attribute hierarchical feature interaction module (MHFI) and a detail-guided semantic decoder (DGSD). The MHFI initially filters features by considering the characteristics of the low- and high-level features. It gradually combines complementary and common features from various modalities in distinct receptive fields. DGSD uses edge and distribution information to guide semantic decoding, thereby improving the segmentation accuracy at class boundaries. To enhance the performance of the compact student model, our KD strategy includes detail, semantic response distillation (DSRD), and contrastive learning-based feature distillation (CLFD). Practically, DSRD enables the student model to gain knowledge from the teacher model at both the detailed and semantic levels. At the same time, CLFD increases the similarity of features within the same categories and emphasizes the distinctiveness of features between different categories in both the student and teacher models. Extensive experiments conducted on two standard datasets have consistently demonstrated that both CLNet-T and CLNet-S* outperform other state-of-the-art methods. The code and results are available at https://github.com/xiaodonguo/CLNet.

MGSGNet-S*: Multilayer Guided Semantic Graph Network Via Knowledge Distillation for RGB-Thermal Urban Scene Parsing

NLFNet: Non-Local Fusion Towards Generalized Multimodal Semantic Segmentation Across RGB-Depth, Polarization, and Thermal Images

A Scalable Real-time Semantic Segmentation Network for Autonomous Driving

GSGNet-S*: Graph Semantic Guidance Network Via Knowledge Distillation for Optical Remote Sensing Image Scene Analysis

Real-time Semantic Segmentation in Traffic Scene Using Cross Stage Partial-based Encoder–decoder Network

Contrastive learning-based knowledge distillation for RGB-thermal urban scene semantic segmentation

Real-time Semantic Segmentation Network for Urban Scenes Based on Global Attention Mechanism

NDNet: Spacewise Multiscale Representation Learning via Neighbor Decoupling for Real-Time Driving Scene Parsing

SPNet: an RGB-D Sequence Progressive Network for Road Semantic Segmentation.

MTANet: Multitask-Aware Network with Hierarchical Multimodal Fusion for RGB-T Urban Scene Understanding

MMSMCNet: Modal Memory Sharing and Morphological Complementary Networks for RGB-T Urban Scene Semantic Segmentation

Graph Attention Guidance Network with Knowledge Distillation for Semantic Segmentation of Remote Sensing Images.

SPGNet: Semantic Prediction Guidance for Scene Parsing

Edge-aware Guidance Fusion Network for RGB Thermal Scene Parsing

Lightweight cross-guided contextual perceptive network for visible–infrared urban road scene parsing

SYGNET: A SVD-YOLO BASED GHOSTNET FOR REAL-TIME DRIVING SCENE PARSING

Deep Dual-resolution Networks for Real-time and Accurate Semantic Segmentation of Road Scenes

Multifeature Selective Fusion Network for Real-Time Driving Scene Parsing

Multispectral Fusion Transformer Network for RGB-Thermal Urban Scene Semantic Segmentation

Revisiting Multi-modal 3D Semantic Segmentation in Real-world Autonomous Driving