Abstract:RGB thermal semantic segmentation facilitates unmanned platforms to perceive and characterize their surrounding environment, which is critical for autonomous driving tasks. Deep-learning-based algorithms have achieved dominance in terms of accuracy and robustness. However, their large parameter sizes and significant computational demands impede their application in terminal devices. To address this challenge, we propose a novel strategy for achieving a balance between effectiveness and compactness. It includes a robust teacher network, CLNet-T, and a streamlined student network, CLNet-S. Using knowledge distillation (KD), we obtained an optimized model called CLNet-S*. Specifically, CLNet-T and CLNet-S were identical in all aspects except for the feature extraction component. They included a multi-attribute hierarchical feature interaction module (MHFI) and a detail-guided semantic decoder (DGSD). The MHFI initially filters features by considering the characteristics of the low- and high-level features. It gradually combines complementary and common features from various modalities in distinct receptive fields. DGSD uses edge and distribution information to guide semantic decoding, thereby improving the segmentation accuracy at class boundaries. To enhance the performance of the compact student model, our KD strategy includes detail, semantic response distillation (DSRD), and contrastive learning-based feature distillation (CLFD). Practically, DSRD enables the student model to gain knowledge from the teacher model at both the detailed and semantic levels. At the same time, CLFD increases the similarity of features within the same categories and emphasizes the distinctiveness of features between different categories in both the student and teacher models. Extensive experiments conducted on two standard datasets have consistently demonstrated that both CLNet-T and CLNet-S* outperform other state-of-the-art methods. The code and results are available at https://github.com/xiaodonguo/CLNet.

PST900: RGB-Thermal Calibration, Dataset and Segmentation Network

Human Body Parsing in Thermal InfraRed Domain

Modality-transfer Generative Adversarial Network and Dual-Level Unified Latent Representation for Visible Thermal Person Re-Identification

NLFNet: Non-Local Fusion Towards Generalized Multimodal Semantic Segmentation Across RGB-Depth, Polarization, and Thermal Images

Channel and Spatial Relation-Propagation Network for RGB-Thermal Semantic Segmentation

RGB-T Semantic Segmentation with Location, Activation, and Sharpening

Caltech Aerial RGB-Thermal Dataset in the Wild

Residual Spatial Fusion Network for RGB-Thermal Semantic Segmentation

MMSMCNet: Modal Memory Sharing and Morphological Complementary Networks for RGB-T Urban Scene Semantic Segmentation

Semantics from Space: Satellite-Guided Thermal Semantic Segmentation Annotation for Aerial Field Robots

Dual-branch deep cross-modal interaction network for semantic segmentation with thermal images

Mask-guided Modality Difference Reduction Network for RGB-T Semantic Segmentation

Segmenting Objects in Day and Night:Edge-Conditioned CNN for Thermal Image Semantic Segmentation

Contrastive learning-based knowledge distillation for RGB-thermal urban scene semantic segmentation

Region-adaptive and context-complementary cross modulation for RGB-T semantic segmentation

Test-Time Adaptation for Nighttime Color-Thermal Semantic Segmentation

DCFNet: Dense Complementary Fusion for RGB-Thermal Urban Scene Perception

SFAF-MA: Spatial Feature Aggregation and Fusion With Modality Adaptation for RGB-Thermal Semantic Segmentation

Multi-branch Differential Bidirectional Fusion Network for RGB-T Semantic Segmentation

Semantic segmentation of longitudinal thermal images for identification of hot and cool spots in urban areas

Convolutional Neural Networks for Semantic Segmentation as a Tool for Multiclass Face Analysis in Thermal Infrared