Abstract:RGB thermal semantic segmentation facilitates unmanned platforms to perceive and characterize their surrounding environment, which is critical for autonomous driving tasks. Deep-learning-based algorithms have achieved dominance in terms of accuracy and robustness. However, their large parameter sizes and significant computational demands impede their application in terminal devices. To address this challenge, we propose a novel strategy for achieving a balance between effectiveness and compactness. It includes a robust teacher network, CLNet-T, and a streamlined student network, CLNet-S. Using knowledge distillation (KD), we obtained an optimized model called CLNet-S*. Specifically, CLNet-T and CLNet-S were identical in all aspects except for the feature extraction component. They included a multi-attribute hierarchical feature interaction module (MHFI) and a detail-guided semantic decoder (DGSD). The MHFI initially filters features by considering the characteristics of the low- and high-level features. It gradually combines complementary and common features from various modalities in distinct receptive fields. DGSD uses edge and distribution information to guide semantic decoding, thereby improving the segmentation accuracy at class boundaries. To enhance the performance of the compact student model, our KD strategy includes detail, semantic response distillation (DSRD), and contrastive learning-based feature distillation (CLFD). Practically, DSRD enables the student model to gain knowledge from the teacher model at both the detailed and semantic levels. At the same time, CLFD increases the similarity of features within the same categories and emphasizes the distinctiveness of features between different categories in both the student and teacher models. Extensive experiments conducted on two standard datasets have consistently demonstrated that both CLNet-T and CLNet-S* outperform other state-of-the-art methods. The code and results are available at https://github.com/xiaodonguo/CLNet.

Normalization of data for training and analysis by the MaskRCNN model using the k-means method for a smart refrigerator’s computer vision

Data normalization for training and analysis of the MaskRCNN model using the k-means method for computer vision of smart refrigerator

Comparison of the YOLOv3 and Mask R-CNN architectures’ efficiency in the smart refrigerator’s computer vision

Computer Image Content Retrieval Considering K-Means Clustering Algorithm

Smart eco-friendly refrigerator based on implementation of architectures of convolutional neural networks

An RGB-D Fusion Based Semantic Segmentation Algorithm Based on Neighborhood Metric Relations

Application of Image Segmentation Algorithm Based on Entropy Clustering in Apple Harvesting Robot

Convolutional Neural Networks for Semantic Segmentation as a Tool for Multiclass Face Analysis in Thermal Infrared

Research on Food Recognition of Smart Refrigerator Based on SSD Target Detection Algorithm

Normalization in Training Deep Convolutional Neural Networks for 2D Bio-medical Semantic Segmentation.

Contrastive learning-based knowledge distillation for RGB-thermal urban scene semantic segmentation

Deep Learning based Food Instance Segmentation using Synthetic Data

Knowledge distillation to effectively attain both region-of-interest and global semantics from an image where multiple objects appear

Refined Image Segmentation for Calorie Estimation of Multiple-dish food items

Hybrid Deep Learning Algorithm-Based Food Recognition and Calorie Estimation

A semantic segmentation algorithm for fashion images based on modified mask RCNN

Batch Normalization Free Rigorous Feature Flow Neural Network for Grocery Product Recognition

Mask-R-FCN: A Deep Fusion Network for Semantic Segmentation.

A novel level set method for remote sensing image based on K-means algorithm

Food Image Segmentation Using Multi-Modal Imaging Sensors with Color and Thermal Data

Fully Automated Skull Stripping from Brain Magnetic Resonance Images Using Mask RCNN-Based Deep Learning Neural Networks