Abstract:Thermal infrared (TIR) object detection plays a crucial role in diverse around-the-clock applications, such as search-and-rescue operations and wildlife protection. Achieving rapid and robust detection of small objects from an aerial perspective is particularly significant in these scenarios. However, the task is compounded by two interrelated challenges, rendering it even more tricky. For one, small objects only occupy a few pixels and contain limited information. For another, TIR sensors are typically of low resolution (LR) due to inherent challenges associated with the imaging mechanism of the TIR spectrum. In contrast, high-resolution (HR) RGB sensors are readily available due to their cost-effectiveness and widespread application. Recognizing the importance of HR information, especially in the context of small-object detection, we propose a cross-modality HR knowledge distillation framework (CMHRD), which leverages knowledge from the HR-RGB modality and provides a novel strategy for TIR small-object detection. The proposed framework introduces three key components: a super-resolution generative (SRG) distillation loss for cross-modal HR representation learning, a cross-modality affinity (CMA) distillation loss to extract scene-level cross-modality information, and a response distillation loss aimed at mimicking the HR prediction. To facilitate research on small-object detection with HR-RGB and LR-TIR data, we have curated and annotated two datasets, namely NOAA-Seal and VTUAV-det-small. Experimental results on the NOAA-Seal demonstrate that CMHRD yields significant improvements, achieving a remarkable 6.39 mAP50 increase over a strong baseline without introducing additional computational cost during inference. Experiments on the single-category dataset VTUAV-det-small and the multicategory dataset RTDOD also show consistent improvements brought by CMHRD. The project is available at https://github.com/NNNNerd/CMHRD.

Modality Conversion Meets Superresolution: A Collaborative Framework for High- Resolution Thermal UAV Image Generation

Parallel-Connected Residual Channel Attention Network for Remote Sensing Image Super-Resolution

Boosting Image Super-Resolution Via Fusion of Complementary Information Captured by Multi-Modal Sensors

Single Remote Sensing Image Super-Resolution Via a Generative Adversarial Network with Stratified Dense Sampling and Chain Training

General cross-modality registration framework for visible and infrared UAV target image registration

Fusion of Infrared and Visible Images Via Multi-Layer Convolutional Sparse Representation

Learning Cross-Modality High-Resolution Representation for Thermal Small-Object Detection

SCINet: Spatial and Contrast Interactive Super-Resolution Assisted Infrared UAV Target Detection

Improved Thermal Infrared Image Super-Resolution Reconstruction Method Base on Multimodal Sensor Fusion

Connecting Low-Level and High-Level Visions: A Joint Optimization for Hyperspectral Image Super-Resolution and Target Detection

CMEFusion: Cross-Modal Enhancement and Fusion of FIR and Visible Images

A New Super Resolution Framework Based on Multi-Task Learning for Remote Sensing Images

Alignment and Fusion Using Distinct Sensor Data for Multimodal Aerial Scene Classification

A Super-Resolution Algorithm Based on Hybrid Network for Multi-Channel Remote Sensing Images

High-resolution Depth Maps Imaging Via Attention-based Hierarchical Multi-modal Fusion

Multimodal Sensor Fusion in Single Thermal Image Super-Resolution

ConvMambaSR: Leveraging State-Space Models and CNNs in a Dual-Branch Architecture for Remote Sensing Imagery Super-Resolution

Multi-Branch Parallel Networks for Object Detection in High-Resolution UAV Remote Sensing Images

Multispectral Pan-sharpening via Dual-Channel Convolutional Network with Convolutional LSTM Based Hierarchical Spatial-Spectral Feature Fusion

Guidance Disentanglement Network for Optics-Guided Thermal UAV Image Super-Resolution

A Unified Generative Adversarial Network With Convolution and Transformer for Remote Sensing Image Fusion