Abstract:Due to the limitations and costs of thermal sensors, unmanned aerial vehicle (UAV) platforms often equip with high-resolution (HR) visible imaging and low-resolution (LR) thermal imaging cameras for all-day monitoring capability. Existing works generate the HR thermal UAV images by either superresolution (SR) from HR visible and LR thermal images or modality conversion (MC) from HR visible images. However, the modality gap between visible and thermal sources might degrade the generation quality. We observe that the MC task is beneficial in addressing the cross-modal gap in the SR task, while the SR task can provide the condition of thermal information to boost the MC task. Moreover, these two tasks have the same output and can thus be carried out simultaneously without any additional annotation. Based on this observation, we propose a collaborative enhancement network (CENet), which performs thermal UAV image SR and visible image MC in a joint manner, for HR thermal UAV image generation. In particular, we design a mutual guidance module (MGM) to interact the features from SR and MC tasks in an alternating bidirectional manner. Considering that low-level vision tasks are position-sensitive, to further enhance the feature alignment between the two tasks, we design a bidirectional alignment fusion module (BAFM) to maintain feature consistency of the MC and SR branches. The proposed collaborative framework not only achieves joint and unified training of the two tasks, but also generates two types of complementary HR images. Extensive experiments on public datasets demonstrate that the proposed CENet outperforms current state-of-the-art SR methods in generating HR thermal UAV images, as quantified by peak signal-to-noise ratio (PSNR) and structural similarity index measure (SSIM).

Boosting Image Super-Resolution Via Fusion of Complementary Information Captured by Multi-Modal Sensors

Multispectral Image Super-Resolution Via RGB Image Fusion and Radiometric Calibration

Deep Networks with Detail Enhancement for Infrared Image Super-Resolution

Lightweight Multi-Attention Fusion Network for Image Super-Resolution

Improved Thermal Infrared Image Super-Resolution Reconstruction Method Base on Multimodal Sensor Fusion

Fusion of Infrared and Visible Images Via Multi-Layer Convolutional Sparse Representation

Multimodal Sensor Fusion in Single Thermal Image Super-Resolution

Depth and Thermal Sensor Fusion to Enhance 3D Thermographic Reconstruction

Cascaded Deep Networks with Multiple Receptive Fields for Infrared Image Super-Resolution

Infrared Thermal Imaging Super-Resolution via Multiscale Spatio-Temporal Feature Fusion Network

Remote Sensing Hyperspectral Image Super-Resolution via Multidomain Spatial Information and Multiscale Spectral Information Fusion

A Super-Resolution Algorithm Based on Hybrid Network for Multi-Channel Remote Sensing Images

Semantic-Aware Fusion Network Based on Super-Resolution

Efficient Adaptive Feature Fusion Network for Remote-Sensing Image Super-Resolution

Image Super-Resolution Reconstruction Model Based on Multi-Feature Fusion

Adaptive Multi-modal Fusion of Spatially Variant Kernel Refinement with Diffusion Model for Blind Image Super-Resolution

When Fusion Meets Super-Resolution: Implicit Edge Calibration for Higher Resolution Multispectral Image Reconstruction

Modality Conversion Meets Superresolution: A Collaborative Framework for High- Resolution Thermal UAV Image Generation

Multi-image Hybrid Super-Resolution Reconstruction via Interpolation and Multi-Scale Residual Networks

Universal high spatial resolution hyperspectral imaging using hybrid-resolution image fusion

High-to-low-level Feature Matching and Complementary Information Fusion for Reference-Based Image Super-Resolution