Abstract:When traditional super-resolution reconstruction methods are applied to infrared thermal images, they often ignore the problem of poor image quality caused by the imaging mechanism, which makes it difficult to obtain high-quality reconstruction results even with the training of simulated degraded inverse processes. To address these issues, we proposed a thermal infrared image super-resolution reconstruction method based on multimodal sensor fusion, aiming to enhance the resolution of thermal infrared images and rely on multimodal sensor information to reconstruct high-frequency details in the images, thereby overcoming the limitations of imaging mechanisms. First, we designed a novel super-resolution reconstruction network, which consisted of primary feature encoding, super-resolution reconstruction, and high-frequency detail fusion subnetwork, to enhance the resolution of thermal infrared images and rely on multimodal sensor information to reconstruct high-frequency details in the images, thereby overcoming limitations of imaging mechanisms. We designed hierarchical dilated distillation modules and a cross-attention transformation module to extract and transmit image features, enhancing the network's ability to express complex patterns. Then, we proposed a hybrid loss function to guide the network in extracting salient features from thermal infrared images and reference images while maintaining accurate thermal information. Finally, we proposed a learning strategy to ensure the high-quality super-resolution reconstruction performance of the network, even in the absence of reference images. Extensive experimental results show that the proposed method exhibits superior reconstruction image quality compared to other contrastive methods, demonstrating its effectiveness.

Multimodal Sensor Fusion in Single Thermal Image Super-Resolution

Boosting Image Super-Resolution Via Fusion of Complementary Information Captured by Multi-Modal Sensors

Deep Networks with Detail Enhancement for Infrared Image Super-Resolution

Improved Thermal Infrared Image Super-Resolution Reconstruction Method Base on Multimodal Sensor Fusion

Multispectral Image Super-Resolution Via RGB Image Fusion and Radiometric Calibration

Infrared Thermal Imaging Super-Resolution via Multiscale Spatio-Temporal Feature Fusion Network

Depth and Thermal Sensor Fusion to Enhance 3D Thermographic Reconstruction

CoReFusion: Contrastive Regularized Fusion for Guided Thermal Super-Resolution

Fusion of Infrared and Visible Images Via Multi-Layer Convolutional Sparse Representation

Cascaded Deep Networks with Multiple Receptive Fields for Infrared Image Super-Resolution

Thermal Image Super-Resolution Using Second-Order Channel Attention with Varying Receptive Fields

Multimodal Registration and Fusion for 3D Thermal Imaging

Assessing thermal imagery integration into object detection methods on air-based collection platforms

Semantic-Aware Fusion Network Based on Super-Resolution

Multispectral image fusion based on super pixel segmentation

SMFD: an end-to-end infrared and visible image fusion model based on shared-individual multi-scale feature decomposition

A Novel Domain Transfer-Based Approach for Unsupervised Thermal Image Super-Resolution

MFST: Multi-Modal Feature Self-Adaptive Transformer for Infrared and Visible Image Fusion

SwinFuSR: an image fusion-inspired model for RGB-guided thermal image super-resolution

A robust infrared and visible image fusion framework via multi-receptive-field attention and color visual perception

Self-supervised spectral super-resolution for a fast hyperspectral and multispectral image fusion