Abstract:When traditional super-resolution reconstruction methods are applied to infrared thermal images, they often ignore the problem of poor image quality caused by the imaging mechanism, which makes it difficult to obtain high-quality reconstruction results even with the training of simulated degraded inverse processes. To address these issues, we proposed a thermal infrared image super-resolution reconstruction method based on multimodal sensor fusion, aiming to enhance the resolution of thermal infrared images and rely on multimodal sensor information to reconstruct high-frequency details in the images, thereby overcoming the limitations of imaging mechanisms. First, we designed a novel super-resolution reconstruction network, which consisted of primary feature encoding, super-resolution reconstruction, and high-frequency detail fusion subnetwork, to enhance the resolution of thermal infrared images and rely on multimodal sensor information to reconstruct high-frequency details in the images, thereby overcoming limitations of imaging mechanisms. We designed hierarchical dilated distillation modules and a cross-attention transformation module to extract and transmit image features, enhancing the network's ability to express complex patterns. Then, we proposed a hybrid loss function to guide the network in extracting salient features from thermal infrared images and reference images while maintaining accurate thermal information. Finally, we proposed a learning strategy to ensure the high-quality super-resolution reconstruction performance of the network, even in the absence of reference images. Extensive experimental results show that the proposed method exhibits superior reconstruction image quality compared to other contrastive methods, demonstrating its effectiveness.

Super-resolution image restoration based on maximum-likelihood estimation

Method of Super-Resolution Based on Array Detection and Maximum-Likelihood Estimation

Imaging Resolution and Properties Analysis of Super Resolution Microscopy with Parallel Detection under Different Noise, Detector and Image Restoration Conditions

Parallel Detecting Super-Resolution Microscopy Using Correlation Based Image Restoration

Image Super-resolution Based on Tiny Recurrent Convolutional Neural Network

Super-resolution Reconstruction Algorithms Based on Fusion of Deep Learning Mechanism and Wavelet

Restoration method of TDI remote sensing image based on optimization of numerical fidelity term

Practical super-resolution from dynamic video sequences

Image Restoration Via Improved Wiener Filter Applied to Optical Sparse Aperture Systems

Interpolation-based Image Super-Resolution Using Multisurface Fitting.

Self-adaptive Blind Super-Resolution Image Reconstruction

A New Image Super-Resolution Method in the Wavelet Domain

Image super-resolution reconstruction based on instance spatial feature modulation and feedback mechanism

Super Resolution with Edge-Constrained Motion Estimation

Image Super-Resolution Based on Structure-Modulated Sparse Representation.

A Fast Super-Resolution Method Based on Sparsity Properties.

A new method of super-resolution reconstruction based on wavelet and subpixel interpolation

Improved Thermal Infrared Image Super-Resolution Reconstruction Method Base on Multimodal Sensor Fusion

Super-Resolution Image Reconstruction of Wavefront Coding Imaging System Based on Deep Learning Network

Generalized Expectation Maximization Framework for Blind Image Super Resolution.

Regularized super-resolution restoration algorithm for single medical image based on fuzzy similarity fusion