Abstract:When traditional super-resolution reconstruction methods are applied to infrared thermal images, they often ignore the problem of poor image quality caused by the imaging mechanism, which makes it difficult to obtain high-quality reconstruction results even with the training of simulated degraded inverse processes. To address these issues, we proposed a thermal infrared image super-resolution reconstruction method based on multimodal sensor fusion, aiming to enhance the resolution of thermal infrared images and rely on multimodal sensor information to reconstruct high-frequency details in the images, thereby overcoming the limitations of imaging mechanisms. First, we designed a novel super-resolution reconstruction network, which consisted of primary feature encoding, super-resolution reconstruction, and high-frequency detail fusion subnetwork, to enhance the resolution of thermal infrared images and rely on multimodal sensor information to reconstruct high-frequency details in the images, thereby overcoming limitations of imaging mechanisms. We designed hierarchical dilated distillation modules and a cross-attention transformation module to extract and transmit image features, enhancing the network's ability to express complex patterns. Then, we proposed a hybrid loss function to guide the network in extracting salient features from thermal infrared images and reference images while maintaining accurate thermal information. Finally, we proposed a learning strategy to ensure the high-quality super-resolution reconstruction performance of the network, even in the absence of reference images. Extensive experimental results show that the proposed method exhibits superior reconstruction image quality compared to other contrastive methods, demonstrating its effectiveness.

Modeling thermal infrared image degradation and real-world super-resolution under background thermal noise and streak interference

Infrared Image Super Resolution Using Gan With Infrared Image Prior

Deep Networks with Detail Enhancement for Infrared Image Super-Resolution

Super-resolution Reconstruction Algorithms Based on Fusion of Deep Learning Mechanism and Wavelet

Ground-to-air Aircraft Infrared Image Deblurring Based on Imaging Degradation Simulation

Cascaded Deep Networks with Multiple Receptive Fields for Infrared Image Super-Resolution

Improved Thermal Infrared Image Super-Resolution Reconstruction Method Base on Multimodal Sensor Fusion

Frequency-Aware Degradation Modeling for Real-World Thermal Image Super-Resolution

Infrared image super-resolution reconstruction by using generative adversarial network with an attention mechanism

Boosting Image Super-Resolution Via Fusion of Complementary Information Captured by Multi-Modal Sensors

Exploiting Raw Images for Real-Scene Super-Resolution

Super-resolution reconstruction of thermal imaging of power equipment based on Generative Adversarial Network with Channel Filtering

Superresolution reconstruction of optical remote sensing images based on a multiscale attention adversarial network

Super-Resolution Reconstruction Method of Remote Sensing Image Based on Multi-Feature Fusion

SMC-SRGAN-Lightning super-resolution algorithm based on optical micro-scanning thermal microscope image

Research on Image Denoising and Super-Resolution Reconstruction Technology of Multiscale-Fusion Images

Generation diffusion degradation: Simple and efficient design for blind super-resolution

Learning to Super-Resolve Blurry Face and Text Images.

Motion Defocus Infrared Image Restoration Based on Multi Scale Generative Adversarial Network

Infrared Thermal Imaging Super-Resolution via Multiscale Spatio-Temporal Feature Fusion Network

Neural Super-Resolution for Real-time Rendering with Radiance Demodulation