Abstract:When traditional super-resolution reconstruction methods are applied to infrared thermal images, they often ignore the problem of poor image quality caused by the imaging mechanism, which makes it difficult to obtain high-quality reconstruction results even with the training of simulated degraded inverse processes. To address these issues, we proposed a thermal infrared image super-resolution reconstruction method based on multimodal sensor fusion, aiming to enhance the resolution of thermal infrared images and rely on multimodal sensor information to reconstruct high-frequency details in the images, thereby overcoming the limitations of imaging mechanisms. First, we designed a novel super-resolution reconstruction network, which consisted of primary feature encoding, super-resolution reconstruction, and high-frequency detail fusion subnetwork, to enhance the resolution of thermal infrared images and rely on multimodal sensor information to reconstruct high-frequency details in the images, thereby overcoming limitations of imaging mechanisms. We designed hierarchical dilated distillation modules and a cross-attention transformation module to extract and transmit image features, enhancing the network's ability to express complex patterns. Then, we proposed a hybrid loss function to guide the network in extracting salient features from thermal infrared images and reference images while maintaining accurate thermal information. Finally, we proposed a learning strategy to ensure the high-quality super-resolution reconstruction performance of the network, even in the absence of reference images. Extensive experimental results show that the proposed method exhibits superior reconstruction image quality compared to other contrastive methods, demonstrating its effectiveness.

Super-Resolution Reconstruction for Cross-Modal Communications in Industrial Internet of Things

Haptic Signal Reconstruction for Cross-Modal Communications

Super-resolution Reconstruction Algorithms Based on Fusion of Deep Learning Mechanism and Wavelet

Deep Networks with Detail Enhancement for Infrared Image Super-Resolution

Perception-Aware Cross-Modal Signal Reconstruction: From Audio-Haptic to Visual

Continuous Digital Zoom with Cross Attention for Dual Camera System

Toward General Cross-Modal Signal Reconstruction for Robotic Teleoperation

Boosting Image Super-Resolution Via Fusion of Complementary Information Captured by Multi-Modal Sensors

Cross-Modal Collaborative Communications

Cascaded Deep Networks with Multiple Receptive Fields for Infrared Image Super-Resolution

Super-Resolution Perception for Industrial Sensor Data

Cross-Medium Communication Combining Infrared Optical and Acoustic Waves

A Terahertz Image Super-Resolution Reconstruction Algorithm Based on the Deep Convolutional Neural Network

Cross-Modal Transmission Strategy

Improved Thermal Infrared Image Super-Resolution Reconstruction Method Base on Multimodal Sensor Fusion

Multi-image Hybrid Super-Resolution Reconstruction via Interpolation and Multi-Scale Residual Networks

Cross-Modal Reconstruction for Tactile Signal in Human-Robot Interaction

Cross-modal Communication Technology: A Survey

Cross-View Human Intention Recognition for Human-Robot Collaboration

Haptic Signal Reconstruction in eHealth Internet of Things

Edge-Based Cross-Modal Communications for Remote Healthcare