Abstract:Infrared and visible image fusion aims to obtain a synthetic image that can simultaneously exhibit salient objects and provide abundant texture details. However, existing deep learning-based methods generally depend on convolutional operations, which indeed have good local feature extraction ability, but the restricted receptive field limits its capability in modeling long-range dependencies. To conquer this dilemma, we propose an infrared and visible image fusion method based on Transformer and cross correlation, named TCCFusion. Specifically, we design a local feature extraction branch (LFEB) to preserve local complementary information, in which a dense-shape network is introduced to reuse the information that may be lost during the convolutional operation. To avoid the limitation of the receptive field and to fully extract the global significant information, a global feature extraction branch (GFEB) is devised that consists of three Transformer blocks for long-range relationship construction. In addition, LFEB and GFEB are arranged in a parallel fashion to maintain local and global useful information in a more effective way. Furthermore, we design a cross correlation loss to train the proposed fusion model in an unsupervised manner, with which the fusion result can obtain adequate thermal radiation information in an infrared image and ample texture details in a visible image. Massive experiments on two mainstream datasets illustrate that our TCCFusion outperforms state-of-the-art algorithms not only on visual quality but also on quantitative assessments. Ablation experiments on the network framework and objective function demonstrate the effectiveness of the proposed method.

Converting Thermal Infrared Face Images Into Normal Gray-Level Images

Hallucinating Faces From Thermal Infrared Images

Adaptive Illumination-Invariant Face Recognition Via Local Nonlinear Multi-Layer Contrast Feature.

Expression Recognition from Visible Images with the Help of Thermal Images

Deep Independent Component Network for Thermal-to-visible Face Recognition

Fusion of visible and thermal images for facial expression recognition

A Bidirectional Conversion Network for Cross-Spectral Face Recognition

Visible-Information-Aided Eyeglasses Removing for Thermal Image Reconstruction

Spontaneous Facial Expression Recognition by Using Feature-Level Fusion of Visible and Thermal Infrared Images

Learning mappings for face synthesis from near infrared to visual light images

Tir/Vis Correlation For Liveness Detection In Face Recognition

Face Hallucination and Recognition Using Kernel Canonical Correlation Analysis

Thermal to Visible Synthesis of Face Images using Multiple Regions

Thermal to Visible Facial Image Translation Using Generative Adversarial Networks.

Common Feature Discriminant Analysis for Matching Infrared Face Images to Optical Face Images

Face emotion recognition based on infrared thermal imagery by applying machine learning and parallelism

Cascaded Generation of High-quality Color Visible Face Images from Thermal Captures

TCCFusion: An Infrared and Visible Image Fusion Method based on Transformer and Cross Correlation

Learning Domain and Pose Invariance for Thermal-to-Visible Face Recognition

Cross-Domain Identification for Thermal-to-Visible Face Recognition

A Synthesis-Based Approach for Thermal-to-Visible Face Verification