Abstract:Near infrared (NIR) imaging has been widely applied in low-light imaging scenarios; however, it is difficult for human and algorithms to perceive the real scene in the colorless NIR domain. While Generative Adversarial Network (GAN) has been widely employed in various image colorization tasks, it is challenging for a direct mapping mechanism, such as a conventional GAN, to transform an image from the NIR to the RGB domain with correct semantic reasoning, well-preserved textures, and vivid color combinations concurrently. In this work, we propose a novel Attention-based NIR image colorization framework via Adaptive Fusion of Semantic and Texture clues, aiming at achieving these goals within the same framework. The tasks of texture transfer and semantic reasoning are carried out in two separate network blocks. Specifically, the Texture Transfer Block (TTB) aims at extracting texture features from the NIR image's Laplacian component and transferring them for subsequent color fusion. The Semantic Reasoning Block (SRB) extracts semantic clues and maps the NIR pixel values to the RGB domain. Finally, a Fusion Attention Block (FAB) is proposed to adaptively fuse the features from the two branches and generate an optimized colorization result. In order to enhance the network's learning capacity in semantic reasoning as well as mapping precision in texture transfer, we have proposed the Residual Coordinate Attention Block (RCAB), which incorporates coordinate attention into a residual learning framework, enabling the network to capture long-range dependencies along the channel direction and meanwhile precise positional information can be preserved along spatial directions. RCAB is also incorporated into FAB to facilitate accurate texture alignment during fusion. Both quantitative and qualitative evaluations show that the proposed method outperforms state-of-the-art NIR image colorization methods.

Contrastive learning with feature fusion for unpaired thermal infrared image colorization

A feature refinement and adaptive generative adversarial network for thermal infrared image colorization

One Color Contrast Enhanced Infrared and Visible Image Fusion Method

Infrared and Visible Image Fusion Based on a Two-Stage Class Conditioned Auto-Encoder Network.

Exploring efficient and effective generative adversarial network for thermal infrared image colorization

Cross-domain colorization of unpaired infrared images through contrastive learning guided by color feature selection attention

A robust infrared and visible image fusion framework via multi-receptive-field attention and color visual perception

EV-Fusion: A Novel Infrared and Low-Light Color Visible Image Fusion Network Integrating Unsupervised Visible Image Enhancement

DBSF-Net: Infrared Image Colorization Based on the Generative Adversarial Model with Dual-Branch Feature Extraction and Spatial-Frequency-Domain Discrimination

Attention-Guided NIR Image Colorization via Adaptive Fusion of Semantic and Texture Clues

CPNet: Continuity Preservation Network for infrared video colorization

Dif-Fusion: Towards High Color Fidelity in Infrared and Visible Image Fusion with Diffusion Models

CLCC: Contrastive Learning for Color Constancy

Interactive residual coordinate attention and contrastive learning for infrared and visible image fusion in triple frequency bands

Dif-Fusion: Toward High Color Fidelity in Infrared and Visible Image Fusion With Diffusion Models

Infrared colorization with cross-modality zero-shot learning

SCFusion: Infrared and Visible Fusion Based on Salient Compensation

Infrared and Visible Image Fusion via Interactive Compensatory Attention Adversarial Learning

Learning Spatially Decoupled Color Representations for Facial Image Colorization

Infrared and visible image fusion based on infrared background suppression

A Cross-scale Iterative Attentional Adversarial Fusion Network for Infrared and Visible Images