TGLFusion: A Temperature-Guided Lightweight Fusion Method for Infrared and Visible Images

Bao Yan,Longjie Zhao,Kehua Miao,Song Wang,Qinghua Li,Delin Luo

DOI: https://doi.org/10.3390/s24061735

IF: 3.9

2024-03-08

Sensors

Abstract:The fusion of infrared and visible images is a well-researched task in computer vision. These fusion methods create fused images replacing the manual observation of single sensor image, often deployed on edge devices for real-time processing. However, there is an issue of information imbalance between infrared and visible images. Existing methods often fail to emphasize temperature and edge texture information, potentially leading to misinterpretations. Moreover, these methods are computationally complex, and challenging for edge device adaptation. This paper proposes a method that calculates the distribution proportion of infrared pixel values, allocating fusion weights to adaptively highlight key information. It introduces a weight allocation mechanism and MobileBlock with a multispectral information complementary module, innovations which strengthened the model's fusion capabilities, made it more lightweight, and ensured information compensation. Training involves a temperature-color-perception loss function, enabling adaptive weight allocation based on image pair information. Experimental results show superiority over mainstream fusion methods, particularly in the electric power equipment scene and publicly available datasets.

engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation

What problem does this paper attempt to address?

### What problem does this paper attempt to solve?

This paper primarily addresses the issue of information imbalance in the fusion process of infrared and visible light images, and proposes a lightweight temperature-guided fusion method (TGLFusion) to improve image fusion effectiveness in power equipment scenarios.

#### Specific Problems:
1. **Information Imbalance**: There is an information imbalance between infrared and visible light images. Existing methods often fail to effectively emphasize temperature information and edge texture information, which may lead to incorrect image interpretation.
2. **Computational Complexity**: Existing methods have high computational complexity, making it difficult to meet the real-time processing needs of edge devices.

#### Main Contributions:
1. **Temperature-Aware Optimized Weight Allocation Module**: By calculating the proportion of high-temperature pixels in the infrared image to allocate fusion weights, a temperature-aware optimized weight allocation module is introduced. This helps to better retain key information during the fusion process.
2. **Lightweight Model Structure**: The model adopts the MobileBlock framework and the Multispectral Information Complementary Module (MICM), making the model more lightweight while ensuring information compensation.
3. **Adaptive Weight Allocation**: Through the temperature color-aware loss function, adaptive weight allocation based on image pairs is achieved, thereby optimizing the fused image.
4. **Experimental Validation**: Objective and subjective experiments compared mainstream fusion models, showing significant advantages in power equipment image fusion evaluation metrics.

These contributions collectively enhance the understanding and effectiveness of image fusion in power equipment scenarios, providing valuable insights and practical models that improve image quality and information extraction.