Joint Principal Component Analysis and Total Variation for Infrared and Visible Image Fusion

Xuefeng Zhang,Xiaobing Dai,Xuemin Zhang,Guang Jin

DOI: https://doi.org/10.1016/j.infrared.2022.104523

IF: 2.997

2023-01-01

Infrared Physics & Technology

Abstract:Multi-sensor data fusion has become a discipline which demands more general solutions to a number of appli- cation cases. The image fusion process is defined as gathering all the important information from multiple im- ages, and usually inclusion into a single one. This single image is more informative and accurate than any single source image, and it consists of all the necessary information. The purpose of image fusion is not only to reduce the amount of data but also to construct images that are more appropriate and understandable for human and machine perception. Image fusion is widely applied in subsequent high-order visual tasks, such as object detection and tracking. The image fusion models generally extract multi-source image features and then design specific fusion rules to complete the fusion task. How to ingeniously extract multi-source sensor image features and design reasonable fusion rules is very challenging in the field of image fusion. Considering the difference in information distribution from the source image pairs and the problem that the weight determination of the fi- delity term in the total variation models is manual and experimental, this paper combined the principal component analysis and the total variation optimal at the pixel level image fusion. The principal component transformation was utilized to reallocate the information of the source images. The L1-norm total variation optimal model, as a fusion strategy, aimed to constrain the difference between the fused images and the principal component image pairs. The optimization problem was solved by the generalized Iterative Reweighted Norm. The weight factor lambda in the total variation model was adjusted to adapt the image fusion in different scenarios. The fused result conducted on the TNO datasets indicated that the weight factor lambda has a good fault tolerance to the fusion in different scenes. The optimization convergence for the fusion could be rapidly reached so that it saved calculation time. Our fusion method has a good performance in subjective visual effects and objective evaluation.