Multi-Exposure Image Fusion via Deformable Self-Attention

Jun Luo,Wenqi Ren,Xinwei Gao,Xiaochun Cao

DOI: https://doi.org/10.1109/tip.2023.3242824

IF: 10.6

2023-03-04

IEEE Transactions on Image Processing

Abstract:Most multi-exposure image fusion (MEF) methods perform unidirectional alignment within limited and local regions, which ignore the effects of augmented locations and preserve deficient global features. In this work, we propose a multi-scale bidirectional alignment network via deformable self-attention to perform adaptive image fusion. The proposed network exploits differently exposed images and aligns them to the normal exposure in varying degrees. Specifically, we design a novel deformable self-attention module that considers variant long-distance attention and interaction and implements the bidirectional alignment for image fusion. To realize adaptive feature alignment, we employ a learnable weighted summation of different inputs and predict the offsets in the deformable self-attention module, which facilitates that the model generalizes well in various scenes. In addition, the multi-scale feature extraction strategy makes the features across different scales complementary and provides fine details and contextual features. Extensive experiments demonstrate that our proposed algorithm performs favorably against state-of-the-art MEF methods.

computer science, artificial intelligence,engineering, electrical & electronic