AE-Netv2: Optimization of Image Fusion Efficiency and Network Architecture

Aiqing Fang,Xinbo Zhao,Jiaqi Yang,Beibei Qin,Yanning Zhang

DOI: https://doi.org/10.48550/arXiv.2010.01863

2020-10-06

Abstract:Existing image fusion methods pay few research attention to image fusion efficiency and network architecture. However, the efficiency and accuracy of image fusion has an important impact in practical applications. To solve this problem, we propose an \textit{efficient autonomous evolution image fusion method, dubed by AE-Netv2}. Different from other image fusion methods based on deep learning, AE-Netv2 is inspired by human brain cognitive mechanism. Firstly, we discuss the influence of different network architecture on image fusion quality and fusion efficiency, which provides a reference for the design of image fusion architecture. Secondly, we explore the influence of pooling layer on image fusion task and propose an image fusion method with pooling layer. Finally, we explore the commonness and characteristics of different image fusion tasks, which provides a research basis for further research on the continuous learning characteristics of human brain in the field of image fusion. Comprehensive experiments demonstrate the superiority of AE-Netv2 compared with state-of-the-art methods in different fusion tasks at a real time speed of 100+ FPS on GTX 2070. Among all tested methods based on deep learning, AE-Netv2 has the faster speed, the smaller model size and the better robustness.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The main problems that this paper attempts to solve are the deficiencies in the efficiency and network architecture design of existing image fusion methods. Specifically:

1. **Image Fusion Efficiency**: Existing image fusion methods rarely pay attention to the efficiency of image fusion, which is an important limiting factor in practical applications. The author points out that although some deep - learning - based methods have achieved remarkable results in the quality of image fusion, these methods are often accompanied by high computational costs and large model sizes, resulting in poor performance in real - time applications.

2. **Network Architecture Design**: The paper also explores the influence of different network architectures on the quality and efficiency of image fusion. The author believes that existing image fusion methods lack consideration of efficiency when designing network architectures. They usually adopt complex network structures to improve the fusion quality, but this leads to low efficiency.

3. **Continuous Learning Ability**: The paper further explores how to improve image fusion methods by introducing the continuous learning mechanism of the human brain. The author points out that existing image fusion methods lack research on the commonalities and characteristics between different tasks, which limits the generalization ability of the model and its ability to adapt to new tasks.

To meet the above challenges, the author proposes an efficient autonomous - evolving image fusion method - AE - Netv2. This method not only improves the efficiency of image fusion but also ensures the fusion quality and has good generalization ability and continuous learning ability. Specifically, the main contributions of AE - Netv2 include:

- Exploring and evaluating the influence of different network architectures on the quality and efficiency of image fusion, and verifying the feasibility of simple network architectures and small - sample data in image fusion tasks.
- Analyzing the problems of multi - scale information extraction in existing deep - learning - based image fusion methods, exploring the influence of the pooling layer on image fusion tasks, and proposing an efficient image fusion method with a pooling layer.
- Studying the continuous learning mechanism of the human brain, exploring the commonalities and characteristics of different image fusion tasks, and providing references for the further development of the image fusion field.

In conclusion, this paper aims to propose an efficient, robust and general - purpose image fusion method by combining the cognitive mechanisms of the human brain to solve the deficiencies of existing methods in efficiency and network architecture design.