Abstract:Fusion of images acquired using different sensors generates a single output with enhanced information for high-level visual perception applications. The transformer architecture has demonstrated its powerful ability to obtain important global contextual dependencies for multi-modal image fusion tasks. However, transformer-based image fusion methods face many critical issues, such as incurring huge computational burdens, limited ability to learn local features, and the difficulty of handling images of arbitrary sizes. To address the above limits, we proposed a novel Laplacian Pyramid Hybrid (LapH) network to combine the advantages of CNN and transformer architectures for multi-modal image fusion tasks. With the divide-and-conquer philosophy, we first build a light-weight CNN-based branch, performing effective extraction and fusion of texture/edge features via central difference convolutions, to process the high-resolution components with abundant details encoded in the lower pyramid levels of the Laplacian pyramid. Then, we design a transformer-based branch to process the low-resolution base components, learning long-range dependencies of global-contextual features without incurring extensive computational loads. Here, we design a multi-scale recurrent modulation mechanism to integrate the edge/texture features from the CNN branch as guidance to progressively refine the feature extraction and fusion on low-frequency components. Finally, we propose a new multi-scale spatial consistency loss term based on the neighbor contrast in source images, generating fused images with more natural and realistic appearances. Extensive experiments on two different multi-modal image fusion tasks verify the superiority of our method. The source codes are made publicly available at https://github.com/rgttadv/LapH .

An image recognition method based on multiple BP neural networks fusion

Research of vehicle type recognition based on multi-BP networks with D-S fusion

Dual-Focal Camera Hdr Imaging Based On Convolutional Neural Network

Application of Improved BP Neural Network in Object Recognition

WSN Data Fusion Scheme Based on Improved BP Neural Network

An Object Identification Method Based on Information Fusion Model of PNN-DS

Multi-Modal Image Fusion Via Deep Laplacian Pyramid Hybrid Network

Multiple networks fusion approach for 3-D object recognition from 2-D view

Research on Image Compression Technology Based on Bp Neural Network.

Author Recognition of Fine-Art Paintings

An Intelligent Fusion Method of Sequential Images Based on Improved DSmT for Target Recognition

Application of particle swarm optimization-based BP neural network to multi-attribute fusion techniques

Sub-pixel Mapping Method Based on BP Neural Network

Multi-target recognition with improved BP algorithm

DeepBP: A bilinear model integrating multi-order statistics for fine-grained recognition

A new pattern recognition method based on the fusion of multiple feature information

Application Research of Image Feature Recognition Algorithm in Visual Image Recognition

Facial expression recognition method based on deep convolutional neural network combined with improved LBP features

Multi-Sensor Data Fusion Algorithm Based on BP Neural Network

Traffic sign recognition method based on multi-feature combination and BP neural network

Hyperspectral and Multispectral Image Fusion Using Cluster-based Multi-branch BP Neural Networks