Abstract:Aiming at the problems of insufficient texture details, low contrast and loss of target information in the infrared and visible fusion image, an image fusion algorithm based on feature optimization and a generative adversarial network is proposed. Firstly, considering the impact of original image quality on the fusion results, an adaptive feature optimization module, which is guided by an objective function and based on the chameleon swarm algorithm, is designed, which can enhance the texture details of visible image and the contrast of infrared image. Then, in order to preserve more multi-modal information in the fused image, the generative adversarial network is introduced into the fusion framework. In the generator model, considering the difference in imaging mechanisms between infrared and visible images, a dual branch feature extraction network is constructed. At the same time, in order to solve the problem of insufficient feature extraction caused by single scale convolutional layer, a multi-scale dense connection module is designed to increase the range of network receptive field feature extraction, so as to comprehensively extract the deep semantic features and shallow texture features of images. To reduce the loss of important target feature information during the fusion process, a parallel attention model based on space and channel is designed in the feature fusion layer. By feeding the infrared and visible image feature information into the spatial and channel attention models, the correlation and dependency between different modal features are captured, and the expression ability of the network is improved, so as to better focus the thermal salient targets of infrared images and the texture details of visible images. In the discriminator model, to preserve the feature information of both infrared and visible images in the fusion results, a dual discriminator network structure for infrared and visible images is constructed, which can enable the fused image to retain as much rich information as possible from the original image in adversarial learning. Moreover, in the network training stage, in order to train a good model and enhance its robustness, 32 sets of images selected from the TNO image fusion data set are cropped and expanded to obtain 24 200 sets of infrared and visible image pairs. Finally, to verify the advantages of the proposed algorithm, subjective and objective results are compared with six classic fusion algorithms, such as DenseFuse, FusionGAN, ResNet-ZCA, MDLatLRR, PMGI and RFN-Nest. The experimental results show that the proposed algorithm has significant advantages in both subjective and objective evaluations. The fused images have richer texture details, clearer edges and targets, and better contrast. In the objective evaluation indexes, the entropy, spatial frequency, joint entropy, visual information fidelity, and gradient-based fusion performance obtain the optimal values, which are improved by 16.11%, 65.46%, 7.96%, 42.67%, and 33.24%, respectively, compared with the DenseFuse algorithm. In addition, to verify the effectiveness of the feature optimization module, multi-scale dense connection module and attention fusion module, ablation experiments are conducted on 21 sets of images and six evaluation indexes, all of which use the same parameter settings. The results of the ablation experiment indicate that compared with the original fusion network(i. e. none of the three modules is added), the six objective evaluation indexes are increased after the introduction of the three designed modules, which verifies the effectiveness of each module of the proposed algorithm.

Privacy protection generalization with adversarial fusion

Transferable Adversarial Facial Images for Privacy Protection

Privacy preservation for image data: A GAN‐based method

Protecting Facial Privacy: Generating Adversarial Identity Masks via Style-robust Makeup Transfer

Transferable face image privacy protection based on federated learning and ensemble models

Generative Adversarial Network Based Image Privacy Protection Algorithm

BeautyGuard: Style Transfer-Based Adversarial Beautification for Facial Privacy Protection

Two Birds with One Stone: Iteratively Learn Facial Attributes with GANs.

Pre-Trained Feature Fusion and Multidomain Identification Generative Adversarial Network for Face Frontalization.

Towards a Controllable and Reversible Privacy Protection System for Facial Images through Enhanced Multi-Factor Modifier Networks

Classifiers Protected Against Attacks by Fusion of Multi-Branch Perturbed GAN

General Adversarial Perturbation Simulating: Protect Unknown System by Detecting Unknown Adversarial Faces.

GAN-Based Fusion Adversarial Training

Generating De-identification facial images based on the attention models and adversarial examples

Infrared and Visible Image Fusion Algorithm Based on Feature Optimization and GAN

Anti-Fake Vaccine: Safeguarding Privacy Against Face Swapping Via Visual-Semantic Dual Degradation

Privacy-preserving Adversarial Facial Features

Inverting Gradient Attack Combined with GAN Network in Federated Learning of Face Recognition

A Novel Infrared and Visible Image Fusion Approach Based on Adversarial Neural Network

Detecting Adversarial Faces Using Only Real Face Self-Perturbations

3D-Aware Adversarial Makeup Generation for Facial Privacy Protection