Adversarial Identity Injection for Semantic Face Image Synthesis

Giuseppe Tarollo,Tomaso Fontanini,Claudio Ferrari,Guido Borghi,Andrea Prati

2024-04-16

Abstract:Nowadays, deep learning models have reached incredible performance in the task of image generation. Plenty of literature works address the task of face generation and editing, with human and automatic systems that struggle to distinguish what's real from generated. Whereas most systems reached excellent visual generation quality, they still face difficulties in preserving the identity of the starting input subject. Among all the explored techniques, Semantic Image Synthesis (SIS) methods, whose goal is to generate an image conditioned on a semantic segmentation mask, are the most promising, even though preserving the perceived identity of the input subject is not their main concern. Therefore, in this paper, we investigate the problem of identity preservation in face image generation and present an SIS architecture that exploits a cross-attention mechanism to merge identity, style, and semantic features to generate faces whose identities are as similar as possible to the input ones. Experimental results reveal that the proposed method is not only suitable for preserving the identity but is also effective in the face recognition adversarial attack, i.e. hiding a second identity in the generated faces.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The paper primarily aims to address the issue of preserving the identity features of the input subject when generating facial images. Although most existing systems excel in visual generation quality, they still face difficulties in retaining the identity features of the initial input subject. Specifically, while Semantic Image Synthesis (SIS) methods can generate images based on semantic segmentation masks, their main focus is not on preserving the perceived identity of the input subject. To tackle this challenge, the paper proposes an SIS architecture based on a cross-attention mechanism that integrates identity, style, and semantic features to generate facial images that are as similar as possible to the input identity. Experimental results show that this method not only helps retain identity features but also performs well in facial recognition adversarial attacks, meaning it can hide a second identity in the generated facial images without being easily detected. In summary, the core issue of the paper is how to effectively preserve and inject the identity information of the input image during the facial image generation process, and it explores the potential application of this technology in adversarial attacks.

Adversarial Identity Injection for Semantic Face Image Synthesis

SIMGAN: Photo-Realistic Semantic Image Manipulation Using Generative Adversarial Networks.

Controllable Face Synthesis with Semantic Latent Diffusion Models

Semi-supervised Adversarial Learning to Generate Photorealistic Face Images of New Identities from 3D Morphable Model

StyleIPSB: Identity-Preserving Semantic Basis of StyleGAN for High Fidelity Face Swapping

Semantic Image Synthesis via Class-Adaptive Cross-Attention

Imperceptible Face Forgery Attack via Adversarial Semantic Mask

Towards Open-Set Identity Preserving Face Synthesis

Deep Face Swapping via Cross-Identity Adversarial Training.

Identity-driven Three-Player Generative Adversarial Network for Synthetic-based Face Recognition

Generate Identity-Preserving Faces by Generative Adversarial Networks.

Detection of AI-Generated Synthetic Faces

Identity-Preserving Face Swapping via Dual Surrogate Generative Models

Toward Identity Preserving Face Synthesis Between Sketches and Photos Using Deep Feature Injection

Adv-Diffusion: Imperceptible Adversarial Face Identity Attack via Latent Diffusion Model

Towards Privacy Protection by Generating Adversarial Identity Masks

Finding AI-Generated Faces in the Wild

Semantic Adversarial Attacks on Face Recognition through Significant Attributes

Exploiting Semantics in Adversarial Training for Image-Level Domain Adaptation

Synthetic And Natural Face Identity Processing Share Common Mechanisms

MFIM: Megapixel Facial Identity Manipulation