Abstract:Face recognition recently has achieved remarkable success in many fields, especially in mobile payment, authentication, criminal investigation, and city management. However, face occlusion is still the key problem in person identification, such as in the field of anti-terrorism, criminal cases and public security. To solve this problem, an improved end-to-end deep generative adversarial network (named EyesGAN) has been proposed to synthesize human face from human eyes in this paper, which can be used as a potential scheme for masked face recognition. BicycleGAN is chosen as the baseline and effective improvements have been achieved. First, the self-attentional mechanism is introduced so that the improved model can more effectively learn about the internal mapping between human eyes and face. Second, the perceptual loss is applied to guide the model cyclic training and help with updating the network parameters so that the synthesized face can be of higher-similarity to the ground truth face. Third, EyesGAN has been designed by getting the utmost out of the performance of the perceptual loss and the self-attentional mechanism in GANs. A dataset of eyes-to-face synthesis has been reconstructed based on the public face datasets for training and testing. Finally, the faces synthesized by EyesGAN have been quantitatively and qualitatively compared with the results of existing methods. Extensive experiments demenstrate that our proposed method has performed better than the state-of-the-art methods in terms of Average Euclidean Distance, Average Cosine Similarity, Synthesis Accuracy Percentage, Fr<math>é</math>chet Inception Distance, and Baidu face recognition rate (the accuracy achieved <math>96.1%</math> on 615 test data of CelebA database). In this paper, the feasibility of synthesizing human face from human eyes has been explored, and the attention map shows that our network can predict other parts of the face from eyes.

Content-Consistent Generation of Realistic Eyes with Style

StyleIPSB: Identity-Preserving Semantic Basis of StyleGAN for High Fidelity Face Swapping

BiOcularGAN: Bimodal Synthesis and Annotation of Ocular Images

Self-driven Dual-path Learning for Reference-based Line Art Colorization under Limited Data

Shape Constrained Network for Eye Segmentation in the Wild

Example-Guided Style Consistent Image Synthesis from Semantic Labeling

Realistic Face Reenactment Via Self-Supervised Disentangling of Identity and Pose

EyesGAN: Synthesize human face from human eyes

Quality Guided Sketch-to-Photo Image Synthesis

E2F-Net: Eyes-to-Face Inpainting via StyleGAN Latent Space

Semantic Image Synthesis via Class-Adaptive Cross-Attention

StyleGAN-Based Advanced Semantic Segment Encoder for Generative AI

Semantic Image Synthesis with Unconditional Generator

Content-Conditioned Generation of Stylized Free hand Sketches

Robust deep learning for eye fundus images: Bridging real and synthetic data for enhancing generalization

Pose with Style: Detail-Preserving Pose-Guided Image Synthesis with Conditional StyleGAN

Face Sketch Synthesis via Semantic-Driven Generative Adversarial Network

Dual Attention GANs for Semantic Image Synthesis

GuidedStyle: Attribute knowledge guided style manipulation for semantic face editing

Sketch-to-Art: Synthesizing Stylized Art Images From Sketches