TRANSFER LEARNING WITH PIX2PIX GAN FOR GENERATING REALISTIC PHOTOGRAPHS FROM VIEWED SKETCH ARTS

M. Shetty K. Raghavendra,Prasad Narasimha Sarappadi
DOI: https://doi.org/10.35741/issn.0258-2724.57.4.17
2022-08-29
Journal of Southwest Jiaotong University
Abstract:Our approach for producing realistic human faces from artist sketches is based on conditional GAN image translation, which we present in this article. Here, face features are centered in front-facing positions in a typical illumination, with an unbiased look and no obstacles. The image-to-image translation technique is used using traits with the conditional generated adversarial network (cGAN) that has been formed to understand the correspondence among a source image input and a target image by employing a training dataset matched set of images. Using the pix2pix convolution neural network to produce images potentially helps overcome challenges such as short datasets and class imbalance. We develop excellent loss functions by constraining pixels, edges, and high-level properties of the generated realistic images in the system design. The CUHK dataset is employed in this research, and image augmentation is accomplished using an effective sampling strategy since the dataset is restricted. A significant variation in the network's performance is observed, caused by fewer amounts of data available. The results of our experiments demonstrate that our approach can produce face sketches/photos that are both aesthetically pleasing and retain the subject's individuality. The deployment of transfer learning for the sketch to image is carried out to get an understanding of the usefulness of this method when the target and source are different. With the usage transfer learning method, the system obtained 10%-15% improved efficiency compared to previous works.
What problem does this paper attempt to address?