Reviewing FID and SID Metrics on Generative Adversarial Networks

Ricardo de Deijn,Aishwarya Batra,Brandon Koch,Naseef Mansoor,Hema Makkena

DOI: https://doi.org/10.5121/csit.2024.140208

2024-02-06

Abstract:The growth of generative adversarial network (GAN) models has increased the ability of image processing and provides numerous industries with the technology to produce realistic image transformations. However, with the field being recently established there are new evaluation metrics that can further this research. Previous research has shown the Fréchet Inception Distance (FID) to be an effective metric when testing these image-to-image GANs in real-world applications. Signed Inception Distance (SID), a founded metric in 2023, expands on FID by allowing unsigned distances. This paper uses public datasets that consist of façades, cityscapes, and maps within Pix2Pix and CycleGAN models. After training these models are evaluated on both inception distance metrics which measure the generating performance of the trained models. Our findings indicate that usage of the metric SID incorporates an efficient and effective metric to complement, or even exceed the ability shown using the FID for the image-to-image GANs

Computer Vision and Pattern Recognition,Image and Video Processing

What problem does this paper attempt to address?

### Problems Addressed by the Paper

This paper primarily explores the application of Generative Adversarial Networks (GANs) in image processing and evaluates two new evaluation metrics—Fréchet Inception Distance (FID) and Signed Inception Distance (SID)—to measure the performance of GANs in image-to-image translation tasks.

#### Main Issues:
1. **Effectiveness of Evaluation Metrics**: Investigating how FID and SID can better assess the performance of GAN models in image-to-image translation tasks.
2. **Diversity of GAN Models**: Addressing issues such as mode collapse, instability, and lack of diversity in current GAN models.
3. **Performance on Different Datasets**: Evaluating the performance of Pix2Pix and CycleGAN models on various public datasets (e.g., building facades, cityscapes, and maps) and comparing their scores under FID and SID metrics.

#### Specific Objectives:
- Compare the effectiveness of FID and SID metrics on Pix2Pix and CycleGAN models across different datasets.
- Explore the impact of different training epochs on the performance of GAN models.
- Analyze the effect of different dataset sizes on the performance of GAN models.

Through these studies, the paper aims to find more effective evaluation methods to improve the performance of GAN models in image-to-image translation tasks. Specifically, SID, as a new evaluation metric, can better capture the diversity and differences between generated images and real images, thereby providing more accurate assessment results.