Abstract:Fairness in image restoration tasks is the desire to treat different sub-groups of images equally well. Existing definitions of fairness in image restoration are highly restrictive. They consider a reconstruction to be a correct outcome for a group (e.g., women) only if it falls within the group's set of ground truth images (e.g., natural images of women); otherwise, it is considered entirely incorrect. Consequently, such definitions are prone to controversy, as errors in image restoration can manifest in various ways. In this work we offer an alternative approach towards fairness in image restoration, by considering the Group Perceptual Index (GPI), which we define as the statistical distance between the distribution of the group's ground truth images and the distribution of their reconstructions. We assess the fairness of an algorithm by comparing the GPI of different groups, and say that it achieves perfect Perceptual Fairness (PF) if the GPIs of all groups are identical. We motivate and theoretically study our new notion of fairness, draw its connection to previous ones, and demonstrate its utility on state-of-the-art face image restoration algorithms.

What problem does this paper attempt to address?

### What problem does this paper attempt to solve? This paper aims to solve the fairness problem in image inpainting tasks. Specifically, the existing definitions of fairness in image inpainting are too strict and controversial because they only consider whether the reconstructed image completely belongs to the real - image set of a specific group (for example, natural images of women). If the reconstructed image does not belong to the real - image set of this group, it is regarded as completely wrong. This definition ignores the fact that image inpainting errors can be manifested in multiple ways. To solve these problems, the author proposes a new concept of fairness - **Perceptual Fairness (PF)**. PF measures the fairness of an algorithm by evaluating the distance between distributions among different groups. Specifically, the author introduces the **Group Perceptual Index (GPI)**, which is defined as the statistical distance (such as Wasserstein distance) between the real - image distribution of a certain group and the reconstructed - image distribution. PF believes that when the GPI of all groups is the same, the algorithm achieves perfect perceptual fairness. ### Specific problems and solutions 1. **Limitations of the existing fairness definitions**: - The existing fairness definitions are too strict, regarding any reconstruction that does not completely conform to the real - image set as completely wrong. - These definitions ignore the distribution characteristics of images within a group, which may lead to some groups being treated unfairly. 2. **Proposal of perceptual fairness**: - The author proposes perceptual fairness (PF) and quantifies the fairness of different groups through the group perceptual index (GPI). - GPI is defined as the statistical distance between the real - image distribution of a certain group and the reconstructed - image distribution. - When the GPI of all groups is equivalent, it is considered that the algorithm has achieved perceptual fairness. 3. **Theoretical analysis and experimental verification**: - The author proves through theoretical analysis that in the case of severe degradation, it is impossible to achieve perfect GPI for all groups simultaneously. - The experimental results show that PF can detect biases that RDP cannot find, especially when dealing with facial image super - resolution tasks. ### Formula representation - **Perceptual Index (PI)**: \[ PI_d := d(p_X, p_{\hat{X}}) \] where \(d(\cdot, \cdot)\) is a distance measure between certain distributions (such as Kullback - Leibler divergence, total variation distance, Wasserstein distance, etc.). - **Group Perceptual Index (GPI)**: \[ GPI_d(a) := d(p_X|A(\cdot|a), p_{\hat{X}}|A(\cdot|a)) \] where \(d(\cdot, \cdot)\) is also a distance measure between certain distributions. ### Conclusion By introducing perceptual fairness (PF) and group perceptual index (GPI), the author provides a more reasonable and comprehensive fairness evaluation method for image inpainting. This method not only solves the limitations of the existing definitions but also can detect and mitigate biases more effectively in practical applications.

Perceptual Fairness in Image Restoration

Fairness Testing of Deep Image Classification with Adequacy Metrics

Improved Restoration Algorithm for Weakly Blurred and Strongly Noisy Image

Fairness for Image Generation with Uncertain Sensitive Attributes

Image restoration for real-world under-display imaging

PIPAL: A Large-Scale Image Quality Assessment Dataset for Perceptual Image Restoration

From Posterior Sampling to Meaningful Diversity in Image Restoration

The Perception-Robustness Tradeoff in Deterministic Image Restoration

Perceive-IR: Learning to Perceive Degradation Better for All-in-One Image Restoration

Posterior-Mean Rectified Flow: Towards Minimum MSE Photo-Realistic Image Restoration

Overcoming False Illusions in Real-World Face Restoration with Multi-Modal Guided Diffusion Model

Fairly Private: Investigating The Fairness of Visual Privacy Preservation Algorithms

Improving Fairness using Vision-Language Driven Image Augmentation

Harvard Glaucoma Fairness: A Retinal Nerve Disease Dataset for Fairness Learning and Fair Identity Normalization

Evaluating Proposed Fairness Models for Face Recognition Algorithms

Looks Too Good To Be True: An Information-Theoretic Analysis of Hallucinations in Generative Restoration Models

Perceptual Image Restoration with High-Quality Priori and Degradation Learning

Deep Optimal Transport: A Practical Algorithm for Photo-realistic Image Restoration

FairDiff: Fair Segmentation with Point-Image Diffusion

Dformer: Learning Efficient Image Restoration with Perceptual Guidance

Distribution-aware Fairness Test Generation