Abstract:Fairness of deepfake detectors in the presence of anomalies are not well investigated, especially if those anomalies are more prominent in either male or female subjects. The primary motivation for this work is to evaluate how deepfake detection model behaves under such anomalies. However, due to the black-box nature of deep learning (DL) and artificial intelligence (AI) systems, it is hard to predict the performance of a model when the input data is modified. Crucially, if this defect is not addressed properly, it will adversely affect the fairness of the model and result in discrimination of certain sub-population unintentionally. Therefore, the objective of this work is to adopt metamorphic testing to examine the reliability of the selected deepfake detection model, and how the transformation of input variation places influence on the output. We have chosen MesoInception-4, a state-of-the-art deepfake detection model, as the target model and makeup as the anomalies. Makeups are applied through utilizing the Dlib library to obtain the 68 facial landmarks prior to filling in the RGB values. Metamorphic relations are derived based on the notion that realistic perturbations of the input images, such as makeup, involving eyeliners, eyeshadows, blushes, and lipsticks (which are common cosmetic appearance) applied to male and female images, should not alter the output of the model by a huge margin. Furthermore, we narrow down the scope to focus on revealing potential gender biases in DL and AI systems. Specifically, we are interested to examine whether MesoInception-4 model produces unfair decisions, which should be considered as a consequence of robustness issues. The findings from our work have the potential to pave the way for new research directions in the quality assurance and fairness in DL and AI systems.

Chameleon: Foundation Models for Fairness-Aware Multi-Modal Data Augmentation to Enhance Coverage of Minorities

Chameleon: Foundation Models for Fairness-aware Multi-modal Data Augmentation to Enhance Coverage of Minorities

Chameleon: Mixed-Modal Early-Fusion Foundation Models

FairCoT: Enhancing Fairness in Diffusion Models via Chain of Thought Reasoning of Multimodal Language Models

Generative models improve fairness of medical classifiers under distribution shifts

Data Augmentation via Diffusion Model to Enhance AI Fairness

LLM-Guided Counterfactual Data Generation for Fairer AI

Position: Cracking the Code of Cascading Disparity Towards Marginalized Communities

Fairness Evaluation in Deepfake Detection Models using Metamorphic Testing

Improving Fairness using Vision-Language Driven Image Augmentation

Chameleon: Images Are What You Need For Multimodal Learning Robust To Missing Modalities

Digi2Real: Bridging the Realism Gap in Synthetic Data Face Recognition via Foundation Models

3D-VirtFusion: Synthetic 3D Data Augmentation through Generative Diffusion Models and Controllable Editing

Quality-Diversity Generative Sampling for Learning with Synthetic Data

Improving the Fairness of Deep Generative Models without Retraining

Data Augmentation for Image Classification using Generative AI

DGM: a data generative model to improve minority class presence in anomaly detection domain

Chameleon: Increasing Label-Only Membership Leakage with Adaptive Poisoning

FairRAG: Fair Human Generation via Fair Retrieval Augmentation

Fairness Warnings and Fair-MAML: Learning Fairly with Minimal Data

Exploring Racial Bias within Face Recognition via per-subject Adversarially-Enabled Data Augmentation