Distribution-aware Fairness Test Generation

Sai Sathiesh Rajan,Ezekiel Soremekun,Yves Le Traon,Sudipta Chattopadhyay
2024-05-14
Abstract:Ensuring that all classes of objects are detected with equal accuracy is essential in AI systems. For instance, being unable to identify any one class of objects could have fatal consequences in autonomous driving systems. Hence, ensuring the reliability of image recognition systems is crucial. This work addresses how to validate group fairness in image recognition software. We propose a distribution-aware fairness testing approach (called DistroFair) that systematically exposes class-level fairness violations in image classifiers via a synergistic combination of out-of-distribution (OOD) testing and semantic-preserving image mutation. DistroFair automatically learns the distribution (e.g., number/orientation) of objects in a set of images. Then it systematically mutates objects in the images to become OOD using three semantic-preserving image mutations - object deletion, object insertion and object rotation. We evaluate DistroFair using two well-known datasets (CityScapes and MS-COCO) and three major, commercial image recognition software (namely, Amazon Rekognition, Google Cloud Vision and Azure Computer Vision). Results show that about 21% of images generated by DistroFair reveal class-level fairness violations using either ground truth or metamorphic oracles. DistroFair is up to 2.3x more effective than two main baselines, i.e., (a) an approach which focuses on generating images only within the distribution (ID) and (b) fairness analysis using only the original image dataset. We further observed that DistroFair is efficient, it generates 460 images per hour, on average. Finally, we evaluate the semantic validity of our approach via a user study with 81 participants, using 30 real images and 30 corresponding mutated images generated by DistroFair. We found that images generated by DistroFair are 80% as realistic as real-world images.
Computer Vision and Pattern Recognition,Machine Learning,Software Engineering
What problem does this paper attempt to address?
The paper mainly focuses on the fairness validation problem in image recognition software, especially the fairness of categories in multi-label object classification (MLC) tasks. Studies have pointed out that image recognition systems must be able to equally and accurately identify all category objects. For example, failing to recognize a certain class of objects in autonomous driving may lead to serious consequences. The paper proposes a distribution-aware fairness testing method called "DISTRO FAIR", which systematically exposes fairness violations of categories in image classifiers by combining out-of-distribution (OOD) testing and semantically preserved image variations. DISTRO FAIR first automatically learns the distribution of objects in the image dataset, and then generates OOD images with semantically preserved variations such as object removal, insertion, and rotation. The evaluation on CityScapes, MS-COCO datasets, as well as commercial image recognition software such as Amazon Rekognition, Google Cloud Vision, and Azure Computer Vision, shows that approximately 21% of the images generated by DISTRO FAIR reveal fairness errors. Compared to methods that only generate images within the distribution (ID) and fairness analysis using only the original image dataset, DISTRO FAIR can improve the effectiveness by up to 2.3 times. Furthermore, the paper validates the semantic validity of the generated images through user studies, finding that these images achieve an average realism of 80%. This work provides a new OOD testing method for detecting and analyzing fairness errors at the category level in image classification tasks, contributing to ensuring the reliability and fairness of AI systems.