Open Challenges on Fairness of Artificial Intelligence in Medical Imaging Applications

Enzo Ferrante,Rodrigo Echeveste

2024-07-24

Abstract:Recently, the research community of computerized medical imaging has started to discuss and address potential fairness issues that may emerge when developing and deploying AI systems for medical image analysis. This chapter covers some of the pressing challenges encountered when doing research in this area, and it is intended to raise questions and provide food for thought for those aiming to enter this research field. The chapter first discusses various sources of bias, including data collection, model training, and clinical deployment, and their impact on the fairness of machine learning algorithms in medical image computing. We then turn to discussing open challenges that we believe require attention from researchers and practitioners, as well as potential pitfalls of naive application of common methods in the field. We cover a variety of topics including the impact of biased metrics when auditing for fairness, the leveling down effect, task difficulty variations among subgroups, discovering biases in unseen populations, and explaining biases beyond standard demographic attributes.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The paper primarily explores the issue of fairness in the field of artificial intelligence (AI) for medical image analysis and outlines a series of open challenges and key issues encountered in research. Specifically, the paper aims to address the following core issues:

1. **Identifying Sources of Bias**: The paper discusses various sources of bias that may arise at different stages such as data collection, model training, and clinical deployment, and their impact on the fairness of machine learning algorithms.

2. **Addressing Metric Bias in Bias Assessment**: The authors point out the problem of using certain metrics in fairness audits that may lead to misleading results, especially when these metrics have inherent biases towards different population groups.

3. **Avoiding the "Leveling Down" Effect**: Researchers focus on how to avoid achieving superficial fairness by lowering the performance of well-performing groups, which often results in a decline in performance for all groups.

4. **Handling Task Difficulty Differences**: The paper analyzes how variations in task difficulty due to anatomical differences affect the performance of different population subgroups and discusses the complexity of evaluating model fairness in such scenarios.

5. **Uncovering Bias in Unseen Populations**: The paper emphasizes the potential for unexpected bias issues when AI systems are applied in new environments with distribution differences from the original training data.

In summary, the focus of this paper is to propose and discuss various challenges in ensuring fairness when applying AI to medical image analysis and to encourage researchers to explore solutions to overcome these issues.