Navigating Fairness in Radiology AI: Concepts, Consequences,and Crucial Considerations

Vasantha Kumar Venugopal,Abhishek Gupta,Rohit Takhar,Charlene Liew Jin Yee,Catherine Jones,Gilberto Szarf

2023-06-02

Abstract:Artificial Intelligence (AI) has significantly revolutionized radiology, promising improved patient outcomes and streamlined processes. However, it's critical to ensure the fairness of AI models to prevent stealthy bias and disparities from leading to unequal outcomes. This review discusses the concept of fairness in AI, focusing on bias auditing using the Aequitas toolkit, and its real-world implications in radiology, particularly in disease screening scenarios. Aequitas, an open-source bias audit toolkit, scrutinizes AI models' decisions, identifying hidden biases that may result in disparities across different demographic groups and imaging equipment brands. This toolkit operates on statistical theories, analyzing a large dataset to reveal a model's fairness. It excels in its versatility to handle various variables simultaneously, especially in a field as diverse as radiology. The review explicates essential fairness metrics: Equal and Proportional Parity, False Positive Rate Parity, False Discovery Rate Parity, False Negative Rate Parity, and False Omission Rate Parity. Each metric serves unique purposes and offers different insights. We present hypothetical scenarios to demonstrate their relevance in disease screening settings, and how disparities can lead to significant real-world impacts.

Machine Learning,Artificial Intelligence,Computers and Society

What problem does this paper attempt to address?

### Problems the paper attempts to solve

The paper "Navigating Fairness in AI Radiology: Concepts, Consequences, and Crucial Considerations" attempts to solve the fairness issues existing in the use of artificial intelligence (AI) in medical imaging. Specifically, the paper focuses on the following aspects:

1. **Bias audit**: How to detect and evaluate biases in AI models through tools and techniques, especially using the Aequitas toolkit for bias audit.
2. **Definition of fairness**: Clearly define what bias, disparity, and fairness are, and explain the specific meanings of these concepts in AI and healthcare.
3. **Fairness metrics**: Introduce multiple fairness metrics, such as Equal and Proportional Parity, False Positive Rate Parity, False Discovery Rate Parity, False Negative Rate Parity, and False Omission Rate Parity, and discuss their applications in disease screening.
4. **Practical impacts**: Demonstrate the impacts of bias in actual medical scenarios through hypothetical situations, especially the importance in disease screening.
5. **Bias mitigation strategies**: Propose some strategies to mitigate biases in AI models, including the representativeness of datasets, algorithm adjustment, post - processing of model decisions, transparency and interpretability, and regular audits.

### Formula presentation

Here are some key fairness metrics and their formulas:

1. **False Positive Rate (FPR)**:
\[
\text{FPR}=\frac{\text{FP}}{\text{FP}+\text{TN}}
\]
where:
- \(\text{FP}\) represents False Positive (incorrectly identifying a healthy individual as diseased)
- \(\text{TN}\) represents True Negative (correctly identifying a healthy individual as healthy)

2. **False Discovery Rate (FDR)**:
\[
\text{FDR}=\frac{\text{FP}}{\text{PP}}=\frac{\text{FP}}{\text{FP}+\text{TP}}
\]
where:
- \(\text{FP}\) represents False Positive
- \(\text{PP}\) represents Predicted Positive (the total number of positive predictions by the model)
- \(\text{TP}\) represents True Positive (correctly identifying a diseased individual as diseased)

3. **False Negative Rate (FNR)**:
\[
\text{FNR}=\frac{\text{FN}}{\text{FN}+\text{TP}}
\]
where:
- \(\text{FN}\) represents False Negative (incorrectly identifying a diseased individual as healthy)
- \(\text{TP}\) represents True Positive

4. **False Omission Rate (FOR)**:
\[
\text{FOR}=\frac{\text{FN}}{\text{PN}}=\frac{\text{FN}}{\text{FN}+\text{TN}}
\]
where:
- \(\text{FN}\) represents False Negative
- \(\text{PN}\) represents Predicted Negative (the total number of negative predictions by the model)
- \(\text{TN}\) represents True Negative