Abstract:In June 2024, the new EU AI Act came into force. The AI Act includes obligations for the provider of an AI system. Article 10 of the AI Act includes a new obligation for providers to evaluate whether their training, validation and testing datasets meet certain quality criteria, including an appropriate examination of biases in the datasets and correction measures. With the obligation comes a new provision in Article 10(5) AI Act, allowing providers to collect sensitive data to fulfil the obligation. The exception aims to prevent discrimination. In this paper, I research the scope and implications of Article 10(5) AI Act. The paper primarily concerns European Union law, but may be relevant in other parts of the world, as policymakers aim to regulate biases in AI systems.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is: within the framework of the EU AI Act, how to eliminate biases in AI systems by collecting and processing sensitive data, thereby preventing discrimination. Specifically, the paper explores the provisions of Article 10(5) of the EU AI Act, which allows providers of AI systems to process sensitive data under specific conditions to detect and correct biases.
### Problem Background
1. **The Problem of Bias in AI Systems**
- AI systems may cause indirect discrimination during the decision - making process. For example, an AI system used by a bank may inadvertently reject mortgage applications from people of certain skin colors.
- Such biases not only affect individual rights but may also violate EU anti - discrimination laws.
2. **Limitations of GDPR**
- According to the General Data Protection Regulation (GDPR), collecting and processing sensitive data (such as race, religion, sexual orientation, etc.) is generally prohibited.
- This makes it difficult for developers to detect and correct biases in AI systems by analyzing these sensitive data.
### Core Issues of the Paper
- **The Necessity of the Exception Clause**: Why is it necessary to introduce an exception clause in the EU AI Act to allow the processing of sensitive data?
- **The Scope and Conditions of the Exception Clause**: What are the specific contents of Article 10(5)? What requirements does it impose on providers of AI systems?
- **Practical Feasibility**: Can providers effectively use this exception clause to prevent discrimination in AI systems?
### Solutions
The paper analyzes in detail the content of Article 10(5) of the EU AI Act and explores the following points:
1. **Defining High - Risk AI Systems**
- High - risk AI systems refer to those systems that may have a significant impact on health, safety, or fundamental rights. For example, AI applications in areas such as biometric systems, critical infrastructure, education, and employment.
2. **Responsibilities of Providers**
- Providers refer to natural or legal persons who develop AI systems and put them into the market or service. They are obliged to ensure the quality of training, validation, and test data sets, especially to detect and correct potential biases.
3. **Bias Detection and Correction**
- Providers must check whether there are biases in the data set that may lead to discrimination and take appropriate measures to prevent and mitigate these biases.
- Only when strictly necessary can providers process sensitive data, and they must abide by strict privacy protection measures.
### Conclusion
The paper believes that Article 10(5) of the EU AI Act provides a legal way for providers of AI systems to detect and correct biases in AI systems by processing sensitive data without violating GDPR, thereby preventing discrimination. However, the practical operability and effectiveness of this exception clause still need to be further explored and verified.
---
Through the above analysis, the paper aims to provide in - depth insights for policy - makers, technology practitioners, and legal scholars on how to effectively prevent AI system discrimination within the legal framework.