Abstract:Machine Learning (ML) has become an integral aspect of many real-world applications. As a result, the need for responsible machine learning has emerged, focusing on aligning ML models to ethical and social values, while enhancing their reliability and trustworthiness. Responsible ML involves many issues. This survey addresses four main issues: interpretability, fairness, adversarial robustness, and domain generalization. Feature selection plays a pivotal role in the responsible ML tasks. However, building upon statistical correlations between variables can lead to spurious patterns with biases and compromised performance. This survey focuses on the current study of causal feature selection: what it is and how it can reinforce the four aspects of responsible ML. By identifying features with causal impacts on outcomes and distinguishing causality from correlation, causal feature selection is posited as a unique approach to ensuring ML models to be ethically and socially responsible in high-stakes applications.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is the liability issue of machine learning (ML) models in practical applications, especially in high - risk fields such as recruitment, disaster relief, anti - disinformation, etc. Specifically, the paper focuses on how to enhance the performance of machine learning models in the following aspects through causal feature selection (CFS):
1. **Interpretability**: Traditional feature selection methods mainly rely on the statistical correlation between variables, which may make it difficult for the model to explain its decision - making process. Through causal feature selection, features that have a direct causal impact on the result can be identified, thereby improving the transparency and interpretability of the model.
2. **Fairness**: Traditional methods may cause the model to be biased based on certain sensitive features (such as gender, race, etc.). Causal feature selection ensures that the model is more fair when evaluating individuals or groups by identifying and removing the causal influence of these sensitive features.
3. **Adversarial Robustness**: The model may be affected by adversarial attacks, that is, a small input perturbation can cause the model to output errors. Causal feature selection can help the model reduce its sensitivity to misleading inputs, thereby improving its adversarial robustness.
4. **Domain Generalization**: The performance of the model in different environments and contexts may decline due to the limitations of training data. Causal feature selection helps the model maintain good performance in unseen scenarios by identifying generally relevant features.
### Core Objectives of the Paper
The main objective of the paper is to bridge the gap between theoretical goals and practical applications through causal feature selection, emphasizing the importance of causal relationships in creating machine learning systems that are not only technically advanced but also ethical and socially responsible. Specifically, the paper achieves this objective in the following ways:
- **Theoretical Framework**: Introduce the basic concepts and methods of causal feature selection, such as the identification of Markov Blanket, and how to distinguish correlation and causality through causal graphs.
- **Application Cases**: Explore the specific applications of causal feature selection in responsible machine learning tasks such as interpretability, fairness, adversarial robustness, and domain generalization.
- **Research Review**: Review existing causal feature selection methods and their applications in different fields, and propose future research directions and improvement measures.
### Formula Example
In the paper, a key concept in causal feature selection is Markov Blanket. Given a causal graph \(G\), where each node is a feature and the edges represent the causal relationships between features, the Markov Blanket \(\text{MB}(Y)\) includes the direct parent nodes, child nodes of the target variable \(Y\), and the parent nodes of its child nodes. Formally, the Markov Blanket satisfies the following condition:
\[Y \perp (X\setminus\text{MB}(Y))\mid\text{MB}(Y)\]
This means that given the Markov Blanket, the target variable \(Y\) is independent of other features.
### Conclusion
Through causal feature selection, the paper aims to promote more research and applications on responsible machine learning, especially in high - risk fields where machine learning decisions have far - reaching impacts. This method not only helps improve the technical performance of the model but also ensures that it complies with ethical and social standards.