Abstract:Supervised machine learning (ML) and deep learning (DL) algorithms excel at predictive tasks, but it is commonly assumed that they often do so by exploiting non-causal correlations, which may limit both interpretability and generalizability. Here, we show that this trade-off between explanation and prediction is not as deep and fundamental as expected. Whereas ML and DL algorithms will indeed tend to use non-causal features for prediction when fed indiscriminately with all data, it is possible to constrain the learning process of any ML and DL algorithm by selecting features according to Pearl's backdoor adjustment criterion. In such a situation, some algorithms, in particular deep neural networks, can provide near unbiased effect estimates under feature collinearity. Remaining biases are explained by the specific algorithmic structures as well as hyperparameter choice. Consequently, optimal hyperparameter settings are different when tuned for prediction or inference, confirming the general expectation of a trade-off between prediction and explanation. However, the effect of this trade-off is small compared to the effect of a causally constrained feature selection. Thus, once the causal relationship between the features is accounted for, the difference between prediction and explanation may be much smaller than commonly assumed. We also show that such causally constrained models generalize better to new data with altered collinearity structures, suggesting generalization failure may often be due to a lack of causal learning. Our results not only provide a perspective for using ML for inference of (causal) effects but also help to improve the generalizability of fitted ML and DL models to new data.

What problem does this paper attempt to address?

The paper primarily explores the potential application of Machine Learning (ML) and Deep Learning (DL) algorithms in causal inference, particularly focusing on the performance of these algorithms when faced with feature collinearity. The core issue of the paper is: **Research Question:** - Investigate whether predictive models can be used for causal inference. The paper points out that it is generally believed in the fields of statistics and machine learning that there is a difference between predictive modeling and explanatory or causal modeling. Specifically, utilizing the correlation between features and responses can improve predictive performance, even if there is no causal relationship between these variables. This has led to the success of supervised machine learning and deep learning algorithms in predictive tasks, but it has also limited the interpretability and generalizability of the models. Therefore, the main goal of this paper is to explore whether the trade-off between prediction and explanation is as profound and fundamental as commonly perceived. **Key Findings Include:** - By selecting features to satisfy Pearl's backdoor adjustment criterion, the learning process of any ML and DL algorithm can be constrained to some extent, thereby eliminating the influence of non-causal features and achieving approximately unbiased effect estimation. - Certain algorithms, such as deep neural networks, can provide nearly unbiased effect estimates in the presence of feature collinearity. - Unlike predictive tasks, the optimal hyperparameter settings differ when algorithms are tuned for causal inference. - Once the causal relationships between features are considered, the gap between prediction and explanation may be much smaller than usually thought. - The study also shows that such causally constrained models perform better on new datasets, especially when the feature collinearity structure changes in the datasets. In short, the paper attempts to answer the question: **Can ML and DL algorithms effectively perform causal inference when dealing with feature collinearity through appropriate methods?** The research results indicate that by reasonably selecting features and optimizing algorithms, these models can indeed overcome the influence of non-causal associations to a certain extent, thereby better estimating causal effects.

Can predictive models be used for causal inference?

Causal Inference and Counterfactual Prediction in Machine Learning for Actionable Healthcare

Linking a predictive model to causal effect estimation

Prediction can be safely used as a proxy for explanation in causally consistent Bayesian generalized linear models

Explaining the Behavior of Black-Box Prediction Algorithms with Causal Learning

Reliable Estimation of Causal Effects Using Predictive Models

On the Relationship Between Explanation and Prediction: A Causal View

CausaLM: Causal Model Explanation Through Counterfactual Language Models

Towards causality-aware predictions in static anticausal machine learning tasks: the linear structural causal model case

Do causal predictors generalize better to new domains?

Mechanism learning: Reverse causal inference in the presence of multiple unknown confounding through front-door causal bootstrapping

From Dependence to Causation

The Causal Loss: Driving Correlation to Imply Causation

A Causal Lens for Peeking into Black Box Predictive Models: Predictive Model Interpretation via Causal Attribution

Causal Inference Meets Machine Learning

Causal machine learning for predicting treatment outcomes

Explaining The Efficacy of Counterfactually Augmented Data

Predictors from causal features do not generalize better to new domains

Can We Utilize Pre-trained Language Models within Causal Discovery Algorithms?

Reinterpreting causal discovery as the task of predicting unobserved joint statistics