Abstract:Automated decision-making (ADM) systems are being deployed across a diverse range of critical problem areas such as social welfare and healthcare. Recent work highlights the importance of causal ML models in ADM systems, but implementing them in complex social environments poses significant challenges. Research on how these challenges impact the performance in specific downstream decision-making tasks is limited. Addressing this gap, we make use of a comprehensive real-world dataset of jobseekers to illustrate how the performance of a single CATE model can vary significantly across different decision-making scenarios and highlight the differential influence of challenges such as distribution shifts on predictions and allocations.

What problem does this paper attempt to address?

The key problem that this paper attempts to solve is the performance differences of causal machine learning (Causal ML) models in different decision - making scenarios and the challenges they face in automated decision - making systems (ADM). Specifically, the research aims to: 1. **Evaluate the performance of causal ML models in specific downstream decision - making tasks**: - Existing research mainly focuses on how to construct and optimize causal ML models, but there is less research on their application effects in the actual complex social environment, especially their performance in specific decision - making tasks. - This paper shows the performance changes of a single conditional average treatment effect (CATE) model in different decision - making scenarios by using a comprehensive real - world dataset (such as job seeker data). 2. **Explore the impact of distribution shift on prediction and allocation**: - Distribution shift refers to the difference in data distribution between training data and the deployment environment, and this difference may lead to a significant decline in model performance. - The research emphasizes the specific impact of challenges such as distribution shift on prediction and resource allocation, and proves through experiments the different degrees of impact of these challenges in different decision - making scenarios. 3. **Propose the unique complexity of causal decision - making**: - Emphasize the unique requirements of causal ML models in decision - making, and point out that causal ML methods need to be adjusted according to specific decision - making tasks to ensure their effectiveness and reliability. ### Main contributions - **Fill the research gap**: Reveal the performance differences of causal ML models in different decision - making scenarios through real - world datasets. - **Highlight the impact of distribution shift**: Analyze in detail the impact of distribution shift on model performance and show its specific performance in different decision - making scenarios. - **Emphasize the importance of causal decision - making**: Propose the unique requirements of causal ML models in decision - making and emphasize the necessity of adjusting models for specific tasks. ### Formula summary - **Conditional average treatment effect (CATE)**: \[ \tau(x)=E[Y_i(1)-Y_i(0)\mid X_i = x] \] where \(Y_i(1)\) and \(Y_i(0)\) represent the potential outcomes of individual \(i\) when receiving and not receiving the intervention respectively, and \(X_i\) is the covariate vector. - **Optimal allocation strategy (Cost Efficient Allocation, CE)**: \[ \max_{\pi\in\{X\to\{0,1\}\}}\sum_{i = 1}^n\pi(x_i)\hat{\tau}(x_i) \] Subject to the constraint: \[ \sum_{i = 1}^n\pi(x_i)c_i\leq C \] where \(c_i\) represents the intervention cost of each individual, and \(C\) is the total budget. Through these studies, the author hopes to provide directions for future research, especially in evaluating the reliability and robustness of causal ML models in specific decision - making tasks.

The Missing Link: Allocation Performance in Causal Machine Learning

Causal Inference and Counterfactual Prediction in Machine Learning for Actionable Healthcare

Direct Heterogeneous Causal Learning for Resource Allocation Problems in Marketing

Designing Closed-Loop Models for Task Allocation

Making the most of AI and machine learning in organizations and strategy research: Supervised machine learning, causal inference, and matching models

Decision-Aware Predictive Model Selection for Workforce Allocation

Causal Inference Meets Machine Learning

Double Machine Learning for Adaptive Causal Representation in High-Dimensional Data

Mind the Gap: A Causal Perspective on Bias Amplification in Prediction & Decision-Making

Efficient Real-world Testing of Causal Decision Making via Bayesian Experimental Design for Contextual Optimisation

The Mismeasure of Man and Models: Evaluating Allocational Harms in Large Language Models

Bridging the gap: Towards an Expanded Toolkit for AI-driven Decision-Making in the Public Sector

Causal machine learning for healthcare and precision medicine

Deep End-to-end Causal Inference

Integrating Active Learning in Causal Inference with Interference: A Novel Approach in Online Experiments

Machine learning in policy evaluation: new tools for causal inference

Causal Machine Learning: A Survey and Open Problems

CausalConceptTS: Causal Attributions for Time Series Classification using High Fidelity Diffusion Models

What if? Causal Machine Learning in Supply Chain Risk Management

Exploiting Causal Structure for Robust Model Selection in Unsupervised Domain Adaptation

Machine Learning-Powered Course Allocation