Abstract:Personalized decision making requires the knowledge of potential outcomes under different treatments, and confidence intervals about the potential outcomes further enrich this decision-making process and improve its reliability in high-stakes scenarios. Predicting potential outcomes along with its uncertainty in a counterfactual world poses the foundamental challenge in causal inference. Existing methods that construct confidence intervals for counterfactuals either rely on the assumption of strong ignorability, or need access to un-identifiable lower and upper bounds that characterize the difference between observational and interventional distributions. To overcome these limitations, we first propose a novel approach wTCP-DR based on transductive weighted conformal prediction, which provides confidence intervals for counterfactual outcomes with marginal converage guarantees, even under hidden confounding. With less restrictive assumptions, our approach requires access to a fraction of interventional data (from randomized controlled trials) to account for the covariate shift from observational distributoin to interventional distribution. Theoretical results explicitly demonstrate the conditions under which our algorithm is strictly advantageous to the naive method that only uses interventional data. After ensuring valid intervals on counterfactuals, it is straightforward to construct intervals for individual treatment effects (ITEs). We demonstrate our method across synthetic and real-world data, including recommendation systems, to verify the superiority of our methods compared against state-of-the-art baselines in terms of both coverage and efficiency

What problem does this paper attempt to address?

This paper attempts to solve the problem of how to provide confidence intervals with marginal coverage guarantees for counterfactual results in the presence of latent confounding factors. Specifically, existing methods either rely on the strong unignorable assumption that completely ignores latent confounding factors, or require access to unidentifiable upper and lower limits to characterize the difference between the observational distribution and the interventional distribution. These problems limit the effectiveness and reliability of existing methods in practical applications. To address these limitations, the paper proposes two new methods: 1. **Weighted Transductive Conformal Prediction and Density Ratio Estimation (wTCP - DR)**: This method is based on weighted transductive conformal prediction and can provide confidence intervals with marginal coverage guarantees for counterfactual results in the presence of latent confounding factors. It uses a part of the interventional data (such as randomized controlled trial data) to correct the covariate shift from the observational distribution to the interventional distribution. 2. **Weighted Piecewise Conformal Prediction and Density Ratio Estimation (wSCP - DR)**: This is a two - stage variant of wTCP - DR, based on piecewise conformal prediction, with the same marginal coverage guarantee but significantly reduced computational cost. The key to these two methods is to re - weight the confidence intervals of conformal prediction by learning the density ratio between the observational data and the interventional data, thereby providing more reliable confidence interval estimates in the presence of latent confounding factors. The paper also demonstrates the superiority of these methods in synthetic data and real - world data (including recommendation systems) through theoretical analysis and experiments.

Conformal Counterfactual Inference under Hidden Confounding

Conformal Inference of Counterfactuals and Individual Treatment Effects

Conformal causal inference for cluster randomized trials: model-robust inference without asymptotic approximations

Conformal Prediction for Causal Effects of Continuous Treatments

On the Role of Surrogates in Conformal Inference of Individual Causal Effects

Distribution-Free Prediction Intervals Under Covariate Shift, With an Application to Causal Inference

Causal Inference and Counterfactual Prediction in Machine Learning for Actionable Healthcare

Learning to Bound Counterfactual Inference from Observational, Biased and Randomised Data

Counterfactual inference for sequential experiments

Conformal Diffusion Models for Individual Treatment Effect Estimation and Inference

Counterfactual Evaluation of Treatment Assignment Functions with Networked Observational Data.

Conformal Sensitivity Analysis for Individual Treatment Effects

Counterfactual Prediction Under Outcome Measurement Error

Conformal Prediction for Dose-Response Models with Continuous Treatments

Interval Estimation of Individual-Level Causal Effects under Unobserved Confounding

Causal Inference for a Hidden Treatment

Multi-Source Conformal Inference Under Distribution Shift

Hidden yet quantifiable: A lower bound for confounding strength using randomized trials

Mitigating Confounding Bias for Recommendation via Counterfactual Inference