Zero-shot causal learning

Hamed Nilforoshan,Michael Moor,Yusuf Roohani,Yining Chen,Anja Šurina,Michihiro Yasunaga,Sara Oblak,Jure Leskovec

2024-02-23

Abstract:Predicting how different interventions will causally affect a specific individual is important in a variety of domains such as personalized medicine, public policy, and online marketing. There are a large number of methods to predict the effect of an existing intervention based on historical data from individuals who received it. However, in many settings it is important to predict the effects of novel interventions (e.g., a newly invented drug), which these methods do not address. Here, we consider zero-shot causal learning: predicting the personalized effects of a novel intervention. We propose CaML, a causal meta-learning framework which formulates the personalized prediction of each intervention's effect as a task. CaML trains a single meta-model across thousands of tasks, each constructed by sampling an intervention, its recipients, and its nonrecipients. By leveraging both intervention information (e.g., a drug's attributes) and individual features~(e.g., a patient's history), CaML is able to predict the personalized effects of novel interventions that do not exist at the time of training. Experimental results on real world datasets in large-scale medical claims and cell-line perturbations demonstrate the effectiveness of our approach. Most strikingly, \method's zero-shot predictions outperform even strong baselines trained directly on data from the test interventions.

Machine Learning,Artificial Intelligence,Computers and Society,Human-Computer Interaction

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to predict the personalized impact of new interventions on specific individuals in the absence of historical data. Specifically, many existing methods can predict the effects of existing interventions based on historical data, but these methods are unable to deal with newly emerging interventions (such as newly invented drugs). Therefore, this paper focuses on zero - shot causal learning, that is, predicting the personalized effects of new interventions without training data.

To achieve this goal, the authors propose a Causal Meta - Learning (CaML) framework. This framework estimates the Conditional Average Treatment Effect (CATE) of multiple interventions, including new interventions that did not exist during model training, by constructing a meta - model. The key to CaML is to individualize the prediction task for each intervention. It constructs tasks by sampling interventions, recipients, and non - recipients, and uses intervention information (such as drug properties) and personal characteristics (such as patient history) to predict the personalized effects of new interventions.

The experimental results show that CaML outperforms strong baseline models trained directly on test intervention data on large - scale real - world datasets. In particular, on large - scale datasets of medical claims and cell - line perturbations, CaML's zero - shot prediction performance is particularly prominent. In addition, CaML can accurately predict the effects of new intervention combinations when trained based on only a single intervention, further demonstrating its advantage in zero - shot generalization ability.