Benchmarking Bayesian Causal Discovery Methods for Downstream Treatment Effect Estimation

Chris Chinenye Emezue,Alexandre Drouin,Tristan Deleu,Stefan Bauer,Yoshua Bengio

2023-07-30

Abstract:The practical utility of causality in decision-making is widespread and brought about by the intertwining of causal discovery and causal inference. Nevertheless, a notable gap exists in the evaluation of causal discovery methods, where insufficient emphasis is placed on downstream inference. To address this gap, we evaluate seven established baseline causal discovery methods including a newly proposed method based on GFlowNets, on the downstream task of treatment effect estimation. Through the implementation of a distribution-level evaluation, we offer valuable and unique insights into the efficacy of these causal discovery methods for treatment effect estimation, considering both synthetic and real-world scenarios, as well as low-data scenarios. The results of our study demonstrate that some of the algorithms studied are able to effectively capture a wide range of useful and diverse ATE modes, while some tend to learn many low-probability modes which impacts the (unrelaxed) recall and precision.

Machine Learning,Methodology

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the insufficient attention to downstream tasks (especially treatment effect estimation) in the evaluation of causal discovery methods. Specifically, the author points out that although causal discovery and causal inference are widely used in practical decision - making, the existing evaluation of causal discovery methods mainly focuses on the accuracy of structure learning and ignores the performance of these methods in subsequent causal inference tasks. To bridge this gap, the paper evaluates the performance of seven established causal discovery methods (including a new method based on GFlowNets) on the downstream task of treatment effect estimation. By conducting a distribution - level evaluation, the paper provides valuable and unique insights into the effectiveness of these causal discovery methods in treatment effect estimation, covering synthetic data and real - world data scenarios, as well as low - data - volume scenarios.

The main objective of the paper is to provide researchers and practitioners with a more comprehensive perspective on the performance of different causal discovery methods in practical applications, especially in the performance of this key task of treatment effect estimation, by systematically evaluating different causal discovery methods. This not only helps to improve existing causal discovery methods, but also promotes the development of new methods to better serve downstream causal inference tasks.