Introducing CausalBench: A Flexible Benchmark Framework for Causal Analysis and Machine Learning

Ahmet Kapkiç,Pratanu Mandal,Shu Wan,Paras Sheth,Abhinav Gorantla,Yoonhyuk Choi,Huan Liu,K. Selçuk Candan

2024-09-25

Abstract:While witnessing the exceptional success of machine learning (ML) technologies in many applications, users are starting to notice a critical shortcoming of ML: correlation is a poor substitute for causation. The conventional way to discover causal relationships is to use randomized controlled experiments (RCT); in many situations, however, these are impractical or sometimes unethical. Causal learning from observational data offers a promising alternative. While being relatively recent, causal learning aims to go far beyond conventional machine learning, yet several major challenges remain. Unfortunately, advances are hampered due to the lack of unified benchmark datasets, algorithms, metrics, and evaluation service interfaces for causal learning. In this paper, we introduce {\em CausalBench}, a transparent, fair, and easy-to-use evaluation platform, aiming to (a) enable the advancement of research in causal learning by facilitating scientific collaboration in novel algorithms, datasets, and metrics and (b) promote scientific objectivity, reproducibility, fairness, and awareness of bias in causal learning research. CausalBench provides services for benchmarking data, algorithms, models, and metrics, impacting the needs of a broad of scientific and engineering disciplines.

Machine Learning

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is the lack of unified benchmark datasets, algorithms, evaluation metrics and service interfaces in the current research field of causal learning, which hinders the progress in the field of causal learning. Specifically, the paper points out that although machine - learning (ML) techniques have achieved remarkable success in many applications, one of its key flaws is that correlation cannot be a good substitute for causality. The traditional method of discovering causal relationships is to use randomized controlled trials (RCT), but in many cases, these methods are impractical or sometimes even unethical. Therefore, causal learning from observational data provides a promising alternative. However, due to the lack of a unified benchmark framework, the research progress of causal learning has been limited.

To meet this challenge, the author introduces **CausalBench**, which is a transparent, fair and easy - to - use evaluation platform, aiming to:
1. **Promote the development of causal learning research**: By promoting scientific cooperation on new algorithms, datasets and evaluation metrics.
2. **Promote the objectivity, repeatability, fairness and awareness of bias in scientific research**: Ensure the scientific rigor and transparency of causal learning research.

CausalBench provides services for benchmarking data, algorithms, models and evaluation metrics, meeting the needs of a wide range of scientific and engineering disciplines. Through these services, CausalBench hopes to be able to systematically, objectively and transparently evaluate causal learning models and algorithms, thus promoting the research and development in this field.