Abstract:Scientists try to design experiments that will yield maximal information. For instance, given the available evidence and a limitation on the number of variables that can be observed simultaneously, it may be more informative to intervene on variable $X$ and observe the response of variable $Y$ than to intervene on $X$ and observe $Z$ ; in other situations, the opposite may be true. Scientists must often make these decisions without primary data. To address this problem, in previous work, we created software for annotating aggregate statistics in the literature and deriving consistent causal explanations, expressed as causal graphs. This meta-analytic pipeline is useful not only for synthesizing evidence but also for planning experiments: one can use it strategically to select experiments that could further eliminate causal graphs from consideration. In this paper, we introduce interpretable policies for selecting experiments in the context of piecemeal causal discovery, a common setting in biological sciences in which each experiment can measure not an entire system but rather a strict subset of its variables. The limits of this piecemeal approach are only beginning to be fully characterized, with crucial theoretical work published recently. With simulations, we show that our experiment-selection policies identify causal structures more efficiently than random experiment selection. Unlike methods that require primary data, our meta-analytic approach offers a flexible alternative for those seeking to incorporate qualitative domain knowledge into their search for causal mechanisms. We also present a method that categorizes hypotheses with respect to their utility for identifying a system’s causal structure. Although this categorization is usually infeasible to perform manually, it is critical for conducting research efficiently.

Beyond integrative experiment design: Systematic experimentation guided by causal discovery AI

Experimental Design For Causal Inference Through An Optimization Lens

Adaptive Experimental Design and Counterfactual Inference

Adaptive Experiment Design for Probabilistic Integration

Causal Inference in Generalizable Environments: Systematic Representative Design

Experiment Selection in Meta-Analytic Piecemeal Causal Discovery

Neuroadaptive Bayesian optimisation can allow integrative design spaces at the individual level in the social and behavioural sciences... and beyond

Bayesian Optimal Experimental Design for Inferring Causal Structure

Integrative design for thought-experiments

Computer-Aided Experiment Planning Toward Causal Discovery in Neuroscience

Active learning for optimal intervention design in causal models

Explore your experimental designs and theories before you exploit them!

Integrative experiments require a shared theoretical and methodological basis

The future of experimental design: Integrative, but is the sample diverse enough?

[Observation and Experiment: a Causal Perspective].

There are no shortcuts to theory

Getting lost in an infinite design space is no solution

AI - Based On The Fly Design of Experiments in Physics and Engineering

Bayesian Intervention Optimization for Causal Discovery

Adaptive Online Experimental Design for Causal Discovery