Abstract:While traditional program evaluations typically rely on surveys to measure outcomes, certain economic outcomes such as living standards or environmental quality may be infeasible or costly to collect. As a result, recent empirical work estimates treatment effects using remotely sensed variables (RSVs), such mobile phone activity or satellite images, instead of ground-truth outcome measurements. Common practice predicts the economic outcome from the RSV, using an auxiliary sample of labeled RSVs, and then uses such predictions as the outcome in the experiment. We prove that this approach leads to biased estimates of treatment effects when the RSV is a post-outcome variable. We nonparametrically identify the treatment effect, using an assumption that reflects the logic of recent empirical research: the conditional distribution of the RSV remains stable across both samples, given the outcome and treatment. Our results do not require researchers to know or consistently estimate the relationship between the RSV, outcome, and treatment, which is typically mis-specified with unstructured data. We form a representation of the RSV for downstream causal inference by predicting the outcome and predicting the treatment, with better predictions leading to more precise causal estimates. We re-evaluate the efficacy of a large-scale public program in India, showing that the program's measured effects on local consumption and poverty can be replicated using satellite

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to effectively use Remotely Sensed Variables (RSVs) to replace traditional field survey data in project evaluation to estimate treatment effects. Specifically, when certain economic outcomes (such as living standards or environmental quality) are difficult or costly to collect, researchers try to use RSVs (such as satellite images, mobile phone activities, etc.) to replace ground - truth measurement results. However, existing practices may lead to bias, especially when RSV is a post - hoc variable. ### Main problems 1. **Limitations of traditional methods**: Traditional project evaluation relies on survey data to measure outcomes, but for certain economic outcomes (such as living standards or environmental quality), these data may be difficult or too costly to obtain. 2. **Bias in existing methods**: The current practice is to predict economic outcomes through auxiliary samples and use these predicted values as outcomes in experiments. This method will lead to arbitrary bias when RSV is a post - hoc variable. 3. **Challenges in causal inference**: How to use RSVs and auxiliary samples for causal inference without directly observing the target outcome. ### Core contributions of the paper 1. **Identifying treatment effects**: The paper proposes a non - parametric method to identify treatment effects, assuming that the conditional distribution of RSVs remains stable in both samples given the outcome and treatment variables. 2. **Avoiding bias**: The paper proves that existing methods may lead to arbitrary bias when RSV is a post - hoc variable and provides a method to avoid this bias. 3. **Application examples**: The paper shows how to use satellite images to replace field survey data by re - evaluating the effectiveness of a large - scale public project in India, thus saving a great deal of research costs. ### Specific problems and solutions - **Problem**: How to use RSVs for causal inference without directly observing the target outcome? - **Solutions**: - Assume that the conditional distribution of RSVs remains stable in the experimental sample and the observational sample given the outcome and treatment variables. - Use Bayes' rule to infer the response of the outcome to the treatment by learning the conditional probability of the treatment variable given RSVs. - Provide an intuitive non - parametric identification of the outcome, which is applicable to the case of binary outcomes. - Extend this intuition in a broader context, including the case where the treatment variable may directly affect RSVs. ### Empirical application The paper shows how to use satellite images to evaluate the effectiveness of a large - scale anti - poverty project through an empirical application. By collecting village - level coordinates, constructing a data set containing satellite images and night - light data, as well as village - level consumption and poverty level measurements, the paper reproduces the point estimates and confidence intervals of the experiment and verifies the effectiveness of the method. ### Conclusion This paper provides a new framework for using RSVs in project evaluation, solves the bias problem that may be caused by existing methods, and shows its effectiveness in practical applications.

Program Evaluation with Remotely Sensed Outcomes

Parameter Recovery Using Remotely Sensed Variables

Using Satellite Imagery and Machine Learning to Estimate the Livelihood Impact of Electricity Access

Improved Policy Evaluation for Randomized Trials of Algorithmic Resource Allocation

Image-based Treatment Effect Heterogeneity

Ground Control to Major Tom: the importance of field surveys in remotely sensed data analysis

Impact Evaluations in Data Poor Settings: The Case of Stress-Tolerant Rice Varieties in Bangladesh

Estimating Treatment Effects using Multiple Surrogates: The Role of the Surrogate Score and the Surrogate Index

A Spatial Perspective on the Econometrics of Program Evaluation

RCT Rejection Sampling for Causal Estimation Evaluation

Estimating Causal Impacts of Scaling a Voluntary Policy Intervention

Identification and estimation of treatment effects on long-term outcomes in clinical trials with external observational data

Precise unbiased estimation in randomized experiments using auxiliary observational data

Program Evaluation and Causal Inference with High-Dimensional Data

Improving uplift model evaluation on RCT data

Machine-Learning Tests for Effects on Multiple Outcomes

Estimating treatment effect heterogeneity in randomized program evaluation

Transporting stochastic direct and indirect effects to new populations

Predicting Counterfactuals from Large Historical Data and Small Randomized Trials

Increasing utilization of public health insurance programs: Evidence from an experiment in India

On the Evaluation of Surrogate Markers in Real World Data Settings