Comment: Estimating the Effects Caused by Treatments

Paul R. Rosenbaum,Donald B. Rubin

DOI: https://doi.org/10.1080/01621459.1984.10477057

IF: 4.369

1984-01-01

Journal of the American Statistical Association

Abstract:(1984). Comment: Estimating the Effects Caused by Treatments. Journal of the American Statistical Association: Vol. 79, No. 385, pp. 26-28.

What problem does this paper attempt to address?

Estimating causal effects of treatments in experimental and observational studies

Donald Rubin

DOI: https://doi.org/10.1002/j.2333-8504.1972.tb00631.x

1972-01-01

ETS Research Bulletin Series

Abstract:ABSTRACT A discussion of matching, randomization, random sampling, and other methods of controlling extraneous variation is presented. The objective is to specify the benefits of randomization in estimating causal effects of treatments. The basic conclusion is that randomization should be employed whenever possible, but the use of carefully controlled nonrandomized data to estimate causal effects is a reasonable and necessary procedure in many cases.
Estimating Causal Effects of Treatments in Randomized and Nonrandomized Studies

DB RUBIN

DOI: https://doi.org/10.1037/h0037350

1974-01-01

Journal of Educational Psychology

Abstract:A discussion of matching, randomization, random sampling, and other methods of controlling extraneous variation is presented. The objective is to specify the benefits of randomization in estimating causal effects of treatments. The basic conclusion is that randomization should be employed whenever possible but that the use of carefully controlled nonrandomized data to estimate causal effects is a reasonable and necessary procedure in many cases. Recent psychological and educational literature has included extensive criticism of the use of nonrandomized studies to estimate causal effects of treatments (e.g., Campbell & Erlebacher, 1970). The implication in much of this literature is that only properly randomized experiments can lead to useful estimates of causal effects. If taken as applying to all fields of study, this position is untenable. Since the extensive use of randomized experiments is limited to the last half century,8 and in fact is not used in much scientific investigation today,4 one is led to the conclusion that most scientific truths have been established without using randomized experiments. In addition, most of us successfully determine the causal effects of many of our everyday actions, even interpersonal behaviors, without the benefit of randomization. Even if the position that causal effects of treatments can only be well established from randomized experiments is taken as applying only to the social sciences in which
Estimating Average Treatment Effects: Supplementary Analyses and Remaining Challenges

Susan Athey,Guido Imbens,Thai Pham,Stefan Wager

DOI: https://doi.org/10.48550/arXiv.1702.01250

2017-02-04

Abstract:There is a large literature on semiparametric estimation of average treatment effects under unconfounded treatment assignment in settings with a fixed number of covariates. More recently attention has focused on settings with a large number of covariates. In this paper we extend lessons from the earlier literature to this new setting. We propose that in addition to reporting point estimates and standard errors, researchers report results from a number of supplementary analyses to assist in assessing the credibility of their estimates.

Methodology,Econometrics
THE ASSESSMENT OF A TREATMENT EFFECT WHEN A COVARIATE IS OBSERVED ON A SUBSET OF THE UNITS: I. SIMPLE ESTIMATORS

Donald B. Rubin,Sanford Weisberg

DOI: https://doi.org/10.1002/j.2333-8504.1974.tb00671.x

1974-01-01

Research Bulletin

Abstract:ABSTRACTConsider a completely randomized experiment with two groups, say treatment and control, and a covariate X that has been observed on a subset of the units. The objective is to estimate τ, the treatment effect on the variable Y which is recorded for all units. Although the average Y difference in the two groups, , is unbiased for τ, it is common to try to use X to increase the precision of the estimate of τ by forming a covariance adjustment, especially if the correlation between X and Y, ρ, is large. If X is not fully observed it is not clear whether to and/or how to form the covariance adjusted estimate. In addition to , we consider two covariance adjusted estimates of τ, which ignores the units without X, and which fills in group means for the missing values. We compare the three estimators' variances and find the conditions under which each estimator is best.
Treatment Effect Measures Under Nonproportional Hazards

Dan Jackson,Michael Sweeting,Rose Baker

DOI: https://doi.org/10.1002/pst.2449

2024-10-28

Pharmaceutical Statistics

Abstract:'Treatment effect measures under nonproportional hazards' by Snapinn et al. (Pharmaceutical Statistics, 22, 181–193) recently proposed some novel estimates of treatment effect for time‐to‐event endpoints. In this note, we clarify three points related to the proposed estimators that help to elucidate their properties. We hope that their work, and this commentary, will motivate further discussion concerning treatment effect measures that do not require the proportional hazards assumption.

pharmacology & pharmacy,statistics & probability
Robust Estimation of Causal Effects of Binary Treatments in Unconfounded Studies with Dichotomous Outcomes

R. Gutman,D. B. Rubin

DOI: https://doi.org/10.1002/sim.5627

2013-01-01

Statistics in Medicine

Abstract:The estimation of causal effects has been the subject of extensive research. In unconfounded studies with a dichotomous outcome, Y, Cangul, Chretien, Gutman and Rubin (2009) demonstrated that logistic regression for a scalar continuous covariate X is generally statistically invalid for testing null treatment effects when the distributions of X in the treated and control populations differ and the logistic model for Y given X is misspecified. In addition, they showed that an approximately valid statistical test can be generally obtained by discretizing X followed by regression adjustment within each interval defined by the discretized X. This paper extends the work of Cangul et al. 2009 in three major directions. First, we consider additional estimation procedures, including a new one that is based on two independent splines and multiple imputation; second, we consider additional distributional factors; and third, we examine the performance of the procedures when the treatment effect is non‐null. Of all the methods considered and in most of the experimental conditions that were examined, our proposed new methodology appears to work best in terms of point and interval estimation. Copyright © 2012 John Wiley & Sons, Ltd.
Identification of Causal Effects Using Instrumental Variables: Comment

JD Angrist,GW Imbens,DB Rubin

DOI: https://doi.org/10.2307/2291630

IF: 4.369

1996-01-01

Journal of the American Statistical Association

Abstract:We outline a framework for causal inference in settings where assignment to a binary treatment is ignorable, but compliance with the assignment is not perfect so that the receipt of treatment is nonignorable. To address the problems associated with comparing subjects by the ignorable assignment-an intention-to-treat analysis-we make use of instrumental variables, which have long been used by economists in the context of regression models with constant treatment effects. We show that the instrumental variables (IV) estimand can be embedded within the Rubin Causal Model (RCM) and that under some simple and easily interpretable assumptions, the IV estimand is the average causal effect for a subgroup of units, the compliers. Without these assumptions, the IV estimand is simply the ratio of intention-to-treat causal estimands with no interpretation as an average causal effect. The advantages of embedding the IV approach in the RCM are that it clarifies the nature of critical assumptions needed for a causal interpretation, and moreover allows us to consider sensitivity of the results to deviations from key assumptions in a straightforward manner. We apply our analysis to estimate the effect of veteran status in the Vietnam era on mortality, using the lottery number that assigned priority for the draft as an instrument, and we use our results to investigate the sensitivity of the conclusions to critical assumptions.
Estimation in Parallel Randomized Experiments

Donald B. Rubin

DOI: https://doi.org/10.3102/10769986006004377

1981-01-01

Journal of Educational Statistics

Abstract:Many studies comparing new treatments to standard treatments consist of parallel randomized experiments. In the example considered here, randomized experiments were conducted in eight schools to determine the effectiveness of special coaching programs for the SAT. The purpose here is to illustrate Bayesian and empirical Bayesian techniques that can be used to help summarize the evidence in such data about differences among treatments, thereby obtaining improved estimates of the treatment effect in each experiment, including the one having the largest observed effect. Three main tools are illustrated: 1) graphical techniques for displaying sensitivity within an empirical Bayes framework, 2) simple simulation techniques for generating Bayesian posterior distributions of individual effects and the largest effect, and 3) methods for monitoring the adequacy of the Bayesian model specification by simulating the posterior predictive distribution in hypothetical replications of the same treatments in the same eight schools.
24 Statistical Inference for Causal Effects, with Emphasis on Applications in Psychometrics and Education

Donald B. Rubin

DOI: https://doi.org/10.1016/s0169-7161(06)26024-3

2007-01-01

Abstract:A central problem in statistics, psychometrics and education, is how to draw inferences about the causal effects of treatments (i.e., interventions) from randomized and nonrandomized data. For example, does the new job-training program really improve the quality of jobs for those trained relative to the jobs they would get without training, or does exposure to that chemical in drinking water increase cancer rates relative to drinking water without that chemical, or, of particular relevance in this volume, does that educational intervention improve the test scores of students relative to the standard program? This presentation provides an overview of the approach to the estimation of such causal effects based on the concept of potential outcomes, including a relatively detailed presentation of the Bayesian approach.
Estimation of standard errors and treatment effects in empirical economics—methods and applications

Olaf Hübler

DOI: https://doi.org/10.1007/s12651-013-0135-0

2013-07-10

Journal for Labour Market Research

Abstract:This paper discusses methodological problems of standard errors and treatment effects. First, heteroskedasticity- and cluster-robust estimates are considered as well as problems with Bernoulli distributed regressors, outliers and partially identified parameters. Second, procedures to determine treatment effects are analyzed. Four principles are in the focus: difference-in-differences estimators, matching procedures, treatment effects in quantile regression analysis and regression discontinuity approaches. These methods are applied to Cobb-Douglas functions using IAB establishment panel data.Different heteroskedasticity-consistent procedures lead to similar results of standard errors. Cluster-robust estimates show evident deviates. Dummies with a mean near 0.5 have a smaller variance of the coefficient estimates than others. Not all outliers have a strong influence on significance. New methods to handle the problem of partially identified parameters lead to more efficient estimates.The four discussed treatment procedures are applied to the question whether company-level pacts affect the output. In contrast to unconditional difference-in-differences and to estimates without matching the company-level effect is positive but insignificant if conditional difference-in-differences, nearest-neighbor or Mahalanobis metric matching is applied. The latter result has to be specified under quantile treatment effects analysis. The higher the quantile the higher is the positive company-level pact effect and there is a tendency from insignificant to significant effects. A sharp regression discontinuity analysis shows a structural break at a probability of 0.5 that a company-level pact exists. No specific effect of the Great Recession can be detected. Fuzzy regression discontinuity estimates reveal that the company-level pact effect is significantly lower in East than in West Germany.
Statistical Inference For Causal Effects, With Emphasis On Applications In Epidemiology And Medical Statistics

Donald B. Rubin

DOI: https://doi.org/10.1016/S0169-7161(07)27002-6

2008-01-01

Abstract:A central problem in epidemiology and medical statistics is how to draw inferences about the causal effects of treatments (i.e., interventions) from randomized and nonrandomized data. For example, does the new drug really reduce heart disease, or does exposure to that chemical in drinking water increase cancer rates relative to drinking water without that chemical? This chapter provides an overview of the approach to the estimation of such causal effects based on the concept of potential outcomes. We discuss randomization-based approaches and the Bayesian posterior predictive approach.
A Note on Percent Variance Explained As A Measure of the Importance of Effects

R ROSENTHAL,DB RUBIN

DOI: https://doi.org/10.1111/j.1559-1816.1979.tb02713.x

1979-01-01

Journal of Applied Social Psychology

Abstract:Journal of Applied Social PsychologyVolume 9, Issue 5 p. 395-396 A Note on Percent Variance Explained as A Measure of the Importance of Effects Robert Rosenthal, Corresponding Author Robert Rosenthal Harvard UniversityRequests for reprints should be sent to Dr. Robert Rosenthal, Department of Psychology and Social Relations, Harvard University, Cambridge, MA 02138. Order of authors was determined alphabetically.Search for more papers by this authorDonald B. Rubin, Donald B. Rubin Educational Testing ServiceSearch for more papers by this author Robert Rosenthal, Corresponding Author Robert Rosenthal Harvard UniversityRequests for reprints should be sent to Dr. Robert Rosenthal, Department of Psychology and Social Relations, Harvard University, Cambridge, MA 02138. Order of authors was determined alphabetically.Search for more papers by this authorDonald B. Rubin, Donald B. Rubin Educational Testing ServiceSearch for more papers by this author First published: October 1979 https://doi.org/10.1111/j.1559-1816.1979.tb02713.xCitations: 141AboutPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL No abstract is available for this article. References Mayo, R. J. Statistical considerations in analyzing the results of a collection of experiments. The Behavioral and Brain Sciences, 1978, 3, 400– 401. Citing Literature Volume9, Issue5October 1979Pages 395-396 ReferencesRelatedInformation
Transporting stochastic direct and indirect effects to new populations

Kara E. Rudolph,Jonathan Levy,Mark J. Laan

DOI: https://doi.org/10.1111/biom.13274

IF: 1.701

2020-05-04

Biometrics

Abstract:<p>Transported mediation effects may contribute to understanding how interventions work differently when applied to new populations. However, we are not aware of any estimators for such effects. Thus, we propose two doubly robust, efficient estimators of transported stochastic (also called randomized interventional) direct and indirect effects. We demonstrate their finite sample properties in a simulation study. We then apply the preferred substitution estimator to longitudinal data from the Moving to Opportunity Study, a large‐scale housing voucher experiment, to transport stochastic indirect effect estimates of voucher receipt in childhood on subsequent risk of mental health or substance use disorder mediated through parental employment across sites, thereby gaining understanding of drivers of the site differences.</p><p>This article is protected by copyright. All rights reserved</p>

statistics & probability,mathematical & computational biology,biology
Estimation of the Number Needed to Treat, the Number Needed to be Exposed, and the Exposure Impact Number with Instrumental Variables

Valentin Vancak,Arvid Sjölander

DOI: https://doi.org/10.48550/arXiv.2307.09319

2024-08-14

Abstract:The Number needed to treat (NNT) is an efficacy index defined as the average number of patients needed to treat to attain one additional treatment benefit. In observational studies, specifically in epidemiology, the adequacy of the populationwise NNT is questionable since the exposed group characteristics may substantially differ from the unexposed. To address this issue, groupwise efficacy indices were defined: the Exposure Impact Number (EIN) for the exposed group and the Number Needed to be Exposed (NNE) for the unexposed. Each defined index answers a unique research question since it targets a unique sub-population. In observational studies, the group allocation is typically affected by confounders that might be unmeasured. The available estimation methods that rely either on randomization or the sufficiency of the measured covariates for confounding control will result in inconsistent estimators of the true NNT (EIN, NNE) in such settings. Using Rubin's potential outcomes framework, we explicitly define the NNT and its derived indices as causal contrasts. Next, we introduce a novel method that uses instrumental variables to estimate the three aforementioned indices in observational studies. We present two analytical examples and a corresponding simulation study. The simulation study illustrates that the novel estimators are statistically consistent, unlike the previously available methods, and their analytical confidence intervals' empirical coverage rates converge to their nominal values. Finally, a real-world data example of an analysis of the effect of vitamin D deficiency on the mortality rate is presented.

Methodology
On Estimating the Causal Effects of DNR Orders

Martin W. McIntosh,Donald B. Rubin

DOI: https://doi.org/10.1097/00005650-199908000-00002

1999-01-01

Medical Care

Abstract:Department of Biostatistics; University of Washington and Cancer Prevention; Research Program, Fred Hutchinson Cancer; Research Center; Seattle, Washington (McIntosh) Department of Statistics, Harvard University; Cambridge, Massachusetts (Rubin) Supported by National Cancer Institute grant NCI 5R01 CA63146-05 (Dr. McIntosh) and the National Science Foundation grant NSF SBR 9709359 (Dr. Rubin). Address correspondence to: Martin W. McIntosh, PhD, Fred Hutchinson Cancer Research Center, Cancer Prevention Research Program, 1100 Fairview Avenue, MP-804, P.O. Box 19024, Seattle, Washington, 98109-1024.
Estimating Outcome Distributions for Compliers in Instrumental Variables Models

GW Imbens,DB Rubin

DOI: https://doi.org/10.2307/2971731

1997-01-01

Abstract:In Imbens and Ingrist (1994), Angrist, Imbens and Rubin (1996) and Imbens and Rubin (1997), assumptions have been outlined under which instrumental variables estimands can be given a causal interpretation as a local average treatment effect without requiring functional form or constant treatment effect assumptions. We extend these results by showing that under these assumptions one can estimate more from the data than the average causal effect for the subpopulation of compliers; one can, in principle, estimate the entire marginal distribution of the outcome under different treatments for this subpopulation. These distributions might be useful for a policy maker who wishes to take into account not only differences in average of earnings when contemplating the merits of one job training programme vs. another. We also show that the standard instrumental variables estimator implicitly estimates these underlying outcome distributions without imposing the required nonnegativity on these implicit density estimates, and that imposing nonnegativity can substantially alter the estimates of the local average treatment effect. We illustrate these points by presenting an analysis of the returns-to a high school education using quarter of birth as an instrument. We show that the standard instrumental variables estimates implicitly estimate the outcome distributions to be negative over a substantial range, and that the estimates of the local average treatment effect change considerably when we impose nonnegativity in any of a variety of ways.
A practical guide to estimating treatment effects in patients with rheumatic diseases using real-world data

Are Hugo Pripp,Katarzyna Łosińska,Mariusz Korkosz,Glenn Haugeberg

DOI: https://doi.org/10.1007/s00296-024-05597-2

2024-04-24

Rheumatology International

Abstract:Abstract Objective Randomized controlled trials are considered the gold standard in study methodology. However, due to their study design and inclusion criteria, these studies may not capture the heterogeneity of real-world patient populations. In contrast, the lack of randomization and the presence of both measured and unmeasured confounding factors could bias the estimated treatment effect when using observational data. While causal inference methods allow for the estimation of treatment effects, their mathematical complexity may hinder their application in clinical research. Methods We present a practical, nontechnical guide using a common statistical package (Stata) and a motivational simulated dataset that mirrors real-world observational data from patients with rheumatic diseases. We demonstrate regression analysis, regression adjustment, inverse-probability weighting, propensity score (PS) matching and two robust estimation methods. Results Although the methods applied to control for confounding factors produced similar results, the commonly used one-to-one PS matching method could yield biased results if not thoroughly assessed. Conclusion The guide we propose aims to facilitate the use of readily available methods in a common statistical package. It may contribute to robust and transparent epidemiological and statistical methods, thereby enhancing effectiveness research using observational data in rheumatology.

rheumatology
Matching Estimators for Causal Effects of Multiple Treatments

Anthony D. Scotina,Francesca L. Beaudoin,Roee Gutman

DOI: https://doi.org/10.48550/arXiv.1809.00268

2019-04-26

Abstract:Matching estimators for average treatment effects are widely used in the binary treatment setting, in which missing potential outcomes are imputed as the average of observed outcomes of all matches for each unit. With more than two treatment groups, however, estimation using matching requires additional techniques. In this paper, we propose a nearest-neighbors matching estimator for use with multiple, nominal treatments, and use simulations to show that this method is precise and has coverage levels that are close to nominal. In addition, we implement the proposed inference methods to examine the effects of different medication regimens on long-term pain for patients experiencing motor vehicle collision.

Methodology
Estimation of Causal Effects of Multiple Treatments in Observational Studies with a Binary Outcome

Liangyuan Hu,Chenyang Gu,Michael Lopez,Jiayi Ji,Juan Wisnivesky

DOI: https://doi.org/10.48550/arXiv.2001.06483

2020-01-17

Abstract:There is a dearth of robust methods to estimate the causal effects of multiple treatments when the outcome is binary. This paper uses two unique sets of simulations to propose and evaluate the use of Bayesian Additive Regression Trees (BART) in such settings. First, we compare BART to several approaches that have been proposed for continuous outcomes, including inverse probability of treatment weighting (IPTW), targeted maximum likelihood estimator (TMLE), vector matching and regression adjustment. Results suggest that under conditions of non-linearity and non-additivity of both the treatment assignment and outcome generating mechanisms, BART, TMLE and IPTW using generalized boosted models (GBM) provide better bias reduction and smaller root mean squared error. BART and TMLE provide more consistent 95 per cent CI coverage and better large-sample convergence property. Second, we supply BART with a strategy to identify a common support region for retaining inferential units and for avoiding extrapolating over areas of the covariate space where common support does not exist. BART retains more inferential units than the generalized propensity score based strategy, and shows lower bias, compared to TMLE or GBM, in a variety of scenarios differing by the degree of covariate overlap. A case study examining the effects of three surgical approaches for non-small cell lung cancer demonstrates the methods.

Methodology,Applications
Estimation of boundary of average treatment effects in clinical trials with noncompliance

Hong-xiang LIU,Zhi GEN,Hua CHEN

DOI: https://doi.org/10.3969/j.issn.1000-2375.2009.01.006

2009-01-01

Abstract:Noncompliance or partial compliance often occurs in a clinical trial.In this case,average treatment effects,or called average causal effect(ACE) of treatment,may not easily estimated because there are too many potential variables which can not be simultaneously observed.Balke and Pearl have obtained the upper and lower boundary of ACE under the assumption of non-compliance or total compliance.By their method,lower boundary may sometimes be negative,which is abnormal.This paper proposes a method to calculate estimates of the upper and lower boundary of ACE under the assumptions in Angrist,Imbebns and Rubin,and it shows that the lower boundary must be positive and that the interval between the lower and upper boundary is shorter than Balke and Pearl's.The paper has also discussed estimation of upper and lower boundary of ACE under partial compliance.

Comment: Estimating the Effects Caused by Treatments

Estimating causal effects of treatments in experimental and observational studies

Estimating Causal Effects of Treatments in Randomized and Nonrandomized Studies

Estimating Average Treatment Effects: Supplementary Analyses and Remaining Challenges

THE ASSESSMENT OF A TREATMENT EFFECT WHEN A COVARIATE IS OBSERVED ON A SUBSET OF THE UNITS: I. SIMPLE ESTIMATORS

Treatment Effect Measures Under Nonproportional Hazards

Robust Estimation of Causal Effects of Binary Treatments in Unconfounded Studies with Dichotomous Outcomes

Identification of Causal Effects Using Instrumental Variables: Comment

Estimation in Parallel Randomized Experiments

24 Statistical Inference for Causal Effects, with Emphasis on Applications in Psychometrics and Education

Estimation of standard errors and treatment effects in empirical economics—methods and applications

Statistical Inference For Causal Effects, With Emphasis On Applications In Epidemiology And Medical Statistics

A Note on Percent Variance Explained As A Measure of the Importance of Effects

Transporting stochastic direct and indirect effects to new populations

Estimation of the Number Needed to Treat, the Number Needed to be Exposed, and the Exposure Impact Number with Instrumental Variables

On Estimating the Causal Effects of DNR Orders

Estimating Outcome Distributions for Compliers in Instrumental Variables Models

A practical guide to estimating treatment effects in patients with rheumatic diseases using real-world data

Matching Estimators for Causal Effects of Multiple Treatments

Estimation of Causal Effects of Multiple Treatments in Observational Studies with a Binary Outcome

Estimation of boundary of average treatment effects in clinical trials with noncompliance