[Malignant epithelial tumors of the oral cavity (II)].

H. Reychler

1984-06-13

Abstract:

What problem does this paper attempt to address?

Matching Algorithms for Causal Inference with Multiple Treatments

Anthony D. Scotina,Roee Gutman

DOI: https://doi.org/10.48550/arXiv.1809.00269

2019-02-27

Abstract:Randomized clinical trials (RCTs) are ideal for estimating causal effects, because the distributions of background covariates are similar in expectation across treatment groups. When estimating causal effects using observational data, matching is a commonly used method to replicate the covariate balance achieved in a RCT. Matching algorithms have a rich history dating back to the mid-1900s, but have been used mostly to estimate causal effects between two treatment groups. When there are more than two treatments, estimating causal effects requires additional assumptions and techniques. We propose matching algorithms that address the drawbacks of the current methods, and we use simulations to compare current and new methods. All of the methods display improved covariate balance in the matched sets relative to the pre-matched cohorts. In addition, we provide advice to investigators on which matching algorithms are preferred for different covariate distributions.

Methodology
Distinguishing Between Mediators and Confounders is Important for the Causal Inference in Observational Studies

Zhongheng Zhang

DOI: https://doi.org/10.21037/amj.2019.09.03

2019-01-01

AME Medical Journal

Abstract:Observational studies are common in clinical literature because they are less costly and easy to perform, as compared to the randomized controlled trials (RCT). The limitation of such kind of studies is their inability to draw a causal inference due to potential bias. However, there are many approaches being proposed in recent years for the causal inference in observational studies (1-3). Although these methods cannot replace RCTs for causal inference, analysis in their framework can help to make the causal evidence more reliable. In the following discuss, I would like to discuss some of these points by using a published article.
The covariate-adjusted residual estimator and its use in both randomized trials and observational settings

Stephen A. Lauer,Nicholas G. Reich,Laura B. Balzer

DOI: https://doi.org/10.48550/arXiv.1910.11397

2019-10-25

Abstract:We often seek to estimate the causal effect of an exposure on a particular outcome in both randomized and observational settings. One such estimation method is the covariate-adjusted residuals estimator, which was designed for individually or cluster randomized trials. In this manuscript, we study the properties of this estimator and develop a new estimator that utilizes both covariate adjustment and inverse probability weighting We support our theoretical results with a simulation study and an application in an infectious disease setting. The covariate-adjusted residuals estimator is an efficient and unbiased estimator of the average treatment effect in randomized trials; however, it is not guaranteed to be unbiased in observational studies. Our novel estimator, the covariate-adjusted residuals estimator with inverse probability weighting, is unbiased in randomized and observational settings, under a reasonable set of assumptions. Furthermore, when these assumptions hold, it provides efficiency gains over inverse probability weighting in observational studies. The covariate-adjusted residuals estimator is valid for use in randomized trials, but should not be used in observational studies. The covariate-adjusted residuals estimator with inverse probability weighting provides an efficient alternative for use in randomized and observational settings.

Methodology,Applications
RCT Rejection Sampling for Causal Estimation Evaluation

Katherine A. Keith,Sergey Feldman,David Jurgens,Jonathan Bragg,Rohit Bhattacharya

2024-01-31

Abstract:Confounding is a significant obstacle to unbiased estimation of causal effects from observational data. For settings with high-dimensional covariates -- such as text data, genomics, or the behavioral social sciences -- researchers have proposed methods to adjust for confounding by adapting machine learning methods to the goal of causal estimation. However, empirical evaluation of these adjustment methods has been challenging and limited. In this work, we build on a promising empirical evaluation strategy that simplifies evaluation design and uses real data: subsampling randomized controlled trials (RCTs) to create confounded observational datasets while using the average causal effects from the RCTs as ground-truth. We contribute a new sampling algorithm, which we call RCT rejection sampling, and provide theoretical guarantees that causal identification holds in the observational data to allow for valid comparisons to the ground-truth RCT. Using synthetic data, we show our algorithm indeed results in low bias when oracle estimators are evaluated on the confounded samples, which is not always the case for a previously proposed algorithm. In addition to this identification result, we highlight several finite data considerations for evaluation designers who plan to use RCT rejection sampling on their own datasets. As a proof of concept, we implement an example evaluation pipeline and walk through these finite data considerations with a novel, real-world RCT -- which we release publicly -- consisting of approximately 70k observations and text data as high-dimensional covariates. Together, these contributions build towards a broader agenda of improved empirical evaluation for causal estimation.

Artificial Intelligence,Computation and Language,Machine Learning,Methodology
Applying the estimand and target trial frameworks to external control analyses using observational data: a case study in the solid tumor setting

Letizia Polito,Qixing Liang,Navdeep Pal,Philani Mpofu,Ahmed Sawas,Olivier Humblet,Kaspar Rufibach,Dominik Heinzmann

DOI: https://doi.org/10.3389/fphar.2024.1223858

IF: 5.6

2024-01-26

Frontiers in Pharmacology

Abstract:Introduction: In causal inference, the correct formulation of the scientific question of interest is a crucial step. The purpose of this study was to apply causal inference principles to external control analysis using observational data and illustrate the process to define the estimand attributes. Methods: This study compared long-term survival outcomes of a pooled set of three previously reported randomized phase 3 trials studying patients with metastatic non-small cell lung cancer receiving front-line chemotherapy and similar patients treated with front-line chemotherapy as part of routine clinical care. Causal inference frameworks were applied to define the estimand aligned with the research question and select the estimator to estimate the estimand of interest. Results: The estimand attributes of the ideal trial were defined using the estimand framework. The target trial framework was used to address specific issues in defining the estimand attributes using observational data from a nationwide electronic health record-derived de-identified database. The two frameworks combined allow to clearly define the estimand and the aligned estimator while accounting for key baseline confounders, index date, and receipt of subsequent therapies. The hazard ratio estimate (point estimate with 95% confidence interval) comparing the randomized clinical trial pooled control arm with the external control was close to 1, which is indicative of similar survival between the two arms. Discussion: The proposed combined framework provides clarity on the causal contrast of interest and the estimator to adopt, and thus facilitates design and interpretation of the analyses.

pharmacology & pharmacy
A meta-analytic framework to adjust for bias in external control studies

Devin Incerti,Michael T Bretscher,Ray Lin,Chris Harbron

DOI: https://doi.org/10.48550/arXiv.2110.03827

2021-10-08

Abstract:While randomized controlled trials (RCTs) are the gold standard for estimating treatment effects in medical research, there is increasing use of and interest in using real-world data for drug development. One such use case is the construction of external control arms for evaluation of efficacy in single-arm trials, particularly in cases where randomization is either infeasible or unethical. However, it is well known that treated patients in non-randomized studies may not be comparable to control patients -- on either measured or unmeasured variables -- and that the underlying population differences between the two groups may result in biased treatment effect estimates as well as increased variability in estimation. To address these challenges for analyses of time-to-event outcomes, we developed a meta-analytic framework that uses historical reference studies to adjust a log hazard ratio estimate in a new external control study for its additional bias and variability. The set of historical studies is formed by constructing external control arms for historical RCTs, and a meta-analysis compares the trial controls to the external control arms. Importantly, a prospective external control study can be performed independently of the meta-analysis using standard causal inference techniques for observational data. We illustrate our approach with a simulation study and an empirical example based on reference studies for advanced non-small cell lung cancer. In our empirical analysis, external control patients had lower survival than trial controls (hazard ratio: 0.907), but our methodology is able to correct for this bias. An implementation of our approach is available in the R package ecmeta.

Methodology,Applications
Causal Inference With Two Versions of Treatment

Raiden B. Hasegawa,Sameer K. Deshpande,Dylan S. Small,Paul R. Rosenbaum

DOI: https://doi.org/10.3102/1076998620914003

2020-03-31

Journal of Educational and Behavioral Statistics

Abstract:Causal effects are commonly defined as comparisons of the potential outcomes under treatment and control, but this definition is threatened by the possibility that either the treatment or the control condition is not well defined, existing instead in more than one version. This is often a real possibility in nonexperimental or observational studies of treatments because these treatments occur in the natural or social world without the laboratory control needed to ensure identically the same treatment or control condition occurs in every instance. We consider the simplest case: Either the treatment condition or the control condition exists in two versions that are easily recognized in the data but are of uncertain, perhaps doubtful, relevance, for example, branded Advil versus generic ibuprofen. Common practice does not address versions of treatment: Typically, the issue is either ignored or explicitly stated but assumed to be absent. Common practice is reluctant to address two versions of treatment because the obvious solution entails dividing the data into two parts with two analyses, thereby (a) reducing power to detect versions of treatment in each part, (b) creating problems of multiple inference in coordinating the two analyses, and (c) failing to report a single primary analysis that uses everyone. We propose and illustrate a new method of analysis that begins with a single primary analysis of everyone that would be correct if the two versions do not differ, adds a second analysis that would be correct were there two different effects for the two versions, controls the family-wise error rate in all assertions made by the several analyses, and yet pays no price in power to detect a constant treatment effect in the primary analysis of everyone. Our method can be applied to analyses of constant additive treatment effects on continuous outcomes. Unlike conventional simultaneous inferences, the new method is coordinating several analyses that are valid under different assumptions, so that one analysis would never be performed if one knew for certain that the assumptions of the other analysis are true. It is a multiple assumptions problem rather than a multiple hypotheses problem. We discuss the relative merits of the method with respect to more conventional approaches to analyzing multiple comparisons. The method is motivated and illustrated using a study of the possibility that repeated head trauma in high school football causes an increase in risk of early onset cognitive decline.

education & educational research,psychology, mathematical,social sciences, mathematical methods
Causal inference with observational data in addiction research

Gary C. K. Chan,Carmen Lim,Tianze Sun,Daniel Stjepanovic,Jason Connor,Wayne Hall,Janni Leung

DOI: https://doi.org/10.1111/add.15972

2022-06-08

Addiction

Abstract:Randomised controlled trials (RCTs) are the gold standard for making causal inferences, but RCTs are often not feasible in addiction research for ethical and logistic reasons. Observational data from real world settings have been increasingly used to guide clinical decisions and public health policies. This paper introduces the potential outcomes framework for causal inference and summarises well‐established causal analysis methods for observational data, including matching, inverse probability treatment weighting, the instrumental variable method, and interrupted time series analysis with controls. It provides examples in addiction research and guidance and analysis codes for conducting these analyses with example datasets.

psychiatry,substance abuse
Matching within a hybrid RCT/RWD: framework on associated causal estimands

Junjing Lin,Guanglei Yu,Margaret Gamalo

DOI: https://doi.org/10.1080/10543406.2022.2105346

2023-07-04

Abstract:As the regulatory environment becomes progressively receptive toward utilizing real-world evidence, a spectrum of real-world data incorporation techniques in trial conduct and analysis has seen increasing interest and adoption in different stages of drug development. Of particular interest is leveraging external control data to augment the control arm in a concurrent randomized controlled trial, where patients are enrolled in both investigational treatment arm and the control arm. Yet despite the emerging literature in external data borrowing in a hybrid trial setting, very little discussion focuses on delineating what should be matched and what is actually being estimated, especially when a variety of matching schemes can be considered. In general, external control can be matched in four different ways: (1) matching with the intersection between investigational treatment and concurrent control, (2) matching with the union of concurrent investigational treatment and concurrent control, (3) matching with concurrent control alone, and (4) matching with investigational treatment alone. In this article, the formulation of estimands for different matching schemes are detailed to describe what these matching methods facilitate to answer. Simulation studies are also conducted to evaluate the performance characteristics under different matching schemes, estimation methods, effect size assumptions, and missingness of confounders.
Causal inference methods for combining randomized trials and observational studies: a review

Bénédicte Colnet,Imke Mayer,Guanhua Chen,Awa Dieng,Ruohong Li,Gaël Varoquaux,Jean-Philippe Vert,Julie Josse,Shu Yang

DOI: https://doi.org/10.48550/arXiv.2011.08047

2023-01-10

Abstract:With increasing data availability, causal effects can be evaluated across different data sets, both randomized controlled trials (RCTs) and observational studies. RCTs isolate the effect of the treatment from that of unwanted (confounding) co-occurring effects but they may suffer from unrepresentativeness, and thus lack external validity. On the other hand, large observational samples are often more representative of the target population but can conflate confounding effects with the treatment of interest. In this paper, we review the growing literature on methods for causal inference on combined RCTs and observational studies, striving for the best of both worlds. We first discuss identification and estimation methods that improve generalizability of RCTs using the representativeness of observational data. Classical estimators include weighting, difference between conditional outcome models, and doubly robust estimators. We then discuss methods that combine RCTs and observational data to either ensure uncounfoundedness of the observational analysis or to improve (conditional) average treatment effect estimation. We also connect and contrast works developed in both the potential outcomes literature and the structural causal model literature. Finally, we compare the main methods using a simulation study and real world data to analyze the effect of tranexamic acid on the mortality rate in major trauma patients. A review of available codes and new implementations is also provided.

Methodology
An Evaluation Toolkit to Guide Model Selection and Cohort Definition in Causal Inference

Yishai Shimoni,Ehud Karavani,Sivan Ravid,Peter Bak,Tan Hung Ng,Sharon Hensley Alford,Denise Meade,Yaara Goldschmidt

DOI: https://doi.org/10.48550/arXiv.1906.00442

2019-06-03

Abstract:Real world observational data, together with causal inference, allow the estimation of causal effects when randomized controlled trials are not available. To be accepted into practice, such predictive models must be validated for the dataset at hand, and thus require a comprehensive evaluation toolkit, as introduced here. Since effect estimation cannot be evaluated directly, we turn to evaluating the various observable properties of causal inference, namely the observed outcome and treatment assignment. We developed a toolkit that expands established machine learning evaluation methods and adds several causal-specific ones. Evaluations can be applied in cross-validation, in a train-test scheme, or on the training data. Multiple causal inference methods are implemented within the toolkit in a way that allows modular use of the underlying machine learning models. Thus, the toolkit is agnostic to the machine learning model that is used. We showcase our approach using a rheumatoid arthritis cohort (consisting of about 120K patients) extracted from the IBM MarketScan(R) Research Database. We introduce an iterative pipeline of data definition, model definition, and model evaluation. Using this pipeline, we demonstrate how each of the evaluation components helps drive model selection and refinement of data extraction criteria in a way that provides more reproducible results and ensures that the causal question is answerable with available data. Furthermore, we show how the evaluation toolkit can be used to ensure that performance is maintained when applied to subsets of the data, thus allowing exploration of questions that move towards personalized medicine.

Machine Learning
genRCT: a statistical analysis framework for generalizing RCT findings to real-world population

Dasom Lee,Shu Yang,Mark Berry,Tom Stinchcombe,Harvey Jay Cohen,Xiaofei Wang

DOI: https://doi.org/10.1080/10543406.2024.2333136

2024-04-10

Journal of Biopharmaceutical Statistics

Abstract:When evaluating the real-world treatment effect, the analysis based on randomized clinical trials (RCTs) often introduces generalizability bias due to the difference in risk factors between the trial participants and the real-world patient population. This problem of lack of generalizability associated with the RCT-only analysis can be addressed by leveraging observational studies with large sample sizes that are representative of the real-world population. A set of novel statistical methods, termed "genRCT", for improving the generalizability of the trial has been developed using calibration weighting, which enforces the covariates balance between the RCT and observational study. This paper aims to review statistical methods for generalizing the RCT findings by harnessing information from large observational studies that represent real-world patients. Specifically, we discuss the choices of data sources and variables to meet key theoretical assumptions and principles. We introduce and compare estimation methods for continuous, binary, and survival endpoints. We showcase the use of the R package genRCT through a case study that estimates the average treatment effect of adjuvant chemotherapy for the stage 1B non-small cell lung patients represented by a large cancer registry.

pharmacology & pharmacy,statistics & probability
Estimating treatment effect in randomized trial after control to treatment crossover using external controls

Xiner Zhou,Herbert Pang,Christiana Drake,Hans Ulrich Burger,Jiawen Zhu

DOI: https://doi.org/10.1080/10543406.2024.2330209

2024-04-03

Journal of Biopharmaceutical Statistics

Abstract:In clinical trials, it is common to design a study that permits the administration of an experimental treatment to participants in the placebo or standard of care group post primary endpoint. This is often seen in the open-label extension phase of a phase III, pivotal study of the new medicine, where the focus is on assessing long-term safety and efficacy. With the availability of external controls, proper estimation and inference of long-term treatment effect during the open-label extension phase in the absence of placebo-controlled patients are now feasible. Within the framework of causal inference, we propose several difference-in-differences (DID) type methods and a synthetic control method (SCM) for the combination of randomized controlled trials and external controls. Our realistic simulation studies demonstrate the desirable performance of the proposed estimators in a variety of practical scenarios. In particular, DID methods outperform SCM and are the recommended methods of choice. An empirical application of the methods is demonstrated through a phase III clinical trial in rare disease.

pharmacology & pharmacy,statistics & probability
Multiple-model-based robust estimation of causal treatment effect on a binary outcome with integrated information from secondary outcomes

Chixiang Chen,Shuo Chen,Qi Long,Sudeshna Das,Ming Wang

DOI: https://doi.org/10.1080/00031305.2023.2250399

2023-08-21

The American Statistician

Abstract:An assessment of the causal treatment effect in the development and progression of certain diseases is important in clinical trials and biomedical studies. However, it is not possible to infer a causal relationship when the treatment assignment is imbalanced and confounded by other mechanisms. Specifically, when the treatment assignment is not randomized and the primary outcome is binary, a conventional logistic regression may not be valid to elucidate any causal inference. Moreover, exclusively capturing all confounders is extremely difficult and even impossible in large-scale observational studies. We propose a multiple-model-based robust (MultiMR) estimator for estimating the causal effect with a binary outcome, where multiple propensity score models and conditional mean imputation models are used to ensure estimation robustness. Furthermore, we propose an enhanced MultiMR (eMultiMR) estimator that reduces the estimation variability of MultiMR estimates by incorporating secondary outcomes that are highly correlated with the primary binary outcome. The resulting estimates are less sensitive to model mis-specification compared to those based on state-of-the-art doubly-robust methods. These estimates are verified through both theoretical and numerical assessments. The utility of (e)MultiMR estimation is illustrated using the Uniform Data Set (UDS) from the National Alzheimer’s Coordinating Center with the objective of detecting the causal effect of the short-term use of antihypertensive medications on the development of dementia or mild cognitive impairment. The proposed method has been implemented in an R package and is available at https://github.com/chencxxy28/eMultiMR.

statistics & probability
A Two-Stage Method for Extending Inferences from a Collection of Trials

Nicole Schnitzler,Eloise Kaizar

DOI: https://doi.org/10.48550/arXiv.2309.03693

2023-09-07

Methodology

Abstract:When considering the effect a treatment will cause in a population of interest, we often look to evidence from randomized controlled trials. In settings where multiple trials on a treatment are available, we may wish to synthesize the trials' participant data to obtain causally interpretable estimates of the average treatment effect in a specific target population. Traditional meta-analytic approaches to synthesizing data from multiple studies estimate the average effect among the studies. The resulting estimate is often not causally interpretable in any population, much less a particular population of interest, due to heterogeneity in the effect of treatment across studies. Inspired by traditional two-stage meta-analytic methods, as well as methods for extending inferences from a single study, we propose a two-stage approach to extending inferences from a collection of randomized controlled trials that can be used to obtain causally interpretable estimates of treatment effects in a target population when there is between-study heterogeneity in conditional average treatment effects. We first introduce a collection of assumptions under which the target population's average treatment effect is identifiable when conditional average treatment effects are heterogeneous across studies. We then introduce an estimator that utilizes weighting in two stages, taking a weighted average of study-specific estimates of the treatment effect in the target population. The performance of our proposed approach is assessed through simulation studies and an application to a collection of trials studying an online therapy treatment for symptoms of pediatric traumatic brain injury.
Series 2-19-2009 Multiple Imputation Methods for Treatment Noncompliance and Nonresponse in Randomized Clinical Trials

L. Taylor,X. H. Zhou

2017-01-01

Abstract:This working paper is hosted by The Berkeley Electronic Press (bepress) and may not be commercially reproduced without the permission of the copyright holder. Summary. Randomized clinical trials are a powerful tool for investigating causal treatment effects, but in human trials there are oftentimes problems of noncompliance which standard analyses, such as the intention-to-treat or as-treated analysis, either ignore or incorporate in such a way that the resulting estimand is no longer a causal effect. One alternative to these analyses is the complier average causal effect (CACE) which estimates the average causal treatment effect among a subpopulation that would comply under any treatment assigned. We focus on the setting of a randomized clinical trial with crossover treatment noncompliance (e.g., control subjects could receive the intervention and intervention subjects could receive the control) and outcome nonresponse. In this article, we develop estimators for the CACE using multiple imputation methods, which have been successfully applied to a wide variety of missing data problems, but have not yet been applied to the potential outcomes setting of causal inference. Using simulated data we investigate the finite sample properties of these estimators as well as of competing procedures in a simple setting. Finally we illustrate our methods using a real randomized encouragement design study on the effectiveness of the influenza vaccine. 1. Introduction The focus of empirical studies in medicine is often to estimate the causal effect of treatments, where randomized clinical trials are considered the most acceptable tool for investigating these causal relationships. But in trials involving human subjects there are oftentimes problems of patient noncompliance, where the patient does not adhere to the treatment assigned. In addition, there is also the problem of nonignorable missing data, where the missing-data mechanism may depend on unobserved data. Standard methods and analyses either ignore these complications, which can lead to biased estimates of causal treatment effect, or account for them in such a way that the resulting estimand can no longer be considered a causal effect of treatment. A statistical framework for causal inference that deals with the issue of noncompliance is based on potential outcomes and was first introduced by Neyman (1923) for randomized studies and later developed by Rubin (1974, 1978) for nonrandomized studies and other forms of inference. Rubin's approach, sometimes referred to as the Ru-bin Causal Model (Holland, 1986), provides a framework for defining the parameters of interest and correctly attributing the …
A standardized framework for risk-based assessment of treatment effect heterogeneity in observational healthcare databases

Alexandros Rekkas,David van Klaveren,Patrick B. Ryan,Ewout W. Steyerberg,David M. Kent,Peter R. Rijnbeek

DOI: https://doi.org/10.48550/arXiv.2010.06430

2022-07-01

Abstract:The Predictive Approaches to Treatment Effect Heterogeneity statement focused on baseline risk as a robust predictor of treatment effect and provided guidance on risk-based assessment of treatment effect heterogeneity in the RCT setting. The aim of this study was to extend this approach to the observational setting using a standardized scalable framework. The proposed framework consists of five steps: 1) definition of the research aim, i.e., the population, the treatment, the comparator and the outcome(s) of interest; 2) identification of relevant databases; 3) development of a prediction model for the outcome(s) of interest; 4) estimation of relative and absolute treatment effect within strata of predicted risk, after adjusting for observed confounding; 5) presentation of the results. We demonstrate our framework by evaluating heterogeneity of the effect of angiotensin-converting enzyme (ACE) inhibitors versus beta blockers on three efficacy and six safety outcomes across three observational databases. The proposed framework can supplement any comparative effectiveness study. We provide a publicly available R software package for applying this framework to any database mapped to the Observational Medical Outcomes Partnership Common Data Model. In our demonstration, patients at low risk of acute myocardial infarction received negligible absolute benefits for all three efficacy outcomes, though they were more pronounced in the highest risk quarter, especially for hospitalization with heart failure. However, failing diagnostics showed evidence of residual imbalances even after adjustment for observed confounding. Our framework allows for the evaluation of differential treatment effects across risk strata, which offers the opportunity to consider the benefit-harm trade-off between alternative treatments.

Methodology,Machine Learning
Enhancing Statistical Validity and Power in Hybrid Controlled Trials: A Randomization Inference Approach with Conformal Selective Borrowing

Ke Zhu,Shu Yang,Xiaofei Wang

DOI: https://doi.org/10.48550/arXiv.2410.11713

2024-11-14

Abstract:Randomized controlled trials (RCTs) are the gold standard for causal inference on treatment effects. However, they can be underpowered due to small population sizes in rare diseases and limited number of patients willing to participate due to questions regarding equipoise among treatment groups in common diseases. Hybrid controlled trials use external controls (ECs) from historical studies or large observational databases to enhance statistical efficiency, and as such, are considered for drug evaluation in rare diseases or indications associated with low trial participation. However, non-randomized ECs can introduce biases that compromise validity and inflate type I errors for treatment discovery, particularly in small samples. To address this, we extend the Fisher randomization test (FRT) to hybrid controlled trials. Our approach involves a test statistic combining RCT and EC data and is based solely on randomization in the RCT. This method strictly controls the type I error rate, even with biased ECs, and improves power by incorporating unbiased ECs. To mitigate the power loss caused by biased ECs, we introduce conformal selective borrowing, which uses conformal p-values to individually select unbiased ECs, offering the flexibility to use either computationally efficient parametric models or off-the-shelf machine learning models to construct the conformal score function, along with model-agnostic reliability. We identify a risk-benefit trade-off in the power of FRT associated with different selection thresholds for conformal p-values. We offer both fixed and adaptive selection threshold options, enabling robust performance across varying levels of hidden bias. The advantages of our method are demonstrated through simulations and an application to a small lung cancer RCT with ECs from the National Cancer Database. Our method is available in the R package intFRT on GitHub.

Methodology
Detecting critical treatment effect bias in small subgroups

Piersilvio De Bartolomeis,Javier Abad,Konstantin Donhauser,Fanny Yang

2024-04-30

Abstract:Randomized trials are considered the gold standard for making informed decisions in medicine, yet they often lack generalizability to the patient populations in clinical practice. Observational studies, on the other hand, cover a broader patient population but are prone to various biases. Thus, before using an observational study for decision-making, it is crucial to benchmark its treatment effect estimates against those derived from a randomized trial. We propose a novel strategy to benchmark observational studies beyond the average treatment effect. First, we design a statistical test for the null hypothesis that the treatment effects estimated from the two studies, conditioned on a set of relevant features, differ up to some tolerance. We then estimate an asymptotically valid lower bound on the maximum bias strength for any subgroup in the observational study. Finally, we validate our benchmarking strategy in a real-world setting and show that it leads to conclusions that align with established medical knowledge.

Methodology,Machine Learning
Double-robust and efficient methods for estimating the causal effects of a binary treatment

James Robins,Mariela Sued,Quanhong Lei-Gomez,Andrea Rotnitzky

DOI: https://doi.org/10.48550/arXiv.2008.00507

2020-08-02

Methodology

Abstract:We consider the problem of estimating the effects of a binary treatment on a continuous outcome of interest from observational data in the absence of confounding by unmeasured factors. We provide a new estimator of the population average treatment effect (ATE) based on the difference of novel double-robust (DR) estimators of the treatment-specific outcome means. We compare our new estimator with previously estimators both theoretically and via simulation. DR-difference estimators may have poor finite sample behavior when the estimated propensity scores in the treated and untreated do not overlap. We therefore propose an alternative approach, which can be used even in this unfavorable setting, based on locally efficient double-robust estimation of a semiparametric regression model for the modification on an additive scale of the magnitude of the treatment effect by the baseline covariates $X$. In contrast with existing methods, our approach simultaneously provides estimates of: i) the average treatment effect in the total study population, ii) the average treatment effect in the random subset of the population with overlapping estimated propensity scores, and iii) the treatment effect at each level of the baseline covariates $X$. When the covariate vector $X$ is high dimensional, one cannot be certain, owing to lack of power, that given models for the propensity score and for the regression of the outcome on treatment and $X$ used in constructing our DR estimators are nearly correct, even if they pass standard goodness of fit tests. Therefore to select among candidate models, we propose a novel approach to model selection that leverages the DR-nature of our treatment effect estimator and that outperforms cross-validation in a small simulation study.

[Malignant epithelial tumors of the oral cavity (II)].

Matching Algorithms for Causal Inference with Multiple Treatments

Distinguishing Between Mediators and Confounders is Important for the Causal Inference in Observational Studies

The covariate-adjusted residual estimator and its use in both randomized trials and observational settings

RCT Rejection Sampling for Causal Estimation Evaluation

Applying the estimand and target trial frameworks to external control analyses using observational data: a case study in the solid tumor setting

A meta-analytic framework to adjust for bias in external control studies

Causal Inference With Two Versions of Treatment

Causal inference with observational data in addiction research

Matching within a hybrid RCT/RWD: framework on associated causal estimands

Causal inference methods for combining randomized trials and observational studies: a review

An Evaluation Toolkit to Guide Model Selection and Cohort Definition in Causal Inference

genRCT: a statistical analysis framework for generalizing RCT findings to real-world population

Estimating treatment effect in randomized trial after control to treatment crossover using external controls

Multiple-model-based robust estimation of causal treatment effect on a binary outcome with integrated information from secondary outcomes

A Two-Stage Method for Extending Inferences from a Collection of Trials

Series 2-19-2009 Multiple Imputation Methods for Treatment Noncompliance and Nonresponse in Randomized Clinical Trials

A standardized framework for risk-based assessment of treatment effect heterogeneity in observational healthcare databases

Enhancing Statistical Validity and Power in Hybrid Controlled Trials: A Randomization Inference Approach with Conformal Selective Borrowing

Detecting critical treatment effect bias in small subgroups

Double-robust and efficient methods for estimating the causal effects of a binary treatment