Performance of the Nonparametric Trim and Fill Method in Stata

张天嵩,钟文昭

DOI: https://doi.org/10.3969/j.issn.1671-5144.2009.04.026

2009-01-01

Journal of Evidence-Based Medicine

Abstract:The trim and fill method aims to both identify and correct for funnel plot asymmetry arising from publication bias. With a published meta-analysis as an example,this paper explains a procedure how to perform the nonparametric trim and fill method in Stata.

What problem does this paper attempt to address?

Drawing Funnel Plot and Testing for Funnel Plot Asymmetry in Stata

张天嵩,钟文昭,徐同成

DOI: https://doi.org/10.3969/j.issn.1671-5144.2009.02.022

2009-01-01

Journal of Evidence-Based Medicine

Abstract:Funnel plots are a visual tool for investigating publication and other bias in meta-analysis. By using a published meta-analysis as an example, this paper explains many methods of Stata in drawing funnel plot and testing for funnel plot asymmetry.
Cumulative Meta-Analysis in Stata

张天嵩,钟文昭

DOI: https://doi.org/10.3969/j.issn.1671-5144.2010.01.016

2010-01-01

Journal of Evidence-Based Medicine

Abstract:Cumulative meta-analysis is the procedure of performing a (new) meta-analysis at every point during the history of a research field. It is easy to perform cumulative meta-analysis by metacum command in Stata. With a data from the network as an example, this paper explains the procedure of performing cumulative meta-analysis in Stata.
Generalizing Trimming Bounds for Endogenously Missing Outcome Data Using Random Forests

Cyrus Samii,Ye Wang,Junlong Aaron Zhou

DOI: https://doi.org/10.48550/arXiv.2309.08985

2023-09-16

Abstract:In many experimental or quasi-experimental studies, outcomes of interest are only observed for subjects who select (or are selected) to engage in the activity generating the outcome. Outcome data is thus endogenously missing for units who do not engage, in which case random or conditionally random treatment assignment prior to such choices is insufficient to point identify treatment effects. Non-parametric partial identification bounds are a way to address endogenous missingness without having to make disputable parametric assumptions. Basic bounding approaches often yield bounds that are very wide and therefore minimally informative. We present methods for narrowing non-parametric bounds on treatment effects by adjusting for potentially large numbers of covariates, working with generalized random forests. Our approach allows for agnosticism about the data-generating process and honest inference. We use a simulation study and two replication exercises to demonstrate the benefits of our approach.

Methodology,Applications
Propensity Score Method: a Non-Parametric Technique to Reduce Model Dependence

Zhongheng Zhang

DOI: https://doi.org/10.21037/atm.2016.08.57

IF: 3.616

2017-01-01

Annals of Translational Medicine

Abstract:Propensity score analysis (PSA) is a powerful technique that it balances pretreatment covariates, making the causal effect inference from observational data as reliable as possible. The use of PSA in medical literature has increased exponentially in recent years, and the trend continue to rise. The article introduces rationales behind PSA, followed by illustrating how to perform PSA in R with MatchIt package. There are a variety of methods available for PS matching such as nearest neighbors, full matching, exact matching and genetic matching. The task can be easily done by simply assigning a string value to the method argument in the matchit() function. The generic summary() and plot() functions can be applied to an object of class matchit to check covariate balance after matching. Furthermore, there is a useful package PSAgraphics that contains several graphical functions to check covariate balance between treatment groups across strata. If covariate balance is not achieved, one can modify model specifications or use other techniques such as random forest and recursive partitioning to better represent the underlying structure between pretreatment covariates and treatment assignment. The process can be repeated until the desirable covariate balance is achieved.
Correcting Classifiers for Sample Selection Bias in Two-Phase Case-Control Studies

Norbert Krautenbacher,Fabian J. Theis,Christiane Fuchs

DOI: https://doi.org/10.1155/2017/7847531

IF: 2.809

2017-01-01

Computational and Mathematical Methods in Medicine

Abstract:Epidemiological studies often utilize stratified data in which rare outcomes or exposures are artificially enriched. This design can increase precision in association tests but distorts predictions when applying classifiers on nonstratified data. Several methods correct for this so-called sample selection bias, but their performance remains unclear especially for machine learning classifiers. With an emphasis on two-phase case-control studies, we aim to assess which corrections to perform in which setting and to obtain methods suitable for machine learning techniques, especially the randomforest. We propose two newresampling-basedmethods to resemble the original data and covariance structure: stochastic inverse-probability oversampling and parametric inverse-probability bagging. We compare all techniques for the random forest and other classifiers, both theoretically and on simulated and real data. Empirical results show that the random forest profits from only the parametric inverse-probability bagging proposed by us. For other classifiers, correction is mostly advantageous, and methods perform uniformly. We discuss consequences of inappropriate distribution assumptions and reason for different behaviors between the random forest and other classifiers. In conclusion, we provide guidance for choosing correction methods when training classifiers on biased samples. For random forests, our method outperforms state-of-the-art procedures if distribution assumptions are roughly fulfilled. We provide our implementation in the R package sambia.
A nonparametric maximum likelihood estimation for biased-sampling data with zero-inflated truncation

Feipeng Zhang,Jiejing Yang,Min Ye

DOI: https://doi.org/10.1016/j.econlet.2020.109399

IF: 1.469

2020-01-01

Economics Letters

Abstract:This paper considers biased-sampling data with zero-inflated truncation, where the survival times are left truncated by zero-inflated distributed truncation times. We develop a nonparametric estimator of survival function for biased-sampling data with zero-inflated truncation via a new expectation–maximization algorithm. We demonstrate the good performance of the proposed estimator through numerical simulation studies. An empirical application of employment data is conducted to illustrate the method.
Testing for Publication Bias in Diagnostic Meta-Analysis: A Simulation Study

Paul-Christian Bürkner,Philipp Doebler

DOI: https://doi.org/10.48550/arXiv.2211.12538

2022-11-22

Methodology

Abstract:The present study investigates the performance of several statistical tests to detect publication bias in diagnostic meta-analysis by means of simulation. While bivariate models should be used to pool data from primary studies in diagnostic meta-analysis, univariate measures of diagnostic accuracy are preferable for the purpose of detecting publication bias. In contrast to earlier research, which focused solely on the diagnostic odds ratio or its logarithm ($\ln\omega$), the tests are combined with four different univariate measures of diagnostic accuracy. For each combination of test and univariate measure, both type I error rate and statistical power are examined under diverse conditions. The results indicate that tests based on linear regression or rank correlation cannot be recommended in diagnostic meta-analysis, because type I error rates are either inflated or power is too low, irrespective of the applied univariate measure. In contrast, the combination of trim and fill and $\ln\omega$ has non-inflated or only slightly inflated type I error rates and medium to high power, even under extreme circumstances (at least when the number of studies per meta-analysis is large enough). Therefore, we recommend the application of trim and fill combined with $\ln\omega$ to detect funnel plot asymmetry in diagnostic meta-analysis. Please cite this paper as published in Statistics in Medicine (https://doi.org/10.1002/sim.6177).
Outliers in meta-analysis: an asymmetric trimmed-mean approach

Rose Baker

DOI: https://doi.org/10.48550/arXiv.1907.07015

2019-07-16

Applications

Abstract:The adaptive asymmetric trimmed mean is a known way of estimating central location, usually in conjunction with the bootstrap. It is here modified and applied to meta-analysis, as a way of dealing with outlying results by down-weighting the corresponding studies. This requires a modified bootstrap and a method of down-weighting studies, as opposed to removing single observations. This methodology is shown in analysis of some well-travelled datasets to down-weight outliers in agreement with other methods, and Monte-Carlo studies show that it does does not appreciably down-weight studies when outliers are absent. Conceptually simple, it does not make parametric assumptions about the outliers.
Improved Variance Estimation From Trimmed Samples

Daniel Andrade

DOI: https://doi.org/10.1002/sta4.70018

2024-10-26

Stat

Abstract:Several methods for the robust estimation of the variance of a normal random variable are based on trimming. For example, the popular Qn estimator is based on trimming the pairs of samples with the largest absolute distance. Another example is least trimmed squares regression, where all unusually large residuals are removed before the estimation of the variance. Here, in this work, we propose two new classes of estimators that use an optimal linear combination of the trimmed samples to achieve a lower mean squared error (MSE). The first class guarantees the smallest variance among all unbiased estimators that are based on linear combinations, while the second class of estimators guarantees the smallest MSE at the cost of some small bias. Through simulations, we demonstrate that our proposed estimators can have considerably smaller MSE than other robust estimators both in the presence and absence of outliers.

statistics & probability
Using Resampling Methods to Produce an Improved DIMTEST Procedure

William Stout,Amy Goodwin Froelich,Furong Gao

DOI: https://doi.org/10.1007/978-1-4613-0169-1_19

2001-01-01

Abstract: Following in the nonparametric item response theory tradition, DIMTEST (Stout, 1987) is an asymptotically justified nonparametric procedure that provides a test of hypothesis of unidimensionality of a test data set. This chapter introduces a new bias correction method for the DIMTEST procedure based on the statistical principle of resampling. A simulation study shows this new version of DIMTEST has a Type I error rate close to the nominal rate of α = 0.05 in most cases and very high power to detect multidimensionality in a variety of realistic multidimensional models. The result with this new bias correction method is an improved DIMTEST procedure with much wider applicability and good statistical performance.
A modified regression method to test publication bias in meta-analyses with binary outcomes

Zhi-Chao Jin,Cheng Wu,Xiao-Hua Zhou,Jia He

DOI: https://doi.org/10.1186/1471-2288-14-132

2014-12-01

BMC Medical Research Methodology

Abstract:BackgroundThe tendency towards publication bias is greater for observational studies than for randomized clinical trials. Several statistical methods have been developed to test the publication bias. However, almost all existing methods exhibit rather low power or have inappropriate type I error rates.MethodsWe propose a modified regression method, which used a smoothed variance to estimate the precision of a study, to test for publication bias in meta-analyses of observational studies. A comprehensive simulation study is carried out, and a real-world example is considered.ResultsThe simulation results indicate that the performance of tests varies with the number of included studies, level of heterogeneity, event rates, and sample size ratio between two groups. Neither the existing tests nor the newly developed method is particularly powerful in all simulation scenarios. However, our proposed method has a more robust performance across different settings. In the presence of heterogeneity, the arcsine-Thompson test is a suitable alternative, and Peters’ test can be considered as a complementary method when mild or no heterogeneity is present.ConclusionsSeveral factors should be taken into consideration when employing asymmetry tests for publication bias. Based on our simulation results, we provide a concise table to show the appropriate use of regression methods to test for publication bias based on our simulation results.

health care sciences & services
Goodness-of-fit Test for Meta-Analysis

Zhongxue Chen,Guoyi Zhang,Jing Li

DOI: https://doi.org/10.1038/srep16983

IF: 4.6

2015-01-01

Scientific Reports

Abstract:Meta-analysis is a very useful tool to combine information from different sources. Fixed effect and random effect models are widely used in meta-analysis. Despite their popularity, they may give us misleading results if the models don't fit the data but are blindly used. Therefore, like any statistical analysis, checking the model fitting is an important step. However, in practice, the goodness-of-fit in meta-analysis is rarely discussed. In this paper, we propose some tests to check the goodness-of-fit for the fixed and random effect models with assumption of normal distributions in meta-analysis. Through simulation study, we show that the proposed tests control type I error rate very well. To demonstrate the usefulness of the proposed tests, we also apply them to some real data sets. Our study shows that the proposed tests are useful tools in checking the goodness-of-fit of the normal models used in meta-analysis.
Heavy Tail Robust Estimation and Inference for Average Treatment Effects

Jonathan B. Hill,Saraswata Chaudhuri

2024-12-11

Abstract:We study the probability tail properties of Inverse Probability Weighting (IPW) estimators of the Average Treatment Effect (ATE) when there is limited overlap between the covariate distributions of the treatment and control groups. Under unconfoundedness of treatment assignment conditional on covariates, such limited overlap is manifested in the propensity score for certain units being very close (but not equal) to 0 or 1. This renders IPW estimators possibly heavy tailed, and with a slower than sqrt(n) rate of convergence. Trimming or truncation is ultimately based on the covariates, ignoring important information about the inverse probability weighted random variable Z that identifies ATE by E[Z]= ATE. We propose a tail-trimmed IPW estimator whose performance is robust to limited overlap. In terms of the propensity score, which is generally unknown, we plug-in its parametric estimator in the infeasible Z, and then negligibly trim the resulting feasible Z adaptively by its large values. Trimming leads to bias if Z has an asymmetric distribution and an infinite variance, hence we estimate and remove the bias using important improvements on existing theory and methods. Our estimator sidesteps dimensionality, bias and poor correspondence properties associated with trimming by the covariates or propensity score. Monte Carlo experiments demonstrate that trimming by the covariates or the propensity score requires the removal of a substantial portion of the sample to render a low bias and close to normal estimator, while our estimator has low bias and mean-squared error, and is close to normal, based on the removal of very few sample extremes.

Methodology,Statistics Theory
On performance of parametric and distribution‐free models for zero‐inflated and over‐dispersed count responses

wan tang,naiji lu,tian chen,wenjuan wang,douglas gunzler,yu han,xin m tu

DOI: https://doi.org/10.1002/sim.6560

2015-01-01

Statistics in Medicine

Abstract:Zero-inflated Poisson (ZIP) and negative binomial (ZINB) models are widely used to model zero-inflated count responses. These models extend the Poisson and negative binomial (NB) to address excessive zeros in the count response. By adding a degenerate distribution centered at 0 and interpreting it as describing a non-risk group in the population, the ZIP (ZINB) models a two-component population mixture. As in applications of Poisson and NB, the key difference between ZIP and ZINB is the allowance for overdispersion by the ZINB in its NB component in modeling the count response for the at-risk group. Overdispersion arising in practice too often does not follow the NB, and applications of ZINB to such data yield invalid inference. If sources of overdispersion are known, other parametric models may be used to directly model the overdispersion. Such models too are subject to assumed distributions. Further, this approach may not be applicable if information about the sources of overdispersion is unavailable. In this paper, we propose a distribution-free alternative and compare its performance with these popular parametric models as well as a moment-based approach proposed by Yu et al. [Statistics in Medicine 2013; 32: 2390-2405]. Like the generalized estimating equations, the proposed approach requires no elaborate distribution assumptions. Compared with the approach of Yu et al., it is more robust to overdispersed zero-inflated responses. We illustrate our approach with both simulated and real study data. Copyright (c) 2015John Wiley & Sons, Ltd.
Mann-Whitney Test with Adjustments to Pretreatment Variables for Missing Values and Observational Study

Song Xi Chen,Jing Qin,Cheng Yong Tang

DOI: https://doi.org/10.1111/j.1467-9868.2012.01036.x

2012-01-01

Journal of the Royal Statistical Society Series B (Statistical Methodology)

Abstract:SummaryThe conventional Wilcoxon or Mann–Whitney test can be invalid for comparing treatment effects in the presence of missing values or in observational studies. This is because the missingness of the outcomes or the participation in the treatments may depend on certain pretreatment variables. We propose an approach to adjust the Mann–Whitney test by correcting the potential bias via consistently estimating the conditional distributions of the outcomes given the pretreatment variables. We also propose semiparametric extensions of the adjusted Mann–Whitney test which lead to dimension reduction for high dimensional covariates. A novel bootstrap procedure is devised to approximate the null distribution of the test statistics for practical implementations. Results from simulation studies and an economics observational study data analysis are presented to demonstrate the performance of the approach proposed.
Optimal trimming proportion in regression analysis for non-normal distributions

Amitava Mitra,Pankush Kalgotra

DOI: https://doi.org/10.1080/2573234X.2021.2007803

2021-11-29

Journal of Business Analytics

Abstract:Regression analysis is a widely used modelling tool in business decision making. However, proper application of this methodology requires that certain assumptions, associated with the model, be satisfied. The assumption we focus on is the normality of the response variable, which is directly related to the assumption of normality of the error component. In a variety of fields in business, such as finance, marketing, information systems, operations, and healthcare, the selected dependent variable does not inherently have a normal distribution. In the regression context, where the model parameters and independent variables are assumed to be constant, the distribution of the random error component thus influences the distribution of the dependent variable. Here, we study the impact of symmetric and asymmetric error distributions on the performance of the estimated model parameters. To create robust estimates, through a process of trimming the response variable, we study the effectiveness of the trimmed estimators with respect to the ordinary least squares estimator (OLS) via a simulation procedure. Accordingly, to minimise the ratio of the mean squared error of the trimmed estimator to that of the OLS, a recommendation is developed for the optimal trimming proportion.
P‐hacking in meta‐analyses: A formalization and new meta‐analytic methods

Maya B. Mathur

DOI: https://doi.org/10.1002/jrsm.1701

2024-01-26

Research Synthesis Methods

Abstract:As traditionally conceived, publication bias arises from selection operating on a collection of individually unbiased estimates. A canonical form of such selection across studies (SAS) is the preferential publication of affirmative studies (i.e., those with significant, positive estimates) versus nonaffirmative studies (i.e., those with nonsignificant or negative estimates). However, meta‐analyses can also be compromised by selection within studies (SWS), in which investigators "p‐hack" results within their study to obtain an affirmative estimate. Published estimates can then be biased even conditional on affirmative status, which comprises the performance of existing methods that only consider SAS. We propose two new analysis methods that accommodate joint SAS and SWS; both analyze only the published nonaffirmative estimates. First, we propose estimating the underlying meta‐analytic mean by fitting "right‐truncated meta‐analysis" (RTMA) to the published nonaffirmative estimates. This method essentially imputes the entire underlying distribution of population effects. Second, we propose conducting a standard meta‐analysis of only the nonaffirmative studies (MAN); this estimate is conservative (negatively biased) under weakened assumptions. We provide an R package (phacking) and website (metabias.io). Our proposed methods supplement existing methods by assessing the robustness of meta‐analyses to joint SAS and SWS.

multidisciplinary sciences,mathematical & computational biology
Missing at Random or Not: A Semiparametric Testing Approach

Rui Duan,C. Jason Liang,Pamela Shaw,Cheng Yong Tang,Yong Chen

DOI: https://doi.org/10.48550/arXiv.2003.11181

2020-03-25

Abstract:Practical problems with missing data are common, and statistical methods have been developed concerning the validity and/or efficiency of statistical procedures. On a central focus, there have been longstanding interests on the mechanism governing data missingness, and correctly deciding the appropriate mechanism is crucially relevant for conducting proper practical investigations. The conventional notions include the three common potential classes -- missing completely at random, missing at random, and missing not at random. In this paper, we present a new hypothesis testing approach for deciding between missing at random and missing not at random. Since the potential alternatives of missing at random are broad, we focus our investigation on a general class of models with instrumental variables for data missing not at random. Our setting is broadly applicable, thanks to that the model concerning the missing data is nonparametric, requiring no explicit model specification for the data missingness. The foundational idea is to develop appropriate discrepancy measures between estimators whose properties significantly differ only when missing at random does not hold. We show that our new hypothesis testing approach achieves an objective data oriented choice between missing at random or not. We demonstrate the feasibility, validity, and efficacy of the new test by theoretical analysis, simulation studies, and a real data analysis.

Methodology,Econometrics
Using Monte Carlo experiments to select meta‐analytic estimators

Sanghyun Hong,W. Robert Reed

DOI: https://doi.org/10.1002/jrsm.1467

2020-11-17

Research Synthesis Methods

Abstract:The purpose of this study is to show how Monte Carlo analysis of meta‐analytic estimators can be used to select estimators for specific research situations. Our analysis conducts 1,620 individual experiments, where each experiment is defined by a unique combination of sample size, effect size, effect size heterogeneity, publication selection mechanism, and other research characteristics. We compare eleven estimators commonly used in medicine, psychology, and the social sciences. These are evaluated on the basis of bias, mean squared error (MSE), and coverage rates. For our experimental design, we reproduce simulation environments from four recent studies. We demonstrate that relative estimator performance differs across performance measures. Estimator performance is a complex interaction of performance indicator and aspects of the application. An estimator that may be especially good with respect to MSE may perform relatively poorly with respect to coverage rates. We also show that the size of the meta‐analyst's sample and effect heterogeneity are important determinants of relative estimator performance. We use these results to demonstrate how these observable characteristics can guide the meta‐analyst to choose the most appropriate estimator for their research circumstances. All of the programming code and output files associated with this project are available at https://osf.io/pr4mb/.This article is protected by copyright. All rights reserved.

multidisciplinary sciences,mathematical & computational biology
Testing for publication bias in meta-analysis under Copas selection model

Rui Duan,Jin Piao,Arielle Marks-Anglin,Jiayi Tong,Lifeng Lin,Haitao Chu,Jing Ning,Yong Chen

DOI: https://doi.org/10.48550/arXiv.2007.00836

2020-07-02

Methodology

Abstract:In meta-analyses, publication bias is a well-known, important and challenging issue because the validity of the results from a meta-analysis is threatened if the sample of studies retrieved for review is biased. One popular method to deal with publication bias is the Copas selection model, which provides a flexible sensitivity analysis for correcting the estimates with considerable insight into the data suppression mechanism. However, rigorous testing procedures under the Copas selection model to detect bias are lacking. To fill this gap, we develop a score-based test for detecting publication bias under the Copas selection model. We reveal that the behavior of the standard score test statistic is irregular because the parameters of the Copas selection model disappear under the null hypothesis, leading to an identifiability problem. We propose a novel test statistic and derive its limiting distribution. A bootstrap procedure is provided to obtain the p-value of the test for practical applications. We conduct extensive Monte Carlo simulations to evaluate the performance of the proposed test and apply the method to several existing meta-analyses.

Performance of the Nonparametric Trim and Fill Method in Stata

Drawing Funnel Plot and Testing for Funnel Plot Asymmetry in Stata

Cumulative Meta-Analysis in Stata

Generalizing Trimming Bounds for Endogenously Missing Outcome Data Using Random Forests

Propensity Score Method: a Non-Parametric Technique to Reduce Model Dependence

Correcting Classifiers for Sample Selection Bias in Two-Phase Case-Control Studies

A nonparametric maximum likelihood estimation for biased-sampling data with zero-inflated truncation

Testing for Publication Bias in Diagnostic Meta-Analysis: A Simulation Study

Outliers in meta-analysis: an asymmetric trimmed-mean approach

Improved Variance Estimation From Trimmed Samples

Using Resampling Methods to Produce an Improved DIMTEST Procedure

A modified regression method to test publication bias in meta-analyses with binary outcomes

Goodness-of-fit Test for Meta-Analysis

Heavy Tail Robust Estimation and Inference for Average Treatment Effects

On performance of parametric and distribution‐free models for zero‐inflated and over‐dispersed count responses

Mann-Whitney Test with Adjustments to Pretreatment Variables for Missing Values and Observational Study

Optimal trimming proportion in regression analysis for non-normal distributions

P‐hacking in meta‐analyses: A formalization and new meta‐analytic methods

Missing at Random or Not: A Semiparametric Testing Approach

Using Monte Carlo experiments to select meta‐analytic estimators

Testing for publication bias in meta-analysis under Copas selection model