Practical Meta-Analysis

M. Lipsey,David B. Wilson

2000-08-18

Abstract:Introduction Problem Specification and Study Retrieval Selecting, Computing and Coding the Effect Size Statistic Developing a Coding Scheme and Coding Study Reports Data Management Analysis Issues and Strategies Computational Techniques for Meta-Analysis Data Interpreting and Using Meta-Analysis Results

Mathematics,Computer Science

What problem does this paper attempt to address?

How To Perform A Meta-Analysis: A Practical Step-By-Step Guide Using R Software And Rstudio

Camargo,Olavo Pires de

DOI: https://doi.org/10.1590/1413-785220223003e248775

2022-05-24

Abstract:Ariel De Lima, Diego ; Helito, Camilo Partezani ; Lima, Lana Lacerda De ; Clazzer, Renata ; Gonçalves, Romeu Krause ; Camargo, Olavo Pires De ; Meta-analysis is an adequate statistical technique to combine results from different studies, and its use has been growing in the medical field. Thus, not only knowing how to interpret meta-analysis, but also knowing how to perform one, is fundamental today. Therefore, the objective of this article is to present the basic concepts and serve as a guide for conducting a meta-analysis using R and RStudio software. For this, the reader has access to the basic commands in the R and RStudio software, necessary for conducting a meta-analysis. The advantage of R is that it is a free software. For a better understanding of the commands, two examples were presented in a practical way, in addition to revising some basic concepts of this statistical technique. It is assumed that the data necessary for the meta-analysis has already been collected, that is, the description of methodologies for systematic review is not a discussed subject. Finally, it is worth remembering that there are many other techniques used in meta-analyses that were not addressed in this work. However, with the two examples used, the article already enables the reader to proceed with good and robust meta-analyses. Level of Evidence V, Expert Opinion.

orthopedics
How to perform a meta-analysis with R: a practical tutorial

Sara Balduzzi,Gerta Rücker,Guido Schwarzer

DOI: https://doi.org/10.1136/ebmental-2019-300117

2019-09-28

Abstract:Objective Meta-analysis is of fundamental importance to obtain an unbiased assessment of the available evidence. In general, the use of meta-analysis has been increasing over the last three decades with mental health as a major research topic. It is then essential to well understand its methodology and interpret its results. In this publication, we describe how to perform a meta-analysis with the freely available statistical software environment R, using a working example taken from the field of mental health. Methods R package meta is used to conduct standard meta-analysis. Sensitivity analyses for missing binary outcome data and potential selection bias are conducted with R package metasens. All essential R commands are provided and clearly described to conduct and report analyses. Results The working example considers a binary outcome: we show how to conduct a fixed effect and random effects meta-analysis and subgroup analysis, produce a forest and funnel plot and to test and adjust for funnel plot asymmetry. All these steps work similar for other outcome types. Conclusions R represents a powerful and flexible tool to conduct meta-analyses. This publication gives a brief glimpse into the topic and provides directions to more advanced meta-analysis methods available in R.

psychiatry
Meta‐epidemiological Study: a Step‐by‐step Approach by Using R

Zhongheng Zhang

DOI: https://doi.org/10.1111/jebm.12191

2016-01-01

Journal of Evidence-Based Medicine

Abstract:The results of meta-analysis are usually influenced by characteristics of component trials. These characteristics include but are not limited to sample size, publication language, quality of study design and the number of participating centers. Because systematic review and meta-analysis is considered to be evidence of the highest quality in the era of evidence-based medicine, characteristics that may bias the results of meta-analysis should be fully investigated. Meta-epidemiological study aims to investigate the impact of study design characteristics on treatment effect. The article introduces how to perform meta-epidemiological study by using R software. There are several methodological approaches to perform meta-epidemiological study, depending on whether there is heterogeneity between meta-analyses and between trials. When there is no significant heterogeneity, these results would be similar.
Comparison of Three Major Meta-Analytic Approaches.

Blair T. Johnson,Brian Mullen,Eduardo Salas

DOI: https://doi.org/10.1037/0021-9010.80.1.94

IF: 11.8025

1995-01-01

Journal of Applied Psychology

Abstract:Three major meta-analytic approaches have been developed and applied to integrate empirical research. A comparison of their statistical formulas suggested that they should yield different answers to the meta-analytic questions of central tendency, variability, and prediction by moderators. These frameworks were used to analyze systematically differing databases and showed that, although the techniques of L. V. Hedges and I. Olkin (1985) and of R. Rosenthal and D. Rubin (e.g., 1978, 1988; Rosenthal, 1991) tended to produce reasonable and convergent results, the results of J. E. Hunter, F. L. Schmidt, and G. B. Jackson (1982; Hunter & Schmidt, 1990) often diverged from the other 2 frameworks. For example, consistent with the law of large numbers, finding more studies with the same result is less likely to occur because of chance alone; the Hedges and Olkin and the Rosenthal and Rubin approaches confirmed this prediction, but the Hunter et al. approach did not. The distinct tendency of the Hunter et al. framework to produce results that violate conventional expectations suggests that it should be used with caution
Mastering meta‐analysis in Microsoft Excel with MetaXL add‐in: A comprehensive tutorial and guide to meta‐analysis

Ibrahim Elmakaty

DOI: https://doi.org/10.1111/jep.14138

2024-10-03

Journal of Evaluation in Clinical Practice

Abstract:Rationale Meta‐analysis, a powerful technique for combining effect estimates from multiple studies, enhances statistical power and precision. However, its adoption can be hindered by challenges in statistical interpretation and the complexity of specialized software. MetaXL, a freely available Microsoft Excel add‐in, aims to mitigate these barriers by providing comprehensive support and facilitating seamless integration of meta‐analytical results into research publications. Aims and Objectives This tutorial illustrates the practical application of MetaXL for synthesizing meta‐analytical evidence, with a focus on common effect sizes and their presentation. Method This paper reintroduce MetaXL's functions and provide concise explanations of common effect sizes employed in meta‐analysis. The tutorial delves into fundamental concepts such as the selection of appropriate effect sizes for pooling and the choice of meta‐analytical models. Eight illustrative examples are presented, incorporating diverse effect sizes and data formats, including scenarios involving incidence rate ratios, weighted and standardized mean differences, hazard ratios, and prevalence. Additionally, key concepts in network meta‐analysis are discussed, along with their implementation in MetaXL. MetaXL provides convenient access to data formatting templates tailored to various data types and effect sizes encountered in included studies. Results and Conclusion This tutorial offers researchers, particularly those with limited resources, detailed explanations and insights into commonly used methodologies for pooling effect sizes. Furthermore, it introduces the new Excel functions that comes with the MetaXL add‐in. Accurate population of this function and adherence to the correct format are essential to ensure error‐free analyzes.

medicine, general & internal,health care sciences & services,medical informatics
COMPARISON OF 3 MAJOR METAANALYTIC APPROACHES

BT JOHNSON,B MULLEN,E SALAS

DOI: https://doi.org/10.1037/0021-9010.80.1.94

IF: 11.8025

1995-01-01

Journal of Applied Psychology

Abstract:Three major meta-analytic approaches have been developed and applied to integrate empirical research. A comparison of their statistical formulas suggested that they should yield different answers to the meta-analytic questions of central tendency, variability, and prediction by moderators. These frameworks were used to analyze systematically differing databases and showed that, although the techniques of L. V. Hedges and I. Olkin (1985)and of R. Rosenthal and D. Rubin(e.g., 1978, 1988; Rosenthal, 1991)tended to produce reasonable and convergent results, the results of J. E. Hunter, F. L. Schmidt, and G. B. Jackson( 1982; Hunter and Schmidt, 1990) often diverged from the other 2 frameworks. For example, consistent with the law of large numbers, finding more studies with the same result is less likely to occur because of chance alone; the Hedges and Olkin and the Rosenthal and Rubin approaches confirmed this prediction, but the Hunter et al. approach did not. The distinct tendency of the Hunter et al. framework to produce results that violate conventional expectations suggests that it should be used with caution.
Meta‐Analysis of Studies with Missing Data

Ying Yuan,Roderick J. A. Little

DOI: https://doi.org/10.1111/j.1541-0420.2008.01068.x

IF: 1.701

2009-01-01

Biometrics

Abstract:Summary Consider a meta‐analysis of studies with varying proportions of patient‐level missing data, and assume that each primary study has made certain missing data adjustments so that the reported estimates of treatment effect size and variance are valid. These estimates of treatment effects can be combined across studies by standard meta‐analytic methods, employing a random‐effects model to account for heterogeneity across studies. However, we note that a meta‐analysis based on the standard random‐effects model will lead to biased estimates when the attrition rates of primary studies depend on the size of the underlying study‐level treatment effect. Perhaps ignorable within each study, these types of missing data are in fact not ignorable in a meta‐analysis. We propose three methods to correct the bias resulting from such missing data in a meta‐analysis: reweighting the DerSimonian–Laird estimate by the completion rate; incorporating the completion rate into a Bayesian random‐effects model; and inference based on a Bayesian shared‐parameter model that includes the completion rate. We illustrate these methods through a meta‐analysis of 16 published randomized trials that examined combined pharmacotherapy and psychological treatment for depression.
Meta-analysis in a digitalized world: A step-by-step primer

Esther Kaufmann,Ulf-Dietrich Reips

DOI: https://doi.org/10.3758/s13428-024-02374-8

IF: 5.953

2024-04-06

Behavior Research Methods

Abstract:In recent years, much research and many data sources have become digital. Some advantages of digital or Internet-based research, compared to traditional lab research (e.g., comprehensive data collection and storage, availability of data) are ideal for an improved meta-analyses approach.In the meantime, in meta-analyses research, different types of meta-analyses have been developed to provide research syntheses with accurate quantitative estimations. Due to its rich and unique palette of corrections, we recommend to using the Schmidt and Hunter approach for meta-analyses in a digitalized world. Our primer shows in a step-by-step fashion how to conduct a high quality meta-analysis considering digital data and highlights the most obvious pitfalls (e.g., using only a bare-bones meta-analysis, no data comparison) not only in aggregation of the data, but also in the literature search and coding procedure which are essential steps in any meta-analysis. Thus, this primer of meta-analyses is especially suited for a situation where much of future research is headed to: digital research. To map Internet-based research and to reveal any research gap, we further synthesize meta-analyses on Internet-based research (15 articles containing 24 different meta-analyses, on 745 studies, with 1,601 effect sizes), resulting in the first mega meta-analysis of the field. We found a lack of individual participant data (e.g., age and nationality). Hence, we provide a primer for high-quality meta-analyses and mega meta-analyses that applies to much of coming research and also basic hands-on knowledge to conduct or judge the quality of a meta-analyses in a digitalized world.

psychology, experimental, mathematical
A brief note on the common (fixed)-effect meta-analysis model

Areti Angeliki Veroniki,Joanne E McKenzie,Joanne E. McKenzie

DOI: https://doi.org/10.1016/j.jclinepi.2024.111281

IF: 7.407

2024-02-01

Journal of Clinical Epidemiology

Abstract:Meta-analysis is a statistical method used to combine results from multiple studies, providing a quantitative summary of their findings. One of the fundamental decisions in conducting a meta-analysis is choosing an appropriate model to estimate the overall effect size and its CI. In this article, we focus on the common-effect (also referred to as the fixed-effect) model, and in a companion article, the random-effects model. These models are the two prevailing meta-analysis models employed in the literature. In this article, we outline the key assumption underlying the common-effect model, describe different common-effect methods (ie, inverse variance, Peto, and Mantel-Haenszel), and highlight characteristics of the meta-analysis that should be considered when selecting a method. Furthermore, we demonstrate the application of these methods to a dataset. Understanding the common-effect model is important for knowing when to use the model and how to interpret the overall effect size and its CI.

public, environmental & occupational health,health care sciences & services
Meta-Analysis of Research in Social and Personality Psychology

Blair T. Johnson,Alice H. Eagly

DOI: https://doi.org/10.1017/cbo9780511996481.031

2014-01-01

Abstract:Follow this and additional works at: http://digitalcommons.uconn.edu/chip_docs Part of the Applied Statistics Commons, Behavior and Behavior Mechanisms Commons, Biostatistics Commons, Human Geography Commons, Other Statistics and Probability Commons, Personality and Social Contexts Commons, Probability Commons, Psychological Phenomena and Processes Commons, Social Psychology Commons, Spatial Science Commons, Statistical Methodology Commons, Statistical Models Commons, and the Statistical Theory Commons
Robust Point and Variance Estimation for Meta‐analyses with Selective Reporting and Dependent Effect Sizes

Yefeng Yang,Malgorzata Lagisz,Coralie Williams,Daniel W. A. Noble,Jinming Pan,Shinichi Nakagawa

DOI: https://doi.org/10.1111/2041-210x.14377

2024-01-01

Methods in Ecology and Evolution

Abstract:Meta‐analysis produces a quantitative synthesis of evidence‐based knowledge, shaping not only research trends but also policies and practices in biology. However, two statistical issues, selective reporting and statistical dependence, can severely distort meta‐analytic parameter estimation and inference. Here, we re‐analyse 448 meta‐analyses to demonstrate a new two‐step procedure to deal with two common challenges in biological meta‐analyses that often occur simultaneously: publication bias and non‐independence. First, we employ bias‐robust weighting schemes under the generalized least square estimator to obtain average effect sizes that are more robust to selective reporting. We then use cluster‐robust variance estimation to account for statistical dependence, reducing bias in estimating standard errors and ensuring valid statistical inference. The first step of our approach demonstrates comparable performance in estimating average effect sizes to the existing publication‐bias adjustment methods in the presence of selective reporting. This equivalence holds across two publication bias selection processes. The second step achieves estimates of standard errors consistent with the multilevel meta‐analytic model, a benchmark method with adequate control of Type I error rates for multiple, statistically dependent effect sizes. Re‐analyses of 448 meta‐analyses show that ignoring these two issues tends to overestimate effect sizes by an average of 110% and underestimate standard errors by 120%. To facilitate implementation, we have developed a website including a step‐by‐step tutorial. Complementing current meta‐analytic workflows with the proposed method as a sensitivity analysis can facilitate a transition to a more robust approach in quantitative evidence synthesis.
Interpretation of meta-analyses

Pascal Richard David Clephas,Michael Heesen

DOI: https://doi.org/10.1016/j.inpm.2022.100120

2022-08-15

Abstract:Our article provides guidance on how to interpret a meta-analysis and introduces the reader to the basics of the underlying statistical analysis. The multiple steps of a meta-analysis including systematic literature search, risk of bias assessment, data extraction and data aggregation are addressed. The Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach allows to score the quality of the evidence of the results revealed by a meta-analysis. Trial sequential analysis has been suggested in recent years as a method to assess the power of a meta-analysis and the risks of false positive or false negative conclusions. We also provide information on other more complex meta-analytical approaches including network meta-analysis for the comparison of several treatments as well as recent developments such as individual patient data meta-analysis and living meta-analysis.
Calculating and reporting effect sizes to facilitate cumulative science: a practical primer for t-tests and ANOVAs

Daniël Lakens

DOI: https://doi.org/10.3389/fpsyg.2013.00863

IF: 3.8

2013-01-01

Frontiers in Psychology

Abstract:Effect sizes are the most important outcome of empirical studies. Most articles on effect sizes highlight their importance to communicate the practical significance of results. For scientists themselves, effect sizes are most useful because they facilitate cumulative science. Effect sizes can be used to determine the sample size for follow-up studies, or examining effects across studies. This article aims to provide a practical primer on how to calculate and report effect sizes for t-tests and ANOVA's such that effect sizes can be used in a-priori power analyses and meta-analyses. Whereas many articles about effect sizes focus on between-subjects designs and address within-subjects designs only briefly, I provide a detailed overview of the similarities and differences between within- and between-subjects designs. I suggest that some research questions in experimental psychology examine inherently intra-individual effects, which makes effect sizes that incorporate the correlation between measures the best summary of the results. Finally, a supplementary spreadsheet is provided to make it as easy as possible for researchers to incorporate effect size calculations into their workflow.

psychology, multidisciplinary
A Call For Cautious Interpretation Of Meta-Analytic Reviews

Frank Boers,Lara Bryfonski,Farahnaz Faez,Todd McKay

DOI: https://doi.org/10.1017/S0272263120000327

IF: 4.73

2021-04-16

Studies in Second Language Acquisition

Abstract:Meta-analytic reviews collect available empirical studies on a specified domain and calculate the average effect of a factor. Educators as well as researchers exploring a new domain of inquiry may rely on the conclusions from meta-analytic reviews rather than reading multiple primary studies. This article calls for caution in this regard because the outcome of a meta-analysis is determined by how effect sizes are calculated, how factors are defined, and how studies are selected for inclusion. Three recently published meta-analyses are reexamined to illustrate these issues. The first illustrates the risk of conflating effect sizes from studies with different design features; the second illustrates problems with delineating the variable of interest, with implications for cause-effect relations; and the third illustrates the challenge of determining the eligibility of candidate studies. Replication attempts yield outcomes that differ from the three original meta-analyses, suggesting also that conclusions drawn from meta-analyses need to be interpreted cautiously.

linguistics
A Hands-On Tutorial for Systematic Review and Meta-Analysis With Example Data Set and Codes

Xiaojuan Zhang,Bing Cheng,Yang Zhang

DOI: https://doi.org/10.1044/2022_JSLHR-21-00607

2022-09-12

Abstract:Purpose: Systematic review and meta-analysis are regarded as standard and valuable tools for providing an objective and reproducible synthesis of research findings in the literature. Their increasing popularity has led to heightened expectations for comprehensiveness and rigor in conducting scientific reviews and analyses. The purpose of this tutorial is to provide an overview and guidance on how to address the key elements for conducting a high-quality systematic review with meta-analysis. Method: This tutorial describes practical guidelines to conduct systematic review and meta-analysis for researchers who are interested in learning and applying the method. Drawing on previous meta-analytic reviews of second-language learning as illustrative examples, it discusses the methodological choices and judgment calls in each step of the review and analysis process. As a hands-on tutorial, it uses a published data set concerning the role of talker variability in speech training studies as a running example to elucidate the statistical process and interpret the results of meta-analysis with freely available R software. Results/conclusions: This tutorial provides a walk-through of the methodological choices, controversial issues, and common practices in conducting systematic reviews and meta-analyses. The data set and R codes are offered as publicly accessible supplementary materials (https://osf.io/e9bkf/) for replication and practice, which we hope will motivate more applications in the speech, language, and hearing sciences field as well as behavioral and social sciences research in general.
On the Use of Beta Coefficients in Meta-Analysis.

Robert A. Peterson,Steven P. Brown

DOI: https://doi.org/10.1037/0021-9010.90.1.175

IF: 11.8025

2005-01-01

Journal of Applied Psychology

Abstract:This research reports an investigation of the use of standardized regression (beta) coefficients in meta-analyses that use correlation coefficients as the effect-size metric. The investigation consisted of analyzing more than 1,700 corresponding beta coefficients and correlation coefficients harvested from published studies. Results indicate that, under certain conditions, using knowledge of corresponding beta coefficients to input missing correlations (effect sizes) generally produces relatively accurate and precise population effect-size estimates. Potential benefits from applying this knowledge include smaller sampling errors because of increased numbers of effect sizes and smaller non-sampling errors because of the inclusion of a broader array of research designs.

psychology, applied,management
What do meta-analysts need in primary studies? Guidelines and the SEMI checklist for facilitating cumulative knowledge

Belén Fernández-Castilla,Sameh Said-Metwaly,Rodrigo S. Kreitchmann,Wim Van Den Noortgate

DOI: https://doi.org/10.3758/s13428-024-02373-9

IF: 5.953

2024-04-17

Behavior Research Methods

Abstract:Meta-analysis is often recognized as the highest level of evidence due to its notable advantages. Therefore, ensuring the precision of its findings is of utmost importance. Insufficient reporting in primary studies poses challenges for meta-analysts, hindering study identification, effect size estimation, and meta-regression analyses. This manuscript provides concise guidelines for the comprehensive reporting of qualitative and quantitative aspects in primary studies. Adhering to these guidelines may help researchers enhance the quality of their studies and increase their eligibility for inclusion in future research syntheses, thereby enhancing research synthesis quality. Recommendations include incorporating relevant terms in titles and abstracts to facilitate study retrieval and reporting sufficient data for effect size calculation. Additionally, a new checklist is introduced to help applied researchers thoroughly report various aspects of their studies.

psychology, experimental, mathematical
Application of Standardized Regression Coefficient in Meta-Analysis

Pentti Nieminen

DOI: https://doi.org/10.3390/biomedinformatics2030028

2022-08-31

BioMedInformatics

Abstract:The lack of consistent presentation of results in published studies on the association between a quantitative explanatory variable and a quantitative dependent variable has been a long-term issue in evaluating the reported findings. Studies are analyzed and reported in a variety of ways. The main purpose of this review is to illustrate the procedures in summarizing and synthesizing research results from multivariate models with a quantitative outcome variable. The review summarizes the application of the standardized regression coefficient as an effect size index in the context of meta-analysis and describe how it can be estimated and converted from data presented in original research articles. An example of synthesis is provided using research articles on the association between childhood body mass index and carotid intima-media thickness in adult life. Finally, the paper shares practical recommendations for meta-analysts wanting to use the standardized regression coefficient in pooling findings.
Metaanalysis - Literature Synthesis or Effect-Size Surface Estimation

DB RUBIN

DOI: https://doi.org/10.3102/10769986017004363

1992-01-01

Journal of Educational Statistics

Abstract:A traditional meta-analysis can be thought of as a literature synthesis, in which a collection of observed studies is analyzed to obtain summary judgments about overall significance and size of effects. Many aspects of the current set of statistical tools for meta-analysis are highly useful-for example, the development of clear and concise effect-size indicators with associated standard errors. I am less happy, however, with more esoteric statistical techniques and their implied objects of estimation (i.e., their estimands), which are tied to the conceptualization of average effect sizes, weighted or otherwise, in a population of studies. In contrast to these average effect sizes of literature synthesis, I believe that the proper estimand is an effect-size surface, which is a function only of scientifically relevant factors, and which can only be estimated by extrapolating a response surface of observed effect sizes to a region of ideal studies. This effect-size surface perspective is presented and contrasted with the literature synthesis perspective. The presentation is entirely conceptual. Moreover, it is designed to be provocative, thereby prodding researchers to rethink traditional meta-analysis and ideally stimulating meta-analysts to attempt effect-surface estimations.
[Selection Models and the File Drawer Problem]: Comment

Robert Rosenthal,Donald B. Rubin

DOI: https://doi.org/10.1214/ss/1177013012

1988-01-01

Statistical Science

Abstract:Meta-analysis consists of quantitative methods for combining evidence from different studies about a particular issue. A frequent criticism of meta-analysis is that it may be based on a biased sample of all studies that were done. In this paper, we use selection models, or weighted distributions, to deal with one source of bias, namely, the failure to report studies that do not yield statistically significant results. We apply selection models to two approaches that have been suggested for correcting the bias. The fail-safe sample size approach calculates the minimum number of unpublished studies showing nonsignificant results that must have been carried out in order to overturn the conclusion reached from the published studies. The maximum likelihood approach uses a weighted distribution to model the selection bias in the generation of the data and estimates various parameters of interest. We suggest the use of families of weight functions to model plausible biasing mechanisms to study the sensitivity of inferences about effect sizes. By using an example, we show that the maximum likelihood approach has several advantages over the fail-safe sample size approach.

Practical Meta-Analysis

How To Perform A Meta-Analysis: A Practical Step-By-Step Guide Using R Software And Rstudio

How to perform a meta-analysis with R: a practical tutorial

Meta‐epidemiological Study: a Step‐by‐step Approach by Using R

Comparison of Three Major Meta-Analytic Approaches.

Mastering meta‐analysis in Microsoft Excel with MetaXL add‐in: A comprehensive tutorial and guide to meta‐analysis

COMPARISON OF 3 MAJOR METAANALYTIC APPROACHES

Meta‐Analysis of Studies with Missing Data

Meta-analysis in a digitalized world: A step-by-step primer

A brief note on the common (fixed)-effect meta-analysis model

Meta-Analysis of Research in Social and Personality Psychology

Robust Point and Variance Estimation for Meta‐analyses with Selective Reporting and Dependent Effect Sizes

Interpretation of meta-analyses

Calculating and reporting effect sizes to facilitate cumulative science: a practical primer for t-tests and ANOVAs

A Call For Cautious Interpretation Of Meta-Analytic Reviews

A Hands-On Tutorial for Systematic Review and Meta-Analysis With Example Data Set and Codes

On the Use of Beta Coefficients in Meta-Analysis.

What do meta-analysts need in primary studies? Guidelines and the SEMI checklist for facilitating cumulative knowledge

Application of Standardized Regression Coefficient in Meta-Analysis

Metaanalysis - Literature Synthesis or Effect-Size Surface Estimation

[Selection Models and the File Drawer Problem]: Comment