Abstract:Importance: Small study effects are the phenomena that studies with smaller sample sizes tend to report larger and more favorable effect estimates than studies with larger sample sizes. Objective: To evaluate the presence and extent of small study effects in diagnostic imaging accuracy meta-analyses. Data sources: A search was conducted in the PubMed database for diagnostic imaging accuracy meta-analyses published between 2010 and 2019. Study selection: Meta-analyses with 10 or more studies of medical imaging diagnostic accuracy, assessing a single imaging modality, and providing 2 × 2 contingency data were included. Studies that did not assess diagnostic accuracy of medical imaging techniques, compared 2 or more imaging modalities or different methods of 1 imaging modality, were cost analyses, used predictive or prognostic tests, did not provide individual patient data, or were network meta-analyses were excluded. Data extraction and synthesis: Data extraction was performed in accordance with the PRISMA guidelines. Main outcomes and measures: The diagnostic odds ratio (DOR) was calculated for each primary study using 2 × 2 contingency data. Regression analysis was used to examine the association between effect size estimate and precision across meta-analyses. Results: A total of 31 meta-analyses involving 668 primary studies and 80 206 patients were included. Fixed effects analysis produced a regression coefficient for the natural log of DOR against the SE of the natural log of DOR of 2.19 (95% CI, 1.49-2.90; P < .001), with computed tomography as the reference modality. Interaction test for modality and SE of the natural log of DOR did not depend on modality (Wald statistic P = .50). Taken together, this analysis found an inverse association between effect size estimate and precision that was independent of imaging modality. Of 26 meta-analyses that formally assessed for publication bias using funnel plots and statistical tests for funnel plot asymmetry, 21 found no evidence for such bias. Conclusions and relevance: This meta-analysis found evidence of widespread prevalence of small study effects in the diagnostic imaging accuracy literature. One likely contributor to the observed effects is publication bias, which can undermine the results of many meta-analyses. Conventional methods for detecting funnel plot asymmetry conducted by included studies appeared to underestimate the presence of small study effects. Further studies are required to elucidate the various factors that contribute to small study effects.

The Effects of Sample Size on Omics Study: from the Perspective of Robustness and Diagnostic Accuracy

Sample size and statistical power considerations in high-dimensionality data settings: a comparative study of classification algorithms

Small Studies May Overestimate the Effect Sizes in Critical Care Meta-Analyses: A Meta-Epidemiological Study

SSizer: Determining the Sample Sufficiency for Comparative Biological Study.

Small Study Effects in Diagnostic Imaging Accuracy: A Meta-Analysis

Sample size estimation in diagnostic test studies of biomedical informatics

Small effect size leads to reproducibility failure in resting-state fMRI studies

Comparative Integrated Omics Approach Sterically Understanding Hepatic Metabolic Dynamics in Mouse Model: DOI: 10.5584/Jiomics.v7i1.200

Sample size and performance estimation for biomarker combinations based on pilot studies with small sample sizes

MetSizeR: selecting the optimal sample size for metabolomic studies using an analysis based approach

Statistics in Brief: The Importance of Sample Size in the Planning and Interpretation of Medical Research

Evaluation of a decided sample size in machine learning applications

Quantitative Comparison of Statistical Methods for Analyzing Human Metabolomics Data

Addressing statistical challenges in the analysis of proteomics data with extremely small sample size: a simulation study

Data analysis methods for defining biomarkers from omics data

Tips for troublesome sample-size calculation

Microarray experimental design: power and sample size considerations

Minor Issues Escalated to Critical Levels in Large Samples: A Permutation-Based Fix

Sample Size Calculation in Clinical Trial and Animal Studies

Assessing Covariate Balance with Small Sample Sizes

The importance of small samples in medical research