Abstract:Regressions and meta-regressions are widely used to estimate patterns and effect sizes in various disciplines. However, many biological and medical analyses use relatively low sample size (N), contributing to concerns on reproducibility. What is the minimum N to identify the most plausible data pattern using regressions? Statistical power analysis is often used to answer that question, but it has its own problems and logically should follow model selection to first identify the most plausible model. Here we make null, simple linear and quadratic data with different variances and effect sizes. We then sample and use information theoretic model selection to evaluate minimum N for regression models. We also evaluate the use of coefficient of determination (R2) for this purpose; it is widely used but not recommended. With very low variance, both false positives and false negatives occurred at N < 8, but data shape was always clearly identified at N ≥ 8. With high variance, accurate inference was stable at N ≥ 25. Those outcomes were consistent at different effect sizes. Akaike Information Criterion weights (AICc wi) were essential to clearly identify patterns (e.g., simple linear vs. null); R2 or adjusted R2 values were not useful. We conclude that a minimum N = 8 is informative given very little variance, but minimum N ≥ 25 is required for more variance. Alternative models are better compared using information theory indices such as AIC but not R2 or adjusted R2. Insufficient N and R2-based model selection apparently contribute to confusion and low reproducibility in various disciplines. To avoid those problems, we recommend that research based on regressions or meta-regressions use N ≥ 25.

Research on the Prediction Method of Weighted Regression Based on Sample Size

Weighted Support Vector Machines for Regression and Its Application

Research of Reliability Assessment Based on Random Weighting Method

Optimal Subsampling Approaches for Large Sample Linear Regression

Sample Weighting: an Inherent Approach for Outlier Suppressing Discriminant Analysis

A study of composite prediction method

Stable Learning via Sample Reweighting

Evaluation of a decided sample size in machine learning applications

A NEW PERSPECTIVE ON ROBUST M-ESTIMATION: FINITE SAMPLE THEORY AND APPLICATIONS TO DEPENDENCE-ADJUSTED MULTIPLE TESTING

Model Averaging for Prediction with Fragmentary Data

An effective method for weighted support vector regression based on sample simplification

A Statistical Perspective of Sampling Scores for Linear Regression.

Optimal Subsampling for Large Sample Logistic Regression

A Large Sample Study of Randomly Weighted Bootstrap in Linear Models

Subsampling for Big Data Linear Models with Measurement Errors

Sample size determination for multidimensional parameters and the A-optimal subsampling in a big data linear regression model

Improving Load Forecasting Performance via Sample Reweighting

An improved sample size calculation method for score tests in generalized linear models

Nearly optimal capture-recapture sampling and empirical likelihood weighting estimation for M-estimation with big data

Revisiting the Effective Number Theory for Imbalanced Learning

A solution to minimum sample size for regressions