Abstract:Determining the appropriate sample size (N) for comparative biological experiments is critical for obtaining reliable results. In order to determine the N, the usual approach is to perform a power calculation, which involves understanding the variability between samples and the expected effect size. Here, we focused on bulk RNA-seq experiments, which have become ubiquitous in biology, but which have many unknown or difficult to estimate parameters, and so the required analyses to determine the minimum N is typically lacking. We therefore performed two N=30 profiling studies between wild-type mice and mice in which one copy of a gene had been deleted, to determine how many mice would be required to minimize false positives and to maximize true discoveries found in the N of 30 experiment. Results from experiments with N=4 or less are shown to be highly misleading, given the substantial false positive rate, and the lack of discovery of genes later found with higher N. For a cut-off of 2-fold expression differences, we found that an N of 6-7 mice was required to consistently decrease the false positive rate to below 50%, and that more is always better when it came to discovery rates - an N of 8-12 is significantly better in lowering the false positive rate. A common method to reduce false discovery rate in underpowered experiments is to raise the fold cutoff or increase the stringency of the P-value and include only highly perturbed, highly significant genes. A common method to reduce false discovery rate in underpowered experiments is to raise the fold cutoff or increase the stringency of the P-value and include only highly perturbed, highly significant genes. We show that this strategy is no substitute for increasing the N of the experiment, because it consistently inflated effect sizes and caused a substantial drop in sensitivity. These data should be helpful to others in choosing their Ns, since it's often not practical to do such large studies for every mouse model.

A simple method for assessing sample sizes in microarray experiments

A statistical method for the conservative adjustment of false discovery rate (q-value)

A Gene Selection Method for GeneChip Array Data with Small Sample Sizes

Microarray experimental design: power and sample size considerations

Calculation of the minimum number of replicate spots required for detection of significant gene expression fold change in microarray experiments

Efficiently estimating small p-values in permutation tests using importance sampling and cross-entropy method

A two-step strategy for detecting differential gene expression in cDNA microarray data

Effect of Pooling Samples on the Efficiency of Comparative Studies Using Microarrays.

Optimizing murine sample sizes for RNA-seq studies revealed from large-scale comparative analysis

Computing Power and Sample Size for the False Discovery Rate in Multiple Applications

RnaSeqSampleSize: real data based sample size estimation for RNA sequencing

Fast Approximation of Small P-values in Permutation Tests by Partitioning the Permutations

A simple method for statistical analysis of intensity differences in microarray-derived gene expression data

A two-sample Bayesian t-test for microarray data

A Statistical Framework for the Design of Microarray Experiments and Effective Detection of Differential Gene Expression

Considerations when using the significance analysis of microarrays (SAM) algorithm

Accurate and Fast Small P-Value Estimation for Permutation Tests in High-Throughput Genomic Data Analysis with the Cross-Entropy Method.

Power analysis and sample size estimation for RNA-Seq differential expression

A comparative review of statistical methods for discovering differentially expressed genes in replicated microarray experiments

Sample size and performance estimation for biomarker combinations based on pilot studies with small sample sizes

Determination Of Minimum Training Sample Size For Microarray-Based Cancer Outcome Prediction-An Empirical Assessment