Abstract:Improved procedures, in terms of smaller missed discovery rates (MDR), for performing multiple hypotheses testing with weak and strong control of the family-wise error rate (FWER) or the false discovery rate (FDR) are developed and studied. The improvement over existing procedures such as the Šidák procedure for FWER control and the Benjamini--Hochberg (BH) procedure for FDR control is achieved by exploiting possible differences in the powers of the individual tests. Results signal the need to take into account the powers of the individual tests and to have multiple hypotheses decision functions which are not limited to simply using the individual $p$-values, as is the case, for example, with the Šidák, Bonferroni, or BH procedures. They also enhance understanding of the role of the powers of individual tests, or more precisely the receiver operating characteristic (ROC) functions of decision processes, in the search for better multiple hypotheses testing procedures. A decision-theoretic framework is utilized, and through auxiliary randomizers the procedures could be used with discrete or mixed-type data or with rank-based nonparametric tests. This is in contrast to existing $p$-value based procedures whose theoretical validity is contingent on each of these $p$-value statistics being stochastically equal to or greater than a standard uniform variable under the null hypothesis. Proposed procedures are relevant in the analysis of high-dimensional "large $M$, small $n$" data sets arising in the natural, physical, medical, economic and social sciences, whose generation and creation is accelerated by advances in high-throughput technology, notably, but not limited to, microarray technology.

On Power and Sample Size Computation for Multiple Testing Procedures

On limiting behaviors of stepwise multiple testing procedures

On Asymptotic Behaviors of Stepwise Multiple Testing Procedures

Power-enhanced multiple decision functions controlling family-wise error and false discovery rates

Power and sample size calculations. A review and computer program

Power and Stability Properties of Resampling-Based Multiple Testing Procedures with Applications to Gene Oncology Studies

Precise and accurate power of the rank-sum test for a continuous outcome

Impact of Sample Size and Variability on the Power and Type I Error Rates of Equivalence Tests: A Simulation Study.

A New Multiple Testing Method in the Dependent Case

Optimal exact tests for multiple binary endpoints

A Family of Bayes Multiple Testing Procedures

Optimal weighted Bonferroni tests and their graphical extensions

The Extended Anderson and Hauck Tests and Sample Size Procedures for Equivalence Assessment in Simple Linear Regressions

Are the tests overpowered or underpowered? A unified solution to correctly specify type I errors in design of clinical trials for two sample proportions

Power and Sample Size Calculations for Rerandomization

Assessing Agreement Between Two Methods of Quantitative Measurements: Exact Test Procedure and Sample Size Calculation

Closed-Form Power and Sample Size Calculations for Bayes Factors

More Powerful Multiple Testing in Randomized Experiments with Non-Compliance

Sample size determination based on rank tests in clinical trials.

Power and Sample Size for Dose-Finding Studies with Survival Endpoints under Model Uncertainty.

Comparisons of methods for multiple hypothesis testing in neuropsychological research.