Abstract:Abstract Background and objectives: Motivated from a study on breast cancer, we consider the problem of evaluating a statistical hypothesis when some model characteristics are potentially non or weakly identifiable from observed data. Such scenarios are common in longitudinal studies for evaluating a covariate effect when dropouts may be informative. The hypothesis related to treatment effects may not be testable nonparametrically, with untestable model restrictions necessary for inference. A popular approach is to assume a finite dimensional model and to fix a minimal set of parameters, called sensitivity parameters, conditional upon which hypothesized parameters are assumed identifiable. These profile models, viewed as a functional of the secondary or sensitivity parameters, form the basis of sensitivity analysis. In the current literature, however, inference regarding significance of the covariates is ad hoc, ignoring that multiple tests are conducted and that the working model may be misspecified. Methods and results: In this paper, we propose conservatively testing the null hypothesis of interest across the support of the sensitivity parameters using an infimum test statistic. The test must account for the facts that inferences are carried out simultaneously across the entire range of the sensitivity parameters and that the model may be misspecified under certain values of the sensitivity parameters, as might occur if a nonignorable model is fit, when, in reality, missingness is ignorable. We characterize the limiting distribution of the statistic as a functional of processes in the sensitivity parameters, which involves a careful theoretical analysis of its behavior under model misspecification. In practice, we suggest a simple yet flexible resampling procedure to implement the infimum test as well as to construct confidence bands for simultaneous pointwise tests across all values of the sensitivity parameters. The methodology's practical utility is illustrated in simulations and an analysis of quality-of-life outcomes from a longitudinal study on breast cancer. Interestingly in this example, a covariate effect on cancer outcomes is detected over a wide range of the sensitivity parameter when a Wald test that assumes identifiability fails. Implications and next steps: The key implication of this work is that the hypothesis of interest can be conservatively evaluated across the support of the sensitivity parameters using an infimum test statistic over the space of these parameters. A key assumption of this paper is that the working model is finite dimensional although the full specification of distribution of the data is not required. The methodology should be broadly applicable to infinite dimensional models, such as survival models for censored data. This extension is the focus of current research work. Citation Information: Cancer Epidemiol Biomarkers Prev 2011;20(10 Suppl):A6.

Reply to Chen et al.: Parametric methods for cluster inference perform worse for two-sided t-tests

Post-clustering difference testing: Valid inference and practical considerations with applications to ecological and biological data

Cluster failure: Why fMRI inferences for spatial extent have inflated false-positive rates

Correcting Classifiers for Sample Selection Bias in Two-Phase Case-Control Studies

Selective Inference for Hierarchical Clustering

FMRI Clustering in AFNI: False-Positive Rates Redux

Reply to Cox et al. and Kessler et al.: Data and code sharing is the way forward for fMRI

Selective inference for k-means clustering

Valid population inference for information-based imaging: From the second-level $t$-test to prevalence inference

Bayesian Hypothesis Tests Using Nonparametric Statistics

The case for balanced hypothesis tests and equal-tailed confidence intervals

Testing for a difference in means of a single feature after clustering

On the logic of hypothesis testing in functional imaging

Abstract A6: A Global Sensitivity Test for Evaluating Hypotheses with Non- and Weakly Identifiable Statistical Models: an Application to Breast Cancer Outcomes

Parametric versus non-parametric statistics in the analysis of randomized trials with non-normally distributed data

Comparison of tests for categorical data from a stratified cluster randomized trial

Analysis of Bayesian posterior significance and effect size indices for the two-sample t-test to support reproducible medical research

Nonparametric Analysis of Clustered Multivariate Data

Data-analytical stability of cluster-wise and peak-wise inference in fMRI data analysis

Model-free selective inference under covariate shift via weighted conformal p-values

On Controlling the False Discovery Rate in Multiple Testing of the Means of Correlated Normals Against Two-Sided Alternatives