Identification of Graphical Models for Nonignorable Nonresponse in Terms of Information on Odds Ratios

WANG Xue-li,GENG Zhi

DOI: https://doi.org/10.3969/j.issn.1002-1566.2005.03.011

2005-01-01

Abstract:In the case with missing data from a mechanism of nonignorable nonresponse, a graphical model usually is not identifiable. In this paper we present a method for identiying graphical models in terms of external information on odds ratios.

What problem does this paper attempt to address?

Identification Of Graphical Models For Nonignorable Nonresponse Of Binary Outcomes In Longitudinal Studies

Wq Ma,Z Geng,Yh Hu

DOI: https://doi.org/10.1016/S0047-259X(03)00043-5

IF: 1.387

2003-01-01

Journal of Multivariate Analysis

Abstract:In this paper, we use directed acyclic graphs (DAGs) with temporal structure to describe models of nonignorable nonresponse mechanisms for binary outcomes in longitudinal studies, and we discuss identification of these models under an assumption that the sequence of variables has the first-order Markov dependence, that is, the future variables are independent of the past variables conditional on the present variables. We give a stepwise approach for checking identifiability of DAG models. For an unidentifiable model, we propose adding completely observed variables such that this model becomes identifiable. (C) 2003 Elsevier Inc. All rights reserved.
Identification of Nonresponse Mechanisms for Two-Way Contingency Tables

Wen-Qing Ma,Zhi Geng,Xiao-Tong Li

DOI: https://doi.org/10.2333/bhmk.30.125

2003-01-01

Behaviormetrika

Abstract:In this paper, we use decomposable graphical models to describe the mechanisms of nonresponse in contingency tables classified by two binary variables, and we discuss identification of parameters in these models. For an unidentifiable model, we introduce covariates which are always observed such that this model becomes identifiable. We also give conditions for identifiability of odds ratios. These results are useful not only for data analysis but also for study design.
Identifiability of Normal and Normal Mixture Models With Nonignorable Missing Data

Wang Miao,Peng Ding,Zhi Geng

DOI: https://doi.org/10.48550/arXiv.1509.03860

2015-09-13

Abstract:Missing data problems arise in many applied research studies. They may jeopardize statistical inference of the model of interest, if the missing mechanism is nonignorable, that is, the missing mechanism depends on the missing values themselves even conditional on the observed data. With a nonignorable missing mechanism, the model of interest is often not identifiable without imposing further assumptions. We find that even if the missing mechanism has a known parametric form, the model is not identifiable without specifying a parametric outcome distribution. Although it is fundamental for valid statistical inference, identifiability under nonignorable missing mechanisms is not established for many commonly-used models. In this paper, we first demonstrate identifiability of the normal distribution under monotone missing mechanisms. We then extend it to the normal mixture and $t$ mixture models with non-monotone missing mechanisms. We discover that models under the Logistic missing mechanism are less identifiable than those under the Probit missing mechanism. We give necessary and sufficient conditions for identifiability of models under the Logistic missing mechanism, which sometimes can be checked in real data analysis. We illustrate our methods using a series of simulations, and apply them to a real-life dataset.

Statistics Theory,Applications
Using auxiliary data for binomial parameter estimation with nonignorable nonresponse

Xueli Wang,Hua Chen,Zhi Geng,Xiaohua Zhou

DOI: https://doi.org/10.1080/03610926.2011.620209

2012-01-01

Abstract:Nonignorable nonresponse is a nonresponse mechanism that depends on the values of the variable having nonresponse. When an observed data of a binomial distribution suffer missing values from a nonignorable nonresponse mechanism, the binomial distribution parameters become unidentifiable without any other auxiliary information or assumption. To address the problems of non identifiability, existing methods mostly based on the log-linear regression model. In this article, we focus on the model when the nonresponse is nonignorable and we consider to use the auxiliary data to improve identifiability; furthermore, we derive the maximum likelihood estimator (MLE) for the binomial proportion and its associated variance. We present results for an analysis of real-life data from the SARS study in China. Finally, the simulation study shows that the proposed method gives promising results.
Identification Problem for The Analysis of Binary Data with Non-ignorable Missing

Kosuke Morikawa,Yutaka Kano

DOI: https://doi.org/10.48550/arXiv.1405.3380

2014-05-14

Abstract:When a missing-data mechanism is NMAR or non-ignorable, missingness is itself vital information and it must be taken into the likelihood, which, however, needs to introduce additional parameters to be estimated. The incompleteness of the data and introduction of more parameters can cause the identification problem. When a response variable is binary, it becomes a more serious problem because of less information of bi- nary data, however, there are no methods to briefly verify whether a mode is identified or not. Therefore, we provide a new necessary and sufficient condition to easily check model identifiability when analyzing binary data with non-ignorable missing by condi- tional models. This condition can give us what condition is needed for a model to have identifiability as well as make easily check the identifiability of a model.

Methodology
Identification enhanced generalised linear model estimation with nonignorable missing outcomes

Kenji Beppu,Jinung Choi,Kosuke Morikawa,Jongho Im

2024-10-01

Abstract:Missing data often result in undesirable bias and loss of efficiency. These become substantial problems when the response mechanism is nonignorable, such that the response model depends on unobserved variables. It is necessary to estimate the joint distribution of unobserved variables and response indicators to manage nonignorable nonresponse. However, model misspecification and identification issues prevent robust estimates despite careful estimation of the target joint distribution. In this study, we modelled the distribution of the observed parts and derived sufficient conditions for model identifiability, assuming a logistic regression model as the response mechanism and generalised linear models as the main outcome model of interest. More importantly, the derived sufficient conditions are testable with the observed data and do not require any instrumental variables, which are often assumed to guarantee model identifiability but cannot be practically determined beforehand. To analyse missing data, we propose a new imputation method which incorporates verifiable identifiability using only observed data. Furthermore, we present the performance of the proposed estimators in numerical studies and apply the proposed method to two sets of real data: exit polls for the 19th South Korean election data and public data collected from the Korean Survey of Household Finances and Living Conditions.

Methodology,Statistics Theory
Model-based Inference for Two-Stage Cluster Samples Subject to Nonignorable Item Nonresponse

Ying Yuan,Roderick J. A. Little

2009-01-01

Quality Engineering

Abstract:This article concerns item nonresponse adjustment for two-stage cluster samples, where nonresponse depends on covariates and underlying cluster characteristics, or on covariates and the missing outcome. In these circumstances, standard weighting and imputation adjustments are liable to be biased. To obtain consistent estimates, we propose extensions of the standard random-effects model for clustered data to model these two types of missing data mechanisms. These new methods are compared with existing approaches by simulation studies, and illustrated on data on household income from the Behavioral Risk Factor Surveillance System.
Mixed‐Effect Hybrid Models for Longitudinal Data with Nonignorable Dropout

Ying Yuan,Roderick J. A. Little

DOI: https://doi.org/10.1111/j.1541-0420.2008.01102.x

IF: 1.701

2008-01-01

Biometrics

Abstract:Summary Selection models and pattern‐mixture models are often used to deal with nonignorable dropout in longitudinal studies. These two classes of models are based on different factorizations of the joint distribution of the outcome process and the dropout process. We consider a new class of models, called mixed‐effect hybrid models (MEHMs), where the joint distribution of the outcome process and dropout process is factorized into the marginal distribution of random effects, the dropout process conditional on random effects, and the outcome process conditional on dropout patterns and random effects. MEHMs combine features of selection models and pattern‐mixture models: they directly model the missingness process as in selection models, and enjoy the computational simplicity of pattern‐mixture models. The MEHM provides a generalization of shared‐parameter models (SPMs) by relaxing the conditional independence assumption between the measurement process and the dropout process given random effects. Because SPMs are nested within MEHMs, likelihood ratio tests can be constructed to evaluate the conditional independence assumption of SPMs. We use data from a pediatric AIDS clinical trial to illustrate the models.
Pattern graphs: a graphical approach to nonmonotone missing data

Yen-Chi Chen

DOI: https://doi.org/10.48550/arXiv.2004.00744

2020-12-04

Abstract:We introduce the concept of pattern graphs--directed acyclic graphs representing how response patterns are associated. A pattern graph represents an identifying restriction that is nonparametrically identified/saturated and is often a missing not at random restriction. We introduce a selection model and a pattern mixture model formulations using the pattern graphs and show that they are equivalent. A pattern graph leads to an inverse probability weighting estimator as well as an imputation-based estimator. We also study the semi-parametric efficiency theory and derive a multiply-robust estimator using pattern graphs.

Methodology,Statistics Theory
A New Non‐randomized Model for Analysing Sensitive Questions with Binary Outcomes

Guo-Liang Tian,Jun-Wu Yu,Man-Lai Tang,Zhi Geng

DOI: https://doi.org/10.1002/sim.2863

2007-01-01

Statistics in Medicine

Abstract:We propose a new non‐randomized model for assessing the association of two sensitive questions with binary outcomes. Under the new model, respondents only need to answer a non‐sensitive question instead of the original two sensitive questions. As a result, it can protect a respondent's privacy, avoid the usage of any randomizing device, and be applied to both the face‐to‐face interview and mail questionnaire. We derive the constrained maximum likelihood estimates of the cell probabilities and the odds ratio for two binary variables associated with the sensitive questions via the EM algorithm. The corresponding standard error estimates are then obtained by bootstrap approach. A likelihood ratio test and a chi‐squared test are developed for testing association between the two binary variables. We discuss the loss of information due to the introduction of the non‐sensitive question, and the design of the co‐operative parameters. Simulations are performed to evaluate the empirical type I error rates and powers for the two tests. In addition, a simulation is conducted to study the relationship between the probability of obtaining valid estimates and the sample size for any given cell probability vector. A real data set from an AIDS study is used to illustrate the proposed methodologies. Copyright © 2007 John Wiley & Sons, Ltd.
A Model for Adjusting for Nonignorable Verification Bias in Estimation of the ROC Curve and Its Area with Likelihood‐Based Approach

Danping Liu,Xiao-Hua Zhou

DOI: https://doi.org/10.1111/j.1541-0420.2010.01397.x

IF: 1.701

2010-01-01

Biometrics

Abstract:In estimation of the ROC curve, when the true disease status is subject to nonignorable missingness, the observed likelihood involves the missing mechanism given by a selection model. In this article, we proposed a likelihood-based approach to estimate the ROC curve and the area under the ROC curve when the verification bias is nonignorable. We specified a parametric disease model in order to make the nonignorable selection model identifiable. With the estimated verification and disease probabilities, we constructed four types of empirical estimates of the ROC curve and its area based on imputation and reweighting methods. In practice, a reasonably large sample size is required to estimate the nonignorable selection model in our settings. Simulation studies showed that all four estimators of ROC area performed well, and imputation estimators were generally more efficient than the other estimators proposed. We applied the proposed method to a data set from research in Alzheimer's disease.
Bayesian Modeling Longitudinal Dyadic Data with Nonignorable Dropout, with Application to a Breast Cancer Study

Guangyu Zhang,Ying Yuan

DOI: https://doi.org/10.1214/11-aoas515

2012-01-01

Abstract:Dyadic data are common in the social and behavioral sciences, in which members of dyads are correlated due to the interdependence structure within dyads. The analysis of longitudinal dyadic data becomes complex when nonignorable dropouts occur. We propose a fully Bayesian selection-model-based approach to analyze longitudinal dyadic data with nonignorable dropouts. We model repeated measures on subjects by a transition model and account for within-dyad correlations by random effects. In the model, we allow subject's outcome to depend on his/her own characteristics and measure history, as well as those of the other member in the dyad. We further account for the nonignorable missing data mechanism using a selection model in which the probability of dropout depends on the missing outcome. We propose a Gibbs sampler algorithm to fit the model. Simulation studies show that the proposed method effectively addresses the problem of nonignorable dropouts. We illustrate our methodology using a longitudinal breast cancer study.
Drawing Nomograms with R: Applications to categorical outcome and survival data

Zhongheng Zhang,Michael W. Kattan

DOI: https://doi.org/10.21037/atm.2017.04.01

IF: 3.616

2017-01-01

Annals of Translational Medicine

Abstract:Outcome prediction is a major task in clinical medicine. The standard approach to this work is to collect a variety of predictors and build a model of appropriate type. The model is a mathematical equation that connects the outcome of interest with the predictors. A new patient with given clinical characteristics can be predicted for outcome with this model. However, the equation describing the relationship between predictors and outcome is often complex and the computation requires software for practical use. There is another method called nomogram which is a graphical calculating device allowing an approximate graphical computation of a mathematical function. In this article, we describe how to draw nomograms for various outcomes with nomogram() function. Binary outcome is fit by logistic regression model and the outcome of interest is the probability of the event of interest. Ordinal outcome variable is also discussed. Survival analysis can be fit with parametric model to fully describe the distributions of survival time. Statistics such as the median survival time, survival probability up to a specific time point are taken as the outcome of interest.
Identifiability of Subgroup Causal Effects in Randomized Experiments with Nonignorable Missing Covariates

Peng Ding,Zhi Geng

DOI: https://doi.org/10.1002/sim.6014

2014-01-07

Abstract:Although randomized experiments are widely regarded as the gold standard for estimating causal effects, missing data of the pretreatment covariates makes it challenging to estimate the subgroup causal effects. When the missing data mechanism of the covariates is nonignorable, the parameters of interest are generally not pointly identifiable, and we can only get bounds for the parameters of interest, which may be too wide for practical use. In some real cases, we have prior knowledge that some restrictions may be plausible. We show the identifiability of the causal effects and joint distributions for four interpretable missing data mechanisms, and evaluate the performance of the statistical inference via simulation studies. One application of our methods to a real data set from a randomized clinical trial shows that one of the nonignorable missing data mechanisms fits better than the ignorable missing data mechanism, and the results conform to the study's original expert opinions. We also illustrate the potential applications of our methods to observational studies using a data set from a job-training program.

Statistics Theory,Applications
Nonstandard Conditionally Specified Models for Nonignorable Missing Data.

Alexander M. Franks,Edoardo M. Airoldi,Donald B. Rubin

DOI: https://doi.org/10.1073/pnas.1815563117

IF: 11.1

2020-01-01

Proceedings of the National Academy of Sciences

Abstract:Data analyses typically rely upon assumptions about missingness mechanisms that lead to observed versus missing data. When the data are missing not at random, direct assumptions about the missingness mechanism, and indirect assumptions about the distributions of observed and missing data, are typically untestable. We explore an approach, where the joint distribution of observed data and missing data is specified through non-standard conditional distributions. In this formulation, which traces back to a factorization of the joint distribution, apparently proposed by J.W. Tukey, the modeling assumptions about the conditional factors are either testable or are designed to allow the incorporation of substantive knowledge about the problem at hand, thereby offering a possibly realistic portrayal of the data, both missing and observed. We apply Tukey's conditional representation to exponential family models, and we propose a computationally tractable inferential strategy for this class of models. We illustrate the utility of this approach using high-throughput biological data with missing data that are not missing at random.
Identification And Inference With Nonignorable Missing Covariate Data

Wang Miao,Eric Tchetgen Tchetgen

DOI: https://doi.org/10.5705/ss.202016.0322

IF: 1.4

2018-01-01

Statistica Sinica

Abstract:We study identification of parametric and semiparametric models with missing covariate data. When covariate data are missing not at random, identification is not guaranteed even under fairly restrictive parametric assumptions, a fact that is illustrated with several examples. We propose a general approach to establish identification of parametric and semiparametric models when a covariate is missing not at random. Without auxiliary information about the missingness process, identification of parametric models is strongly dependent on model specification. However, in the presence of a fully observed shadow variable that is correlated with the missing covariate but otherwise independent of the missingness conditional on the covariate, identification is more broadly achievable, including in fairly large semiparametric models. Special consideration is given to the generalized linear models with the missingness process unrestricted. Under such a setting, the outcome model is identified for a number of familiar generalized linear models, and we provide counterexamples when identification fails. For estimation, we describe an inverse probability weighted estimator that incorporates the shadow variable to estimate the propensity score model, and we evaluate its performance via simulations. We further illustrate the shadow variable approach with a data example about home prices in China.
Nonparametric Inference about Mean Functionals of Nonignorable Nonresponse Data Without Identifying the Joint Distribution

Wei Li,Wang Miao,Tchetgen

DOI: https://doi.org/10.1093/jrsssb/qkad047

2023-01-01

Abstract:We consider identification and inference about mean functionals of observed covariates and an outcome variable subject to non-ignorable missingness. By leveraging a shadow variable, we establish a necessary and sufficient condition for identification of the mean functional even if the full data distribution is not identified. We further characterize a necessary condition for n-estimability of the mean functional. This condition naturally strengthens the identifying condition, and it requires the existence of a function as a solution to a representer equation that connects the shadow variable to the mean functional. Solutions to the representer equation may not be unique, which presents substantial challenges for non-parametric estimation, and standard theories for non-parametric sieve estimators are not applicable here. We construct a consistent estimator of the solution set and then adapt the theory of extremum estimators to find from the estimated set a consistent estimator of an appropriately chosen solution. The estimator is asymptotically normal, locally efficient and attains the semi-parametric efficiency bound under certain regularity conditions. We illustrate the proposed approach via simulations and a real data application on home pricing.
Identifiability and Estimation of Causal Effects in Randomized Trials with Noncompliance and Completely Nonignorable Missing Data

Hua Chen,Zhi Geng,Xiao‐Hua Zhou

DOI: https://doi.org/10.1111/j.1541-0420.2008.01120.x

IF: 1.701

2009-01-01

Biometrics

Abstract:Summary In this article, we first study parameter identifiability in randomized clinical trials with noncompliance and missing outcomes. We show that under certain conditions the parameters of interest are identifiable even under different types of completely nonignorable missing data: that is, the missing mechanism depends on the outcome. We then derive their maximum likelihood and moment estimators and evaluate their finite‐sample properties in simulation studies in terms of bias, efficiency, and robustness. Our sensitivity analysis shows that the assumed nonignorable missing‐data model has an important impact on the estimated complier average causal effect (CACE) parameter. Our new method provides some new and useful alternative nonignorable missing‐data models over the existing latent ignorable model, which guarantees parameter identifiability, for estimating the CACE in a randomized clinical trial with noncompliance and missing data.
A Nonparametric Measure of Local Association for two-way Contingency Tables

Francis K.C. Hui,Gery Geenens

DOI: https://doi.org/10.48550/arXiv.1211.3460

2012-11-15

Abstract:In contingency table analysis, the odds ratio is a commonly applied measure used to summarize the degree of association between two categorical variables, say R and S. Suppose now that for each individual in the table, a vector of continuous variables X is also observed. It is then vital to analyze whether and how the degree of association varies with X. In this work, we extend the classical odds ratio to the conditional case, and develop nonparametric estimators of this "pointwise odds ratio" to summarize the strength of local association between R and S given X. To allow for maximum flexibility, we make this extension using kernel regression. We develop confidence intervals based on these nonparametric estimators. We demonstrate via simulation that our pointwise odds ratio estimators can outperform model-based counterparts from logistic regression and GAMs, without the need for a linearity or additivity assumption. Finally, we illustrate its application to a dataset of patients from an intensive care unit (ICU), offering a greater insight into how the association between survival of patients admitted for emergency versus elective reasons varies with the patients' ages.

Methodology,Statistics Theory
New possibilities in identification of binary choice models with fixed effects

Yinchu Zhu

2024-12-03

Abstract:We study the identification of binary choice models with fixed effects. We provide a condition called sign saturation and show that this condition is sufficient for the identification of the model. In particular, we can guarantee identification even with bounded regressors. We also show that without this condition, the model is not identified unless the error distribution belongs to a small class. The same sign saturation condition is also essential for identifying the sign of treatment effects. A test is provided to check the sign saturation condition and can be implemented using existing algorithms for the maximum score estimator.

Econometrics,Statistics Theory

Identification of Graphical Models for Nonignorable Nonresponse in Terms of Information on Odds Ratios

Identification Of Graphical Models For Nonignorable Nonresponse Of Binary Outcomes In Longitudinal Studies

Identification of Nonresponse Mechanisms for Two-Way Contingency Tables

Identifiability of Normal and Normal Mixture Models With Nonignorable Missing Data

Using auxiliary data for binomial parameter estimation with nonignorable nonresponse

Identification Problem for The Analysis of Binary Data with Non-ignorable Missing

Identification enhanced generalised linear model estimation with nonignorable missing outcomes

Model-based Inference for Two-Stage Cluster Samples Subject to Nonignorable Item Nonresponse

Mixed‐Effect Hybrid Models for Longitudinal Data with Nonignorable Dropout

Pattern graphs: a graphical approach to nonmonotone missing data

A New Non‐randomized Model for Analysing Sensitive Questions with Binary Outcomes

A Model for Adjusting for Nonignorable Verification Bias in Estimation of the ROC Curve and Its Area with Likelihood‐Based Approach

Bayesian Modeling Longitudinal Dyadic Data with Nonignorable Dropout, with Application to a Breast Cancer Study

Drawing Nomograms with R: Applications to categorical outcome and survival data

Identifiability of Subgroup Causal Effects in Randomized Experiments with Nonignorable Missing Covariates

Nonstandard Conditionally Specified Models for Nonignorable Missing Data.

Identification And Inference With Nonignorable Missing Covariate Data

Nonparametric Inference about Mean Functionals of Nonignorable Nonresponse Data Without Identifying the Joint Distribution

Identifiability and Estimation of Causal Effects in Randomized Trials with Noncompliance and Completely Nonignorable Missing Data

A Nonparametric Measure of Local Association for two-way Contingency Tables

New possibilities in identification of binary choice models with fixed effects