A Note on an R^2 Measure for Fixed Effects in the Generalized Linear Mixed Model

Lloyd J. Edwards

DOI: https://doi.org/10.48550/arXiv.1011.0471

2010-11-02

Abstract:Using the LRT statistic, a model R^2 is proposed for the generalized linear mixed model for assessing the association between the correlated outcomes and fixed effects. The R^2 compares the full model to a null model with all fixed effects deleted.

Methodology,Applications

What problem does this paper attempt to address?

The R2D2 Prior for Generalized Linear Mixed Models

Eric Yanchenko,Howard D. Bondell,Brian J. Reich

DOI: https://doi.org/10.1080/00031305.2024.2352010

2024-05-26

The American Statistician

Abstract:In Bayesian analysis, the selection of a prior distribution is typically done by considering each parameter in the model. While this can be convenient, in many scenarios it may be desirable to place a prior on a summary measure of the model instead. In this work, we propose a prior on the model fit, as measured by a Bayesian coefficient of determination ( R2) , which then induces a prior on the individual parameters. We achieve this by placing a beta prior on R 2 and then deriving the induced prior on the global variance parameter for generalized linear mixed models. We derive closed-form expressions in many scenarios and present several approximation strategies when an analytic form is not possible and/or to allow for easier computation. In these situations, we suggest approximating the prior by using a generalized beta prime distribution and provide a simple default prior construction scheme. This approach is quite flexible and can be easily implemented in standard Bayesian software. Lastly, we demonstrate the performance of the method on simulated and real-world data, where the method particularly shines in high-dimensional settings, as well as modeling random effects.

statistics & probability
A Modification of McFadden's $R^2$ for Binary and Ordinal Response Models

Ejike R. Ugba,Jan Gertheiss

DOI: https://doi.org/10.29220/CSAM.2023.30.1.049

2022-04-04

Abstract:A lot of studies on the summary measures of predictive strength of categorical response models consider the likelihood ratio index (LRI), also known as the McFadden-$R^2$, a better option than many other measures. We propose a simple modification of the LRI that adjusts for the effect of the number of response categories on the measure and that also rescales its values, mimicking an underlying latent measure. The modified measure is applicable to both binary and ordinal response models fitted by maximum likelihood. Results from simulation studies and a real data example on the olfactory perception of boar taint show that the proposed measure outperforms most of the widely used goodness-of-fit measures for binary and ordinal models. The proposed $R^2$ interestingly proves quite invariant to an increasing number of response categories of an ordinal model.

Applications,Methodology
CLT FOR HIGH-DIMENSIONALR2STATISTICS UNDERA GENERAL INDEPENDENT COMPONENTS MODEL

Weiming Li,Shizhe Hong

DOI: https://doi.org/10.5705/ss.202022.0068

IF: 1.4

2024-01-01

Statistica Sinica

Abstract:This study establishes a central limit theorem (CLT) forR2statistics ina moderately high-dimensional asymptotic framework. The underlying populationaccommodates a general independent components model, by which our result unifiestwo existing CLTs. Beyond this, the new CLT characterizes the effect of kurtosisof the latent independent components on the fluctuation ofR2statistics. As anapplication, a novel confidence interval is constructed for the coefficient of multiplecorrelation in a high-dimensional linear regression
A RMT-based LM Test for Error Cross-Sectional Independence in Large Heterogeneous Panel Data Models*

Natalia Bailey,Dandan Jiang,Jianfeng Yao

DOI: https://doi.org/10.1080/07474938.2021.2009705

IF: 1.2

2021-01-01

Econometric Reviews

Abstract:This paper introduces a new test for error cross-sectional independence in large panel data models with exogenous regressors having heterogenous slope coefficients. The proposed statistic, LMRMT, is based on the Lagrange Multiplier (LM) principle and the sample correlation matrix R-N of the model's residuals. Since in large panels R-N poorly estimates its population counterpart, results from Random Matrix Theory (RMT) are used to establish the high-dimensional limiting distribution of LMRMT under heteroskedastic normal errors and assuming that both the panel size N and the sample size T grow to infinity in comparable magnitude. Simulation results show that LMRMT is largely correctly sized (except for some small values of N and D. Further, the empirical size and power outcomes show robustness of our statistic to deviations from the assumptions of normality for the error terms and of strict exogeneity for the regressors. The test has comparable small sample properties to related tests in the literature which have been developed under different asymptotic theory.
An Approximate Restricted Likelihood Ratio Test for Variance Components in Generalized Linear Mixed Models

Stephanie T. Chen,Luo Xiao,Ana-Maria Staicu

DOI: https://doi.org/10.48550/arXiv.1906.03320

2019-06-08

Abstract:Generalized linear mixed models (GLMMs) are used to model responses from exponential families with a combination of fixed and random effects. For variance components in GLMMs, we propose an approximate restricted likelihood ratio test that conducts testing on the working responses used in penalized quasi-likelihood estimation. This presents the hypothesis test in terms of normalized responses, allowing for application of existing testing methods for linear mixed models. Our test is flexible, computationally efficient, and outperforms several competitors. We illustrate the utility of the proposed method with an extensive simulation study and two data applications. An R package is provided.

Methodology
Extension of the Hosmer-Lemeshow Goodness of Fit Statistic to Linear Models with Repeated Measurements

R. Li,Z. Su,A. Mendelsohn,E. Gemmen

DOI: https://doi.org/10.1016/j.jval.2013.03.267

IF: 5.156

2013-01-01

Value in Health

Abstract:Studies of health outcomes commonly involve binary measures, assessed multiple times. Although generalized linear mixed (GLIMMIX) models are well suited for analyzing these data, there does not exist a formal statistic to assess the goodness of fit (GOF) for GLIMMIX models. We developed an extension of the Hosmer-Lemeshow GOF test used for logistic regression that can be applied to GLIMMIX models. The correlation among repeated measurements of the binary outcome variable was accounted for by a random effect in the GLIMMIX model. The principles of Hosmer-Lemeshow method were followed. The linear unbiased estimate of dependent variables were transformed to the original probability, sorted from least to largest, and divided into deciles. A Chi-square statistic and corresponding p-value with eight degrees of freedom, was calculated based upon the expected and observed numbers among deciles. The proposed GOF test was validated by a simulation study with 1000 runs generated from logistic regression models with and without random effects. The results were compared with the Hosmer-Lemeshow GOF test in situations where the latter is appropriate. The proposed method was used in the analysis of a comparative effectiveness (CE) study of ophthalmologic treatments for open-angle glaucoma patients. When there was no random effect, the proposed GOF test results from the GLIMMIX procedure were almost identical to those of Hosmer-Lemeshow GOF test from the logistic procedure . With a random effect built in a correctly specified model, the goodness of fit rejection rate was 5.1%, which is close to the nominal level 5%. The proposed test did not indicate lack of fit for the regression models in the CE study. The proposed GOF test provides an assessment of model fit for models with binary outcomes and repeated measurements for predictor variables.
R-squared for Bayesian Regression Models

Andrew Gelman,Ben Goodrich,Jonah Gabry,Aki Vehtari

DOI: https://doi.org/10.1080/00031305.2018.1549100

2019-05-13

The American Statistician

Abstract:The usual definition of R2 (variance of the predicted values divided by the variance of the data) has a problem for Bayesian fits, as the numerator can be larger than the denominator. We propose an alternative definition similar to one that has appeared in the survival analysis literature: the variance of the predicted values divided by the variance of predicted values plus the expected variance of the errors.
CLT for High-Dimensional R2 Statistics under a General Independent Components Model

Weiming Li,Shizhe Hong

DOI: https://doi.org/10.5705/ss.202022.0068

IF: 1.4

2025-01-01

Statistica Sinica

Abstract:This paper establishes a central limit theorem (CLT) for R 2 statistics in a moderately high-dimensional asymptotic framework.The underlying population accommodates a general independent components model, by which our result unifies the two CLTs proposed separately in Zheng et al. (2014) and Guo and Cheng (2021).Beyond this, the new CLT characterizes the impact of kurtosis of the latent independent components on the fluctuation of R 2 statistics.As an application, a novel confidence interval is constructed for the coefficient of multiple correlation in a high-dimensional linear regression.
Reduction of Restricted Maximum Likelihood for Random Coefficient Models

Kurt S. Riedel

DOI: https://doi.org/10.48550/arXiv.1803.03895

2018-03-11

Methodology

Abstract:The restricted maximum likelihood (REML) estimator of the dispersion matrix for random coefficient models is rewritten in terms of the sufficient statistics of the individual regressions.
Coefficients of Determination for Mixed-Effects Models

Dabao Zhang

DOI: https://doi.org/10.48550/arXiv.2007.08675

2022-05-03

Abstract:The coefficient of determination is well defined for linear models and its extension is long wanted for mixed-effects models. We revisit its extension to define measures for proportions of variation explained by the whole model, fixed effects only, and random effects only. We propose to calculate unexplained variations conditional on individual random and/or fixed effects so as to keep individual heterogeneity brought by available predictors. While naturally defined for linear mixed models, these measures can be defined for a generalized linear mixed model using a distance measured along its variance function, accounting for its heteroscedasticity.

Methodology
Statistical Inference for a Robust Measure of Multiple Correlation

Catherine Dehon,Christophe Croux

DOI: https://doi.org/10.1007/978-3-642-57489-4_86

2002-01-01

Abstract:In regression analysis it is standard practice to report an R-squared statistic. This measure of multiple correlation is based on the Least Squares estimator, which is known to be extremely sensitive to outliers. The associated R-squared suffers from the same lack of robustness. Also the value of the F-statistic, used to see whether the explanatory variables have jointly a significant influence on the dependent variable, exhibits similar problems. Many robust estimation procedures are available now, but there has been less emphasis on the development of the inference part and in particular on the study of robust R-squared measures. It is however clear that applied statisticians desire to have the same tools to validate their model as when using the classical least squares estimator. In this note we will study the stability properties of robust measures of multiple correlation, and also investigate the size and power of F-statistics based on them.
A New Test for Random Effects in Linear Mixed Models with Longitudinal Data

Zaixing Li,Lixing Zhu

DOI: https://doi.org/10.1016/j.jspi.2012.06.023

IF: 1.095

2013-01-01

Journal of Statistical Planning and Inference

Abstract:As it is known, testing the existence of random effects is often transferred to testing their zero variances/covariance matrices. It is a nonstandard testing problem because the hypothetical values are on the boundary of the whole space. In the literature, a difference-based test was proposed, which has asymptotically tractable null distribution and is then easy to implement. However, the projection method on which the difference-based test relies may affect and deteriorate its performance when covariates associated with fixed effects and covariates associated with random effects are highly correlated. In the paper, for linear mixed models (LMM) with longitudinal data, a new test is proposed to avoid this problem. The new test is also asymptotically distribution-free and more powerful than the difference-based test, particularly when the above correlation is high. The new test is consistent against all global alternatives and can detect local alternatives converging to the null at a rate as close as to m−1/2 with m being the number of subjects. Simulations are carried out to examine the performance and a real data analysis is performed for illustration.
Selection of Regression Models under Linear Restrictions for Fixed and Random Designs

Sen Tian,Clifford M. Hurvich,Jeffrey S. Simonoff

DOI: https://doi.org/10.48550/arXiv.2009.10029

2020-09-22

Abstract:Many important modeling tasks in linear regression, including variable selection (in which slopes of some predictors are set equal to zero) and simplified models based on sums or differences of predictors (in which slopes of those predictors are set equal to each other, or the negative of each other, respectively), can be viewed as being based on imposing linear restrictions on regression parameters. In this paper, we discuss how such models can be compared using information criteria designed to estimate predictive measures like squared error and Kullback-Leibler (KL) discrepancy, in the presence of either deterministic predictors (fixed-X) or random predictors (random-X). We extend the justifications for existing fixed-X criteria Cp, FPE and AICc, and random-X criteria Sp and RCp, to general linear restrictions. We further propose and justify a KL-based criterion, RAICc, under random-X for variable selection and general linear restrictions. We show in simulations that the use of the KL-based criteria AICc and RAICc results in better predictive performance and sparser solutions than the use of squared error-based criteria, including cross-validation.

Methodology
A Note on Random Effects Growth Curve Models

罗幼喜,田茂再,李翰芳

DOI: https://doi.org/10.3969/j.issn.1001-4268.2012.05.010

2012-01-01

Abstract:The robustness of regression coefficient estimator is a hot topic in regression analysis all the while.Since the response observations are not independent,it is extraordinarily difficult to study this problem for random effects growth curve models,especially when the design matrix is non-full of rank.The paper not only gives the necessary and sufficient conditions under which the generalized least square estimate is identical to the the best linear unbiased estimate when error covariance matrix is an arbitrary positive definite matrix,but also obtains a concise condition under which the generalized least square estimate is identical to the maximum likelihood estimate when the design matrix is full or non-full of rank respectively.In addition,by using of the obtained results,we get some corollaries for the the generalized least square estimate be equal to the maximum likelihood estimate under several common error covariance matrix assumptions.Illustrative examples for the case that the design matrix is full or non-full of rank are also given.
Score-based Tests for Explaining Upper-Level Heterogeneity in Linear Mixed Models

Ting Wang,Edgar C. Merkle,Joaquin A. Anguera,Brandon M. Turner

DOI: https://doi.org/10.3758/s13428-020-01375-7

2020-02-29

Abstract:Cross-level interactions among fixed effects in linear mixed models (also known as multilevel models) are often complicated by the variances stemming from random effects and residuals. When these variances change across clusters, tests of fixed effects (including cross-level interaction terms) are subject to inflated Type I or Type II error. While the impact of variance change/heterogeneity has been noticed in the literature, few methods have been proposed to detect this heterogeneity in a simple, systematic way. In addition, when heterogeneity among clusters is detected, researchers often wish to know which clusters' variances differed from the others. In this study, we utilize a recently-proposed family of score-based tests to distinguish between cross-level interactions and heterogeneity in variance components, also providing information about specific clusters that exhibit heterogeneity. These score-based tests only require estimation of the null model (when variance homogeneity is assumed to hold), and they have been previously applied to psychometric models. We extend the tests to linear mixed models here, detailing their implementation and performance when the data generating model is known. We also include an empirical example illustrating the tests' use in practice.

Methodology
The Equivalence Between Likelihood Ratio Test and F-Test for Testing Variance Component in A Balanced One-Way Random Effects Model

Yan Lu,Guoyi Zhang

DOI: https://doi.org/10.1080/00949650802695664

2010-01-01

Abstract:In mixed linear models, it is frequently of interest to test hypotheses on the variance components. F-test and likelihood ratio test (LRT) are commonly used for such purposes. Current LRTs available in literature are based on limiting distribution theory. With the development of finite sample distribution theory, it becomes possible to derive the exact test for likelihood ratio statistic. In this paper, we consider the problem of testing null hypotheses on the variance component in a one-way balanced random effects model. We use the exact test for the likelihood ratio statistic and compare the performance of F-test and LRT. Simulations provide strong support of the equivalence between these two tests. Furthermore, we prove the equivalence between these two tests mathematically.
Two Competing Linear Random-Effects Models and Their Connections

Changli Lu,Yuqin Sun,Yongge Tian

DOI: https://doi.org/10.1007/s00362-016-0806-3

2016-01-01

Statistical Papers

Abstract:Assume that an observed random vector is represented in two alternative linear random-effects models (LRMs). In this case, the results of statistical inference under the two LRMs are not necessarily the same. The objective of this paper is to study the performance of the best linear unbiased predictors/best linear unbiased estimators (BLUPs/BLUEs) of all unknown parameters in two LRMs under most general assumptions on the covariance matrices of the random-effects terms and error terms. We establish necessary and sufficient conditions for the BLUPs/BLUEs to be equivalent under the two LRMs using some well-known formulas for calculating ranks of matrices and their generalized inverses, and also present various consequences and conclusions on some special special cases of LRMs.
Asymptotic confidence interval for R2 in multiple linear regression

J Dedecker,O Guedj,M L Taupin

2024-01-23

Abstract:Following White's approach of robust multiple linear regression, we give asymptotic confidence intervals for the multiple correlation coefficient R2 under minimal moment conditions. We also give the asymptotic joint distribution of the empirical estimators of the individual R2's. Through different sets of simulations, we show that the procedure is indeed robust (contrary to the procedure involving the near exact distribution of the empirical estimator of R2 is the multivariate Gaussian case) and can be also applied to count linear regression.

Statistics Theory
Marginally Interpretable Generalized Linear Mixed Models

Jeffrey J. Gory,Peter F. Craigmile,Steven N. MacEachern

DOI: https://doi.org/10.48550/arXiv.1610.01526

2016-10-05

Methodology

Abstract:Two popular approaches for relating correlated measurements of a non-Gaussian response variable to a set of predictors are to fit a marginal model using generalized estimating equations and to fit a generalized linear mixed model by introducing latent random variables. The first approach is effective for parameter estimation, but leaves one without a formal model for the data with which to assess quality of fit or make predictions for future observations. The second approach overcomes the deficiencies of the first, but leads to parameter estimates that must be interpreted conditional on the latent variables. Further complicating matters, obtaining marginal summaries from a generalized linear mixed model often requires evaluation of an analytically intractable integral or use of attenuation factors that are not exact. We define a class of marginally interpretable generalized linear mixed models that lead to parameter estimates with a marginal interpretation while maintaining the desirable statistical properties of a conditionally-specified model. We discuss the form of these models under various common link functions and also address computational issues associated with these models. For logistic mixed effects models, we introduce an accurate and efficient method for evaluating the logistic-normal integral.
On asymptotic distributions of several test statistics for familial relatedness in linear mixed models

Nicholas Devogel,Paul L. Auer,Regina Manansala,Tao Wang

DOI: https://doi.org/10.1002/sim.9762

2023-05-11

Statistics in Medicine

Abstract:In this study, the asymptotic distributions of the likelihood ratio test (LRT), the restricted likelihood ratio test (RLRT), the F and the sequence kernel association test (SKAT) statistics for testing an additive effect of the expected familial relatedness (FR) in a linear mixed model are examined based on an eigenvalue approach. First, the covariance structure for modeling the FR effect in a LMM is presented. Then, the multiplicity of eigenvalues for the log‐likelihood and restricted log‐likelihood is established under a replicate family setting and extended to a more general replicate family setting (GRFS) as well. After that, the asymptotic null distributions of LRT, RLRT, F and SKAT statistics under GRFS are derived. The asymptotic null distribution of SKAT for testing genetic rare variants is also constructed. In addition, a simple formula for sample size calculation is provided based on the restricted maximum likelihood estimate of the effect size for the expected FR. Finally, a power comparison of these test statistics on hypothesis test of the expected FR effect is made via simulation. The four test statistics are also applied to a data set from the UK Biobank.

public, environmental & occupational health,medicine, research & experimental,medical informatics,mathematical & computational biology,statistics & probability

A Note on an R^2 Measure for Fixed Effects in the Generalized Linear Mixed Model

The R2D2 Prior for Generalized Linear Mixed Models

A Modification of McFadden's $R^2$ for Binary and Ordinal Response Models

CLT FOR HIGH-DIMENSIONALR2STATISTICS UNDERA GENERAL INDEPENDENT COMPONENTS MODEL

A RMT-based LM Test for Error Cross-Sectional Independence in Large Heterogeneous Panel Data Models*

An Approximate Restricted Likelihood Ratio Test for Variance Components in Generalized Linear Mixed Models

Extension of the Hosmer-Lemeshow Goodness of Fit Statistic to Linear Models with Repeated Measurements

R-squared for Bayesian Regression Models

CLT for High-Dimensional R2 Statistics under a General Independent Components Model

Reduction of Restricted Maximum Likelihood for Random Coefficient Models

Coefficients of Determination for Mixed-Effects Models

Statistical Inference for a Robust Measure of Multiple Correlation

A New Test for Random Effects in Linear Mixed Models with Longitudinal Data

Selection of Regression Models under Linear Restrictions for Fixed and Random Designs

A Note on Random Effects Growth Curve Models

Score-based Tests for Explaining Upper-Level Heterogeneity in Linear Mixed Models

The Equivalence Between Likelihood Ratio Test and F-Test for Testing Variance Component in A Balanced One-Way Random Effects Model

Two Competing Linear Random-Effects Models and Their Connections

Asymptotic confidence interval for R2 in multiple linear regression

Marginally Interpretable Generalized Linear Mixed Models

On asymptotic distributions of several test statistics for familial relatedness in linear mixed models