Abstract:Finite Gaussian mixture models are widely used for model-based clustering of continuous data. Nevertheless, since the number of model parameters scales quadratically with the number of variables, these models can be easily over-parameterized. For this reason, parsimonious models have been developed via covariance matrix decompositions or assuming local independence. However, these remedies do not allow for direct estimation of sparse covariance matrices nor do they take into account that the structure of association among the variables can vary from one cluster to the other. To this end, we introduce mixtures of Gaussian covariance graph models for model-based clustering with sparse covariance matrices. A penalized likelihood approach is employed for estimation and a general penalty term on the graph configurations can be used to induce different levels of sparsity and incorporate prior knowledge. Model estimation is carried out using a structural-EM algorithm for parameters and graph structure estimation, where two alternative strategies based on a genetic algorithm and an efficient stepwise search are proposed for inference. With this approach, sparse component covariance matrices are directly obtained. The framework results in a parsimonious model-based clustering of the data via a flexible model for the within-group joint distribution of the variables. Extensive simulated data experiments and application to illustrative datasets show that the method attains good classification performance and model quality.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to effectively handle the complex correlation structures of high - dimensional continuous data in model - based clustering, especially when the correlation structures may be different among different clusters. Although the traditional finite Gaussian mixture model is widely used in model - based clustering, the number of its parameters grows quadratically with the increase in the number of variables, which easily leads to over - parameterization. In addition, the existing simplified model methods either do not allow the direct estimation of sparse covariance matrices or do not take into account the differences in variable association structures among different clusters. To solve these problems, the paper introduces a mixture model based on the Gaussian covariance graph model for model - based clustering and is able to directly estimate sparse covariance matrices. This method represents the association structures between variables by using a graphical model within each cluster, thus allowing different clusters to have different association patterns. Specifically, the paper adopts the maximum - likelihood estimation method with a penalty term to estimate the model parameters and the graphical structure, where the penalty term can be used to induce different degrees of sparsity and can incorporate prior knowledge. The model estimation adopts a structured EM algorithm, combined with a genetic algorithm or a step - by - step search strategy to estimate the parameters and the graphical structure. In general, the main contribution of the paper lies in providing a new model - based clustering method, which can not only handle the complex correlation structures in high - dimensional data, but also flexibly adapt to the differences in association patterns among different clusters, thereby improving clustering performance and model quality.

Model-based Clustering with Sparse Covariance Matrices

Model-based clustering based on sparse finite Gaussian mixtures

Model-based clustering and classification using mixtures of multivariate skewed power exponential distributions

Improving model choice in classification: an approach based on clustering of covariance matrices

The parsimonious Gaussian mixture models with partitioned parameters and their application in clustering

Conditional mixture modeling and model-based clustering

Model-based clustering via skewed matrix-variate cluster-weighted models

Sparse covariance estimation in logit mixture models

Variational Inference and Sparsity in High-Dimensional Deep Gaussian Mixture Models

Flexible Clustering with a Sparse Mixture of Generalized Hyperbolic Distributions

Bayesian mixtures of common factor analyzers: Model, variational inference, and applications

Gaussian mixture model with an extended ultrametric covariance structure

Covariance Structure Estimation with Laplace Approximation

Modelling local and global phenomena with sparse Gaussian processes

Clustering based on Mixtures of Sparse Gaussian Processes

Sparse estimation of a covariance matrix

Two New Algorithms for Maximum Likelihood Estimation of Sparse Covariance Matrices With Applications to Graphical Modeling

Clustering with the multivariate normal inverse Gaussian distribution

A parallelizable model-based approach for marginal and multivariate clustering

Penalized model-based clustering with cluster-specific diagonal covariance matrices and grouped variables