Cluster:“Finding groups in data”: Cluster analysis extended Rousseeuw et al

Martin Maechler

2019-01-01

Abstract:

What problem does this paper attempt to address?

Data Clustering Based on the Modified Relaxation Cheeger Cut Model

Yang Yu-Fei,Zhou Haojie,Zhou Bo

DOI: https://doi.org/10.1007/s40314-022-01757-x

2022-01-01

Computational and Applied Mathematics

Abstract:Graph-based spectral clustering techniques have been developed rapidly in various applications such as the image processing, the networks analysis and the pattern recognition. The spectral graph technique mainly transforms the original data into the graph partitioning problem and then some fast numerical methods can be employed to solve its relaxed form. This paper proposes a novel graph-based spectral clustering model via modifying the relaxation ratio Cheeger cut (RRCC) model. To be specific, in order to enhance the robustness of the clustering, we propose to replace the $$\ell ^1$$ -norm in the denominator by the $$\ell ^2$$ -norm for the RRCC model. Since the proposed model is the fractional optimization problem, we transform it the difference convex problem. With this transformation, the alternating direction of method of multipliers can be employed to solve it. Experimental comparisons demonstrate the superior clustering capabilities of our proposed model and algorithm when dealing with several benchmark databases.
Cluster Analysis of Medical Research Data using R

C. V. G. Rao,M. S. Rao,Lavanya Pamulaparty

2016-04-22

Abstract:Cluster analysis divides the data into groups that are meaningful, useful or both. It is also used as a starting point for other purposes of data summarization. This paper discuss some very basic algorithms like K-means, Fuzzy C-means, Hierarchical clustering to come up with clusters, and use R data mining tool. The results are tested on the datasets namely Online News Popularity, Iris Data Set and from UCI data repository and mi RNA dataset for medical data analysis. All datasets was analyzed with different clustering algorithms and the figures we are showing is the working of them in R data mining tool. Every algorithm has its uniqueness and antithetical behavior.

Medicine,Computer Science
Model-Based Clustering, Discriminant Analysis, and Density Estimation

Chris Fraley,Adrian E Raftery

DOI: https://doi.org/10.1198/016214502760047131

IF: 4.369

2002-06-01

Journal of the American Statistical Association

Abstract:Cluster analysis is the automated search for groups of related observations in a dataset. Most clustering done in practice is based largely on heuristic but intuitively reasonable procedures, and most clustering methods available in commercial software are also of this type. However, there is little systematic guidance associated with these methods for solving important practical questions that arise in cluster analysis, such as how many clusters are there, which clustering method should be used, and how should outliers be handled. We review a general methodology for model-based clustering that provides a principled statistical approach to these issues. We also show that this can be useful for other problems in multivariate analysis, such as discriminant analysis and multivariate density estimation. We give examples from medical diagnosis, minefield detection, cluster recovery from noisy data, and spatial density estimation. Finally, we mention limitations of the methodology and discuss recent developments in model-based clustering for non-Gaussian data, high-dimensional datasets, large datasets, and Bayesian estimation.

statistics & probability
Clustering populations by mixed linear models

JUN ZHU,WEIR BRUCE S

1994-01-01

Abstract:Two mixed linear models are proposed for grouping populations by adissimilarity coefficent which has two parameters for squared differenceof marginal mean and varialc-3 c) np. nent of int0} ractioa. CLi; to: tr.:: canbe constructed by the mixed linear model approaches for experimentaldata with sampling errors within populations or. with some missingvalues, Unweighted pair-ro ip met: io3 (UPGM) is suoYe3ted aionmethod. Samplin,, variance of e; timat-d di; simijarity coefficient can bedetecting significance of dissimilarity of populaions within specific group. Unbiasedness and efficiency for estimation of dissimilarity coefficients areproved by Monte Carolo simulations, Worked example from cotton yielddata is given for demonstration of the use of these cluster methods.
Clustering transformed compositional data using K-means, with applications in gene expression and bicycle sharing system data

Antoine Godichon-Baggioni,Cathy Maugis-Rabusseau,Andrea Rau

DOI: https://doi.org/10.1080/02664763.2018.1454894

2017-04-20

Abstract:Although there is no shortage of clustering algorithms proposed in the literature, the question of the most relevant strategy for clustering compositional data (i.e., data made up of profiles, whose rows belong to the simplex) remains largely unexplored in cases where the observed value of an observation is equal or close to zero for one or more samples. This work is motivated by the analysis of two sets of compositional data, both focused on the categorization of profiles but arising from considerably different applications: (1) identifying groups of co-expressed genes from high-throughput RNA sequencing data, in which a given gene may be completely silent in one or more experimental conditions; and (2) finding patterns in the usage of stations over the course of one week in the Velib' bicycle sharing system in Paris, France. For both of these applications, we focus on the use of appropriately chosen data transformations, including the Centered Log Ratio and a novel extension we propose called the Log Centered Log Ratio, in conjunction with the K-means algorithm. We use a nonasymptotic penalized criterion, whose penalty is calibrated with the slope heuristics, to select the number of clusters present in the data. Finally, we illustrate the performance of this clustering strategy, which is implemented in the Bioconductor package coseq, on both the gene expression and bicycle sharing system data.

Statistics Theory
Clustering -- Basic concepts and methods

Jan-Oliver Felix Kapp-Joswig,Bettina G. Keller

DOI: https://doi.org/10.48550/arXiv.2212.01248

IF: 5.414

2022-12-01

Machine Learning

Abstract:We review clustering as an analysis tool and the underlying concepts from an introductory perspective. What is clustering and how can clusterings be realised programmatically? How can data be represented and prepared for a clustering task? And how can clustering results be validated? Connectivity-based versus prototype-based approaches are reflected in the context of several popular methods: single-linkage, spectral embedding, k-means, and Gaussian mixtures are discussed as well as the density-based protocols (H)DBSCAN, Jarvis-Patrick, CommonNN, and density-peaks.
How Many Clusters? Which Clustering Method? Answers Via Model-Based Cluster Analysis

C. Fraley,A. E. Raftery

DOI: https://doi.org/10.1093/comjnl/41.8.578

1998-08-01

The Computer Journal

Abstract:We consider the problem of determining the structure of clustered data, without prior knowledge of the number of clusters or any other information about their composition. Data are represented by a mixture model in which each component corresponds to a different cluster. Models with varying geometric properties are obtained through Gaussian components with different parametrizations and cross-cluster constraints. Noise and outliers can be modelled by adding a Poisson process component. Partitions are determined by the expectation-maximization (EM) algorithm for maximum likelihood, with initial values from agglomerative hierarchical clustering. Models are compared using an approximation to the Bayes factor based on the Bayesian information criterion (BIC); unlike significance tests, this allows comparison of more than two models at the same time, and removes the restriction that the models compared be nested. The problems of determining the number of clusters and the clustering method are solved simultaneously by choosing the best model. Moreover, the EM result provides a measure of uncertainty about the associated classification of each data point. Examples are given, showing that this approach can give performance that is much better than standard procedures, which often fail to identify groups that are either overlapping or of varying sizes and shapes.
Hierarchical Cluster Analysis In Clinical Research With Heterogeneous Study Population: Highlighting Its Visualization With R

Zhongheng Zhang,Fionn Murtagh,Sven Van Poucke,Su Lin,Peng Lan

DOI: https://doi.org/10.21037/atm.2017.02.05

IF: 3.616

2017-01-01

Annals of Translational Medicine

Abstract:Big data clinical research typically involves thousands of patients and there are numerous variables available. Conventionally, these variables can be handled by multivariable regression modeling. In this article, the hierarchical cluster analysis (HCA) is introduced. This method is used to explore similarity between observations and/or clusters. The result can be visualized using heat maps and dendrograms. Sometimes, it would be interesting to add scatter plot and smooth lines into the panels of the heat map. The inherent R heatmap package does not provide this function. A series of scatter plots can be created using lattice package, and then background color of each panel is mapped to the regression coefficient by using custom-made panel functions. This is the unique feature of the lattice package. Dendrograms and color keys can be added as the legend elements of the lattice system. The latticeExtra package provides some useful functions for the work.
Cluster analysis: A modern statistical review

Adam Jaeger,David Banks

DOI: https://doi.org/10.1002/wics.1597

2022-08-29

Wiley Interdisciplinary Reviews: Computational Statistics

Abstract:Dendrograms made using different dissimilarity metrics in the cluster analyses Cluster analysis is a big, sprawling field. This review paper cannot hope to fully survey the territory. Instead, it focuses on hierarchical agglomerative clustering, k‐means clustering, mixture models, and then several related topics of which any cluster analysis practitioner should be aware. Even then, this review cannot do justice to the chosen topics. There is a lot of literature, and often it is somewhat ad hoc. That is generally the nature of cluster analysis—each application requires a bespoke analysis. Nonetheless, clustering has proven itself to be incredibly useful as an exploratory data analysis tool in biology, advertising, recommender systems, and genomics. This article is categorized under: Statistical Learning and Exploratory Methods of the Data Sciences > Clustering and Classification
Parametric and non-parametric unsupervised cluster analysis

Stephen J. Roberts

DOI: https://doi.org/10.1016/s0031-3203(96)00079-9

IF: 8

1997-02-01

Pattern Recognition

Abstract:Much work has been published on methods for assessing the probable number of clusters or structures within unknown data sets. This paper aims to look in more detail at two methods, a broad parametric method, based around the assumption of Gaussian clusters and the other a non-parametric method which utilises methods of scale-space filtering to extract robust structures within a data set. It is shown that, whilst both methods are capable of determining cluster validity for data sets in which clusters tend towards a multivariate Gaussian distribution, the parametric method inevitably fails for clusters which have a non-Gaussian structure whilst the scale-space method is more robust.

computer science, artificial intelligence,engineering, electrical & electronic
An Advanced Cluster Analysis Method Based on Statistical Test

XB Gao,HB Ji,L Jie

DOI: https://doi.org/10.1109/icosp.2002.1179981

2002-01-01

Abstract:The paper presents a novel clustering algorithm supervised by statistical tests. It deals simultaneously with three key data analysis problems: cluster tendency; cluster analysis; cluster validity. It provides an effective tool to analyze the validity and reasonableness of unsupervised pattern classification, especially in the case of a large number of samples. The experimental result illustrates its effectiveness.
Clustering of Galaxy Clusters. II - Rare Events in the Cluster Distribution

M PLIONIS,R VALDARNINI,YP JING

DOI: https://doi.org/10.1086/171831

1992-01-01

Abstract:We analyze subsamples of the Abell and ACO cluster catalogs, which are nearly complete, in order to study the large-scale structure traced by rich clusters. We use a variety of statistical techniques, minimal spanning trees, percolation, void probability functions, and cluster alignments, and we compare our findings with that expected from an ensemble of simulated cluster catalogs having the same selection functions and low-order clustering as the real data. The simulations have been built applying the Zel'dovich algorithm to a delta(k) spectrum Gaussian distributed with zero mean and random phases. The power spectrum is chosen so that the perturbed particle distribution has the same two-point correlation function as the real clusters. In this way we can test the reproducibility of the real data statistics in our simulations. We find that the minimal spanning tree distribution reveals features which are not reproduced in our simulations at a greater than or similar to 3 sigma level. Similarly, not reproduced are the shapes of many superclusters, found by a friends of-friends algorithm. These discrepancies are not alleviated when increasing the clustering strength or the large scale power in the simulations. We suggest that the real cluster distribution might be described by a statistic more complicated that a simple Gaussian.
A simulation study of cluster search algorithms in data set generated by Gaussian mixture models

Ryosuke Motegi,Yoichi Seki

2024-07-27

Abstract:Determining the number of clusters is a fundamental issue in data clustering. Several algorithms have been proposed, including centroid-based algorithms using the Euclidean distance and model-based algorithms using a mixture of probability distributions. Among these, greedy algorithms for searching the number of clusters by repeatedly splitting or merging clusters have advantages in terms of computation time for problems with large sample sizes. However, studies comparing these methods in systematic evaluation experiments still need to be included. This study examines centroid- and model-based cluster search algorithms in various cases that Gaussian mixture models (GMMs) can generate. The cases are generated by combining five factors: dimensionality, sample size, the number of clusters, cluster overlap, and covariance type. The results show that some cluster-splitting criteria based on Euclidean distance make unreasonable decisions when clusters overlap. The results also show that model-based algorithms are insensitive to covariance type and cluster overlap compared to the centroid-based method if the sample size is sufficient. Our cluster search implementation codes are available at <a class="link-external link-https" href="https://github.com/lipryou/searchClustK" rel="external noopener nofollow">this https URL</a>

Machine Learning
Clustrophile 2: Guided Visual Clustering Analysis

Marco Cavallo,Çağatay Demiralp

DOI: https://doi.org/10.48550/arXiv.1804.03048

2018-09-08

Abstract:Data clustering is a common unsupervised learning method frequently used in exploratory data analysis. However, identifying relevant structures in unlabeled, high-dimensional data is nontrivial, requiring iterative experimentation with clustering parameters as well as data features and instances. The number of possible clusterings for a typical dataset is vast, and navigating in this vast space is also challenging. The absence of ground-truth labels makes it impossible to define an optimal solution, thus requiring user judgment to establish what can be considered a satisfiable clustering result. Data scientists need adequate interactive tools to effectively explore and navigate the large clustering space so as to improve the effectiveness of exploratory clustering analysis. We introduce \textit{Clustrophile~2}, a new interactive tool for guided clustering analysis. \textit{Clustrophile~2} guides users in clustering-based exploratory analysis, adapts user feedback to improve user guidance, facilitates the interpretation of clusters, and helps quickly reason about differences between clusterings. To this end, \textit{Clustrophile~2} contributes a novel feature, the Clustering Tour, to help users choose clustering parameters and assess the quality of different clustering results in relation to current analysis goals and user expectations. We evaluate \textit{Clustrophile~2} through a user study with 12 data scientists, who used our tool to explore and interpret sub-cohorts in a dataset of Parkinson's disease patients. Results suggest that \textit{Clustrophile~2} improves the speed and effectiveness of exploratory clustering analysis for both experts and non-experts.

Human-Computer Interaction,Artificial Intelligence,Databases,Machine Learning
An algorithm for deciding the number of clusters and validation using simulated data with application to exploring crop population structure

Mark A. Newell,Dianne Cook,Heike Hofmann,Jean-Luc Jannink

DOI: https://doi.org/10.1214/13-AOAS671

2014-01-08

Abstract:A first step in exploring population structure in crop plants and other organisms is to define the number of subpopulations that exist for a given data set. The genetic marker data sets being generated have become increasingly large over time and commonly are of the high-dimension, low sample size (HDLSS) situation. An algorithm for deciding the number of clusters is proposed, and is validated on simulated data sets varying in both the level of structure and the number of clusters covering the range of variation observed empirically. The algorithm was then tested on six empirical data sets across three small grain species. The algorithm uses bootstrapping, three methods of clustering, and defines the optimum number of clusters based on a common criterion, the Hubert's gamma statistic. Validation on simulated sets coupled with testing on empirical sets suggests that the algorithm can be used for a wide variety of genetic data sets.

Applications
ClusterFinder: a fast tool to find cluster structures from pair distribution function data

Andy S. Anker,Ulrik Friis-Jensen,Frederik L. Johansen,Simon J. L Billinge,Kirsten M. Ø. Jensen,Andy S. Anker et al.

DOI: https://doi.org/10.1107/s2053273324001116

2024-02-22

Acta Crystallographica Section A Foundations and Advances

Abstract:An automated high-throughput screening approach is presented for identifying starting structure models for pair distribution function analysis of nanoclusters.

chemistry, multidisciplinary,crystallography
Clusters that are not there: An R tutorial and a Shiny app to quantify a priori inferential risks when using clustering methods

Enrico Toffalini,Filippo Gambarota,Ambra Perugini,Paolo Girardi,Valentina Tobia,Gianmarco Altoè,David Giofrè,Psicostat Core Team,Tommaso Feraco

DOI: https://doi.org/10.1002/ijop.13246

2024-09-21

International Journal of Psychology

Abstract:Clustering methods are increasingly used in social science research. Generally, researchers use them to infer the existence of qualitatively different types of individuals within a larger population, thus unveiling previously "hidden" heterogeneity. Depending on the clustering technique, however, valid inference requires some conditions and assumptions. Common risks include not only failing to detect existing clusters due to a lack of power but also revealing clusters that do not exist in the population. Simple data simulations suggest that under conditions of sample size, number, correlation and skewness of indicators that are frequently encountered in applied psychological research, commonly used clustering methods are at a high risk of detecting clusters that are not there. Generally, this is due to some violations of assumptions that are not usually considered critical in psychology. The present article illustrates a simple R tutorial and a Shiny app (for those who are not familiar with R) that allow researchers to quantify a priori inferential risks when performing clustering methods on their own data. Doing so is suggested as a much‐needed preliminary sanity check, because conditions that inflate the number of detected clusters are very common in applied psychological research scenarios.

psychology, multidisciplinary
clusterMaker: a multi-algorithm clustering plugin for Cytoscape

John H Morris,Leonard Apeltsin,Aaron M Newman,Jan Baumbach,Tobias Wittkop,Gang Su,Gary D Bader,Thomas E Ferrin

DOI: https://doi.org/10.1186/1471-2105-12-436

IF: 3.307

2011-11-09

BMC Bioinformatics

Abstract:Background: In the post-genomic era, the rapid increase in high-throughput data calls for computational tools capable of integrating data of diverse types and facilitating recognition of biologically meaningful patterns within them. For example, protein-protein interaction data sets have been clustered to identify stable complexes, but scientists lack easily accessible tools to facilitate combined analyses of multiple data sets from different types of experiments. Here we present clusterMaker, a Cytoscape plugin that implements several clustering algorithms and provides network, dendrogram, and heat map views of the results. The Cytoscape network is linked to all of the other views, so that a selection in one is immediately reflected in the others. clusterMaker is the first Cytoscape plugin to implement such a wide variety of clustering algorithms and visualizations, including the only implementations of hierarchical clustering, dendrogram plus heat map visualization (tree view), k-means, k-medoid, SCPS, AutoSOME, and native (Java) MCL. Results: Results are presented in the form of three scenarios of use: analysis of protein expression data using a recently published mouse interactome and a mouse microarray data set of nearly one hundred diverse cell/tissue types; the identification of protein complexes in the yeast Saccharomyces cerevisiae; and the cluster analysis of the vicinal oxygen chelate (VOC) enzyme superfamily. For scenario one, we explore functionally enriched mouse interactomes specific to particular cellular phenotypes and apply fuzzy clustering. For scenario two, we explore the prefoldin complex in detail using both physical and genetic interaction clusters. For scenario three, we explore the possible annotation of a protein as a methylmalonyl-CoA epimerase within the VOC superfamily. Cytoscape session files for all three scenarios are provided in the Additional Files section. Conclusions: The Cytoscape plugin clusterMaker provides a number of clustering algorithms and visualizations that can be used independently or in combination for analysis and visualization of biological data sets, and for confirming or generating hypotheses about biological function. Several of these visualizations and algorithms are only available to Cytoscape users through the clusterMaker plugin. clusterMaker is available via the Cytoscape plugin manager.
Robust Landmark Graph-Based Clustering for High-Dimensional Data

Ben Yang,Jinghan Wu,Aoran Sun,Naying Gao,Xuetao Zhang

DOI: https://doi.org/10.1016/j.neucom.2022.05.011

IF: 6

2022-01-01

Neurocomputing

Abstract:High-dimensional data has attracted much attention because it contains more comprehensive information about samples. How to cluster these high-dimensional data has become a crucial topic in unsupervised learning. Existing clustering methods often show limited applicability due to their high computational complexity and low anti-noise ability. To address this issue, we propose a novel robust landmark graph-based clustering algorithm for high-dimensional data (RLGCH), which inherits the advantages of both k-means++ and graph-based clustering by using the results of k-means++ as pseudo labels for landmark graph-based clustering. In particular, RLGCH can achieve more reasonable clustering effectiveness than methods that just operate in the low-dimensional space or the original space since it performs k-means++ in the low-dimensional space and landmark graph-based spectral clustering in the original feature space. To avoid post-processing after optimization, the embedded factor matrix is constrained as an indicator matrix rather than a simple nonnegative matrix. To enhance the clustering robustness, the L2,1-norm is adopted to minimize the error of results between k-means++ and landmark graph-based clustering. To solve the model of RLGCH, we established a novel efficient optimization strategy to obtain all sample categories directly. Combining our clustering model and optimization strategy, the computational complexity is reduced to linear and insensitive to data dimensions. Extensive experiments on seven real-world datasets and sixteen noisy datasets show that compared with other state-of-the-art methods, RLGCH can improve the clustering efficiency and robustness greatly while guaranteeing comparable or even better clustering effectiveness.
Statistical power for cluster analysis

Edwin S. Dalmaijer,Camilla L. Nord,Duncan E. Astle

DOI: https://doi.org/10.1186/s12859-022-04675-1

IF: 3.307

2022-06-02

BMC Bioinformatics

Abstract:Cluster algorithms are gaining in popularity in biomedical research due to their compelling ability to identify discrete subgroups in data, and their increasing accessibility in mainstream software. While guidelines exist for algorithm selection and outcome evaluation, there are no firmly established ways of computing a priori statistical power for cluster analysis. Here, we estimated power and classification accuracy for common analysis pipelines through simulation. We systematically varied subgroup size, number, separation (effect size), and covariance structure. We then subjected generated datasets to dimensionality reduction approaches (none, multi-dimensional scaling, or uniform manifold approximation and projection) and cluster algorithms (k-means, agglomerative hierarchical clustering with Ward or average linkage and Euclidean or cosine distance, HDBSCAN). Finally, we directly compared the statistical power of discrete (k-means), "fuzzy" (c-means), and finite mixture modelling approaches (which include latent class analysis and latent profile analysis).

biochemical research methods,biotechnology & applied microbiology,mathematical & computational biology

Cluster:“Finding groups in data”: Cluster analysis extended Rousseeuw et al

Data Clustering Based on the Modified Relaxation Cheeger Cut Model

Cluster Analysis of Medical Research Data using R

Model-Based Clustering, Discriminant Analysis, and Density Estimation

Clustering populations by mixed linear models

Clustering transformed compositional data using K-means, with applications in gene expression and bicycle sharing system data

Clustering -- Basic concepts and methods

How Many Clusters? Which Clustering Method? Answers Via Model-Based Cluster Analysis

Hierarchical Cluster Analysis In Clinical Research With Heterogeneous Study Population: Highlighting Its Visualization With R

Cluster analysis: A modern statistical review

Parametric and non-parametric unsupervised cluster analysis

An Advanced Cluster Analysis Method Based on Statistical Test

Clustering of Galaxy Clusters. II - Rare Events in the Cluster Distribution

A simulation study of cluster search algorithms in data set generated by Gaussian mixture models

Clustrophile 2: Guided Visual Clustering Analysis

An algorithm for deciding the number of clusters and validation using simulated data with application to exploring crop population structure

ClusterFinder: a fast tool to find cluster structures from pair distribution function data

Clusters that are not there: An R tutorial and a Shiny app to quantify a priori inferential risks when using clustering methods

clusterMaker: a multi-algorithm clustering plugin for Cytoscape

Robust Landmark Graph-Based Clustering for High-Dimensional Data

Statistical power for cluster analysis