Cluster analysis extended

M Maechler, P Rousseeuw, A Struyf, M Hubert, K Hornik, M Studer, P Roudier

2005-01-01

Abstract:

What problem does this paper attempt to address?

Cluster analysis: A modern statistical review

Adam Jaeger,David Banks

DOI: https://doi.org/10.1002/wics.1597

2022-08-29

Wiley Interdisciplinary Reviews: Computational Statistics

Abstract:Dendrograms made using different dissimilarity metrics in the cluster analyses Cluster analysis is a big, sprawling field. This review paper cannot hope to fully survey the territory. Instead, it focuses on hierarchical agglomerative clustering, k‐means clustering, mixture models, and then several related topics of which any cluster analysis practitioner should be aware. Even then, this review cannot do justice to the chosen topics. There is a lot of literature, and often it is somewhat ad hoc. That is generally the nature of cluster analysis—each application requires a bespoke analysis. Nonetheless, clustering has proven itself to be incredibly useful as an exploratory data analysis tool in biology, advertising, recommender systems, and genomics. This article is categorized under: Statistical Learning and Exploratory Methods of the Data Sciences > Clustering and Classification
Model-Based Clustering, Discriminant Analysis, and Density Estimation

Chris Fraley,Adrian E Raftery

DOI: https://doi.org/10.1198/016214502760047131

IF: 4.369

2002-06-01

Journal of the American Statistical Association

Abstract:Cluster analysis is the automated search for groups of related observations in a dataset. Most clustering done in practice is based largely on heuristic but intuitively reasonable procedures, and most clustering methods available in commercial software are also of this type. However, there is little systematic guidance associated with these methods for solving important practical questions that arise in cluster analysis, such as how many clusters are there, which clustering method should be used, and how should outliers be handled. We review a general methodology for model-based clustering that provides a principled statistical approach to these issues. We also show that this can be useful for other problems in multivariate analysis, such as discriminant analysis and multivariate density estimation. We give examples from medical diagnosis, minefield detection, cluster recovery from noisy data, and spatial density estimation. Finally, we mention limitations of the methodology and discuss recent developments in model-based clustering for non-Gaussian data, high-dimensional datasets, large datasets, and Bayesian estimation.

statistics & probability
How Many Clusters? Which Clustering Method? Answers Via Model-Based Cluster Analysis

C. Fraley,A. E. Raftery

DOI: https://doi.org/10.1093/comjnl/41.8.578

1998-08-01

The Computer Journal

Abstract:We consider the problem of determining the structure of clustered data, without prior knowledge of the number of clusters or any other information about their composition. Data are represented by a mixture model in which each component corresponds to a different cluster. Models with varying geometric properties are obtained through Gaussian components with different parametrizations and cross-cluster constraints. Noise and outliers can be modelled by adding a Poisson process component. Partitions are determined by the expectation-maximization (EM) algorithm for maximum likelihood, with initial values from agglomerative hierarchical clustering. Models are compared using an approximation to the Bayes factor based on the Bayesian information criterion (BIC); unlike significance tests, this allows comparison of more than two models at the same time, and removes the restriction that the models compared be nested. The problems of determining the number of clusters and the clustering method are solved simultaneously by choosing the best model. Moreover, the EM result provides a measure of uncertainty about the associated classification of each data point. Examples are given, showing that this approach can give performance that is much better than standard procedures, which often fail to identify groups that are either overlapping or of varying sizes and shapes.
An Advanced Cluster Analysis Method Based on Statistical Test

XB Gao,HB Ji,L Jie

DOI: https://doi.org/10.1109/icosp.2002.1179981

2002-01-01

Abstract:The paper presents a novel clustering algorithm supervised by statistical tests. It deals simultaneously with three key data analysis problems: cluster tendency; cluster analysis; cluster validity. It provides an effective tool to analyze the validity and reasonableness of unsupervised pattern classification, especially in the case of a large number of samples. The experimental result illustrates its effectiveness.
Clustering of Galaxy Clusters. II - Rare Events in the Cluster Distribution

M PLIONIS,R VALDARNINI,YP JING

DOI: https://doi.org/10.1086/171831

1992-01-01

Abstract:We analyze subsamples of the Abell and ACO cluster catalogs, which are nearly complete, in order to study the large-scale structure traced by rich clusters. We use a variety of statistical techniques, minimal spanning trees, percolation, void probability functions, and cluster alignments, and we compare our findings with that expected from an ensemble of simulated cluster catalogs having the same selection functions and low-order clustering as the real data. The simulations have been built applying the Zel'dovich algorithm to a delta(k) spectrum Gaussian distributed with zero mean and random phases. The power spectrum is chosen so that the perturbed particle distribution has the same two-point correlation function as the real clusters. In this way we can test the reproducibility of the real data statistics in our simulations. We find that the minimal spanning tree distribution reveals features which are not reproduced in our simulations at a greater than or similar to 3 sigma level. Similarly, not reproduced are the shapes of many superclusters, found by a friends of-friends algorithm. These discrepancies are not alleviated when increasing the clustering strength or the large scale power in the simulations. We suggest that the real cluster distribution might be described by a statistic more complicated that a simple Gaussian.
Clustering -- Basic concepts and methods

Jan-Oliver Felix Kapp-Joswig,Bettina G. Keller

DOI: https://doi.org/10.48550/arXiv.2212.01248

IF: 5.414

2022-12-01

Machine Learning

Abstract:We review clustering as an analysis tool and the underlying concepts from an introductory perspective. What is clustering and how can clusterings be realised programmatically? How can data be represented and prepared for a clustering task? And how can clustering results be validated? Connectivity-based versus prototype-based approaches are reflected in the context of several popular methods: single-linkage, spectral embedding, k-means, and Gaussian mixtures are discussed as well as the density-based protocols (H)DBSCAN, Jarvis-Patrick, CommonNN, and density-peaks.
clusterMaker2: a major update to clusterMaker, a multi-algorithm clustering app for Cytoscape

Maija Utriainen,John H Morris

DOI: https://doi.org/10.1186/s12859-023-05225-z

IF: 3.307

2023-04-05

BMC Bioinformatics

Abstract:Background: Since the initial publication of clusterMaker, the need for tools to analyze large biological datasets has only increased. New datasets are significantly larger than a decade ago, and new experimental techniques such as single-cell transcriptomics continue to drive the need for clustering or classification techniques to focus on portions of datasets of interest. While many libraries and packages exist that implement various algorithms, there remains the need for clustering packages that are easy to use, integrated with visualization of the results, and integrated with other commonly used tools for biological data analysis. clusterMaker2 has added several new algorithms, including two entirely new categories of analyses: node ranking and dimensionality reduction. Furthermore, many of the new algorithms have been implemented using the Cytoscape jobs API, which provides a mechanism for executing remote jobs from within Cytoscape. Together, these advances facilitate meaningful analyses of modern biological datasets despite their ever-increasing size and complexity. Results: The use of clusterMaker2 is exemplified by reanalyzing the yeast heat shock expression experiment that was included in our original paper; however, here we explored this dataset in significantly more detail. Combining this dataset with the yeast protein-protein interaction network from STRING, we were able to perform a variety of analyses and visualizations from within clusterMaker2, including Leiden clustering to break the entire network into smaller clusters, hierarchical clustering to look at the overall expression dataset, dimensionality reduction using UMAP to find correlations between our hierarchical visualization and the UMAP plot, fuzzy clustering, and cluster ranking. Using these techniques, we were able to explore the highest-ranking cluster and determine that it represents a strong contender for proteins working together in response to heat shock. We found a series of clusters that, when re-explored as fuzzy clusters, provide a better presentation of mitochondrial processes. Conclusions: clusterMaker2 represents a significant advance over the previously published version, and most importantly, provides an easy-to-use tool to perform clustering and to visualize clusters within the Cytoscape network context. The new algorithms should be welcome to the large population of Cytoscape users, particularly the new dimensionality reduction and fuzzy clustering techniques.
ClusterFinder: a fast tool to find cluster structures from pair distribution function data

Andy S. Anker,Ulrik Friis-Jensen,Frederik L. Johansen,Simon J. L Billinge,Kirsten M. Ø. Jensen,Andy S. Anker et al.

DOI: https://doi.org/10.1107/s2053273324001116

2024-02-22

Acta Crystallographica Section A Foundations and Advances

Abstract:An automated high-throughput screening approach is presented for identifying starting structure models for pair distribution function analysis of nanoclusters.

chemistry, multidisciplinary,crystallography
Clustering transformed compositional data using K-means, with applications in gene expression and bicycle sharing system data

Antoine Godichon-Baggioni,Cathy Maugis-Rabusseau,Andrea Rau

DOI: https://doi.org/10.1080/02664763.2018.1454894

2017-04-20

Abstract:Although there is no shortage of clustering algorithms proposed in the literature, the question of the most relevant strategy for clustering compositional data (i.e., data made up of profiles, whose rows belong to the simplex) remains largely unexplored in cases where the observed value of an observation is equal or close to zero for one or more samples. This work is motivated by the analysis of two sets of compositional data, both focused on the categorization of profiles but arising from considerably different applications: (1) identifying groups of co-expressed genes from high-throughput RNA sequencing data, in which a given gene may be completely silent in one or more experimental conditions; and (2) finding patterns in the usage of stations over the course of one week in the Velib' bicycle sharing system in Paris, France. For both of these applications, we focus on the use of appropriately chosen data transformations, including the Centered Log Ratio and a novel extension we propose called the Log Centered Log Ratio, in conjunction with the K-means algorithm. We use a nonasymptotic penalized criterion, whose penalty is calibrated with the slope heuristics, to select the number of clusters present in the data. Finally, we illustrate the performance of this clustering strategy, which is implemented in the Bioconductor package coseq, on both the gene expression and bicycle sharing system data.

Statistics Theory
A unified framework for model-based clustering, linear regression and multiple cluster structure detection

Giuliano Galimberti,Annamaria Manisi,Gabriele Soffritti

DOI: https://doi.org/10.48550/arXiv.1510.03245

2015-10-12

Methodology

Abstract:A general framework for dealing with both linear regression and clustering problems is described. It includes Gaussian clusterwise linear regression analysis with random covariates and cluster analysis via Gaussian mixture models with variable selection. It also admits a novel approach for detecting multiple clusterings from possibly correlated sub-vectors of variables, based on a model defined as the product of conditionally independent Gaussian mixture models. A necessary condition for the identifiability of such a model is provided. The usefulness and effectiveness of the described methodology are illustrated using simulated and real datasets.
Statistical power for cluster analysis

Edwin S. Dalmaijer,Camilla L. Nord,Duncan E. Astle

DOI: https://doi.org/10.1186/s12859-022-04675-1

IF: 3.307

2022-06-02

BMC Bioinformatics

Abstract:Cluster algorithms are gaining in popularity in biomedical research due to their compelling ability to identify discrete subgroups in data, and their increasing accessibility in mainstream software. While guidelines exist for algorithm selection and outcome evaluation, there are no firmly established ways of computing a priori statistical power for cluster analysis. Here, we estimated power and classification accuracy for common analysis pipelines through simulation. We systematically varied subgroup size, number, separation (effect size), and covariance structure. We then subjected generated datasets to dimensionality reduction approaches (none, multi-dimensional scaling, or uniform manifold approximation and projection) and cluster algorithms (k-means, agglomerative hierarchical clustering with Ward or average linkage and Euclidean or cosine distance, HDBSCAN). Finally, we directly compared the statistical power of discrete (k-means), "fuzzy" (c-means), and finite mixture modelling approaches (which include latent class analysis and latent profile analysis).

biochemical research methods,biotechnology & applied microbiology,mathematical & computational biology
clusterMaker: a multi-algorithm clustering plugin for Cytoscape

John H Morris,Leonard Apeltsin,Aaron M Newman,Jan Baumbach,Tobias Wittkop,Gang Su,Gary D Bader,Thomas E Ferrin

DOI: https://doi.org/10.1186/1471-2105-12-436

IF: 3.307

2011-11-09

BMC Bioinformatics

Abstract:Background: In the post-genomic era, the rapid increase in high-throughput data calls for computational tools capable of integrating data of diverse types and facilitating recognition of biologically meaningful patterns within them. For example, protein-protein interaction data sets have been clustered to identify stable complexes, but scientists lack easily accessible tools to facilitate combined analyses of multiple data sets from different types of experiments. Here we present clusterMaker, a Cytoscape plugin that implements several clustering algorithms and provides network, dendrogram, and heat map views of the results. The Cytoscape network is linked to all of the other views, so that a selection in one is immediately reflected in the others. clusterMaker is the first Cytoscape plugin to implement such a wide variety of clustering algorithms and visualizations, including the only implementations of hierarchical clustering, dendrogram plus heat map visualization (tree view), k-means, k-medoid, SCPS, AutoSOME, and native (Java) MCL. Results: Results are presented in the form of three scenarios of use: analysis of protein expression data using a recently published mouse interactome and a mouse microarray data set of nearly one hundred diverse cell/tissue types; the identification of protein complexes in the yeast Saccharomyces cerevisiae; and the cluster analysis of the vicinal oxygen chelate (VOC) enzyme superfamily. For scenario one, we explore functionally enriched mouse interactomes specific to particular cellular phenotypes and apply fuzzy clustering. For scenario two, we explore the prefoldin complex in detail using both physical and genetic interaction clusters. For scenario three, we explore the possible annotation of a protein as a methylmalonyl-CoA epimerase within the VOC superfamily. Cytoscape session files for all three scenarios are provided in the Additional Files section. Conclusions: The Cytoscape plugin clusterMaker provides a number of clustering algorithms and visualizations that can be used independently or in combination for analysis and visualization of biological data sets, and for confirming or generating hypotheses about biological function. Several of these visualizations and algorithms are only available to Cytoscape users through the clusterMaker plugin. clusterMaker is available via the Cytoscape plugin manager.
An Investigation into Distance Measures in Cluster Analysis

Zoe Shapcott

DOI: https://doi.org/10.48550/arXiv.2404.13664

2024-04-21

Abstract:This report provides an exploration of different distance measures that can be used with the $K$-means algorithm for cluster analysis. Specifically, we investigate the Mahalanobis distance, and critically assess any benefits it may have over the more traditional measures of the Euclidean, Manhattan and Maximum distances. We perform this by first defining the metrics, before considering their advantages and drawbacks as discussed in literature regarding this area. We apply these distances, first to some simulated data and then to subsets of the Dry Bean dataset [1], to explore if there is a better quality detectable for one metric over the others in these cases. One of the sections is devoted to analysing the information obtained from ChatGPT in response to prompts relating to this topic.

Other Statistics
Non-Redundant Subspace Clusterings with Nr-Kmeans and Nr-DipMeans

Dominik Mautz,Wei Ye,Claudia Plant,Christian Böhm

DOI: https://doi.org/10.1145/3385652

IF: 4.157

2020-08-21

ACM Transactions on Knowledge Discovery from Data

Abstract:A huge object collection in high-dimensional space can often be clustered in more than one way, for instance, objects could be clustered by their shape or alternatively by their color. Each grouping represents a different view of the dataset. The new research field of non-redundant clustering addresses this class of problems. In this article, we follow the approach that different, non-redundant k -means-like clusterings may exist in different, arbitrarily oriented subspaces of the high-dimensional space. We assume that these subspaces (and optionally a further noise space without any cluster structure) are orthogonal to each other. This assumption enables a particularly rigorous mathematical treatment of the non-redundant clustering problem and thus a particularly efficient algorithm, which we call N r -K means (for non-redundant k -means). The superiority of our algorithm is demonstrated both theoretically, as well as in extensive experiments. Further, we propose an extension of N r -K means that harnesses Hartigan’s dip test to identify the number of clusters for each subspace automatically.

computer science, information systems, software engineering
Clustrophile 2: Guided Visual Clustering Analysis

Marco Cavallo,Çağatay Demiralp

DOI: https://doi.org/10.48550/arXiv.1804.03048

2018-09-08

Abstract:Data clustering is a common unsupervised learning method frequently used in exploratory data analysis. However, identifying relevant structures in unlabeled, high-dimensional data is nontrivial, requiring iterative experimentation with clustering parameters as well as data features and instances. The number of possible clusterings for a typical dataset is vast, and navigating in this vast space is also challenging. The absence of ground-truth labels makes it impossible to define an optimal solution, thus requiring user judgment to establish what can be considered a satisfiable clustering result. Data scientists need adequate interactive tools to effectively explore and navigate the large clustering space so as to improve the effectiveness of exploratory clustering analysis. We introduce \textit{Clustrophile~2}, a new interactive tool for guided clustering analysis. \textit{Clustrophile~2} guides users in clustering-based exploratory analysis, adapts user feedback to improve user guidance, facilitates the interpretation of clusters, and helps quickly reason about differences between clusterings. To this end, \textit{Clustrophile~2} contributes a novel feature, the Clustering Tour, to help users choose clustering parameters and assess the quality of different clustering results in relation to current analysis goals and user expectations. We evaluate \textit{Clustrophile~2} through a user study with 12 data scientists, who used our tool to explore and interpret sub-cohorts in a dataset of Parkinson's disease patients. Results suggest that \textit{Clustrophile~2} improves the speed and effectiveness of exploratory clustering analysis for both experts and non-experts.

Human-Computer Interaction,Artificial Intelligence,Databases,Machine Learning
Simcluster: clustering enumeration gene expression data on the simplex space

Ricardo ZN Vêncio,Leonardo Varuzza,Carlos AB Pereira,Helena Brentani,Ilya Shmulevich

DOI: https://doi.org/10.1186/1471-2105-8-246

2007-06-23

Abstract:Transcript enumeration methods such as SAGE, MPSS, and sequencing-by-synthesis EST ``digital northern'', are important high-throughput techniques for digital gene expression measurement. As other counting or voting processes, these measurements constitute compositional data exhibiting properties particular to the simplex space where the summation of the components is constrained. These properties are not present on regular Euclidean spaces, on which hybridization-based microarray data is often modeled. Therefore, pattern recognition methods commonly used for microarray data analysis may be non-informative for the data generated by transcript enumeration techniques since they ignore certain fundamental properties of this space. Here we present a software tool, Simcluster, designed to perform clustering analysis for data on the simplex space. We present Simcluster as a stand-alone command-line C package and as a user-friendly on-line tool. Both versions are available at: <a class="link-external link-http" href="http://xerad.systemsbiology.net/simcluster/" rel="external noopener nofollow">this http URL</a> . Simcluster is designed in accordance with a well-established mathematical framework for compositional data analysis, which provides principled procedures for dealing with the simplex space, and is thus applicable in a number of contexts, including enumeration-based gene expression data.

Quantitative Methods,Tissues and Organs
Cluster expansion methods from physical concepts

Paul E. Lammert,Vincent H. Crespi

DOI: https://doi.org/10.48550/arXiv.2210.10937

2022-10-20

Materials Science

Abstract:The cluster expansion formalism used in materials science is reconstructed on an axiomatic basis with the aims of clarifying underlying concepts and improving computational procedures, and without using conventional cluster functions. Instead, cluster components of configuration functions are defined in an intrinsic manner, which can be viewed as Moebius inversion of conditional expectation. The associated method for fitting a model to a configurational sample is grounded entirely in Hilbert space geometry. By constructing models directly from the given data, we avoid an underdetermination problem to which the conventional approach is subject. Tensor observables are treated on an equal footing with scalar observables.
Parametric and non-parametric unsupervised cluster analysis

Stephen J. Roberts

DOI: https://doi.org/10.1016/s0031-3203(96)00079-9

IF: 8

1997-02-01

Pattern Recognition

Abstract:Much work has been published on methods for assessing the probable number of clusters or structures within unknown data sets. This paper aims to look in more detail at two methods, a broad parametric method, based around the assumption of Gaussian clusters and the other a non-parametric method which utilises methods of scale-space filtering to extract robust structures within a data set. It is shown that, whilst both methods are capable of determining cluster validity for data sets in which clusters tend towards a multivariate Gaussian distribution, the parametric method inevitably fails for clusters which have a non-Gaussian structure whilst the scale-space method is more robust.

computer science, artificial intelligence,engineering, electrical & electronic
Silhouettes: A graphical aid to the interpretation and validation of cluster analysis

Peter J. Rousseeuw

DOI: https://doi.org/10.1016/0377-0427(87)90125-7

IF: 2.872

1987-11-01

Journal of Computational and Applied Mathematics

Abstract:A new graphical display is proposed for partitioning techniques. Each cluster is represented by a so-called silhouette, which is based on the comparison of its tightness and separation. This silhouette shows which objects lie well within their cluster, and which ones are merely somewhere in between clusters. The entire clustering is displayed by combining the silhouettes into a single plot, allowing an appreciation of the relative quality of the clusters and an overview of the data configuration. The average silhouette width provides an evaluation of clustering validity, and might be used to select an ‘appropriate’ number of clusters.

mathematics, applied
Robust Bregman Clustering

Aurélie Fischer,Clément Levrard,Claire Brécheteau

DOI: https://doi.org/10.48550/arXiv.1812.04356

2020-09-09

Abstract:Using a trimming approach, we investigate a k-means type method based on Bregman divergences for clustering data possibly corrupted with clutter noise. The main interest of Bregman divergences is that the standard Lloyd algorithm adapts to these distortion measures, and they are well-suited for clustering data sampled according to mixture models from exponential families. We prove that there exists an optimal codebook, and that an empirically optimal codebook converges a.s. to an optimal codebook in the distortion sense. Moreover, we obtain the sub-Gaussian rate of convergence for k-means 1 $\sqrt$ n under mild tail assumptions. Also, we derive a Lloyd-type algorithm with a trimming parameter that can be selected from data according to some heuristic, and present some experimental results.

Statistics Theory,Machine Learning

Cluster analysis extended

Cluster analysis: A modern statistical review

Model-Based Clustering, Discriminant Analysis, and Density Estimation

How Many Clusters? Which Clustering Method? Answers Via Model-Based Cluster Analysis

An Advanced Cluster Analysis Method Based on Statistical Test

Clustering of Galaxy Clusters. II - Rare Events in the Cluster Distribution

Clustering -- Basic concepts and methods

clusterMaker2: a major update to clusterMaker, a multi-algorithm clustering app for Cytoscape

ClusterFinder: a fast tool to find cluster structures from pair distribution function data

Clustering transformed compositional data using K-means, with applications in gene expression and bicycle sharing system data

A unified framework for model-based clustering, linear regression and multiple cluster structure detection

Statistical power for cluster analysis

clusterMaker: a multi-algorithm clustering plugin for Cytoscape

An Investigation into Distance Measures in Cluster Analysis

Non-Redundant Subspace Clusterings with Nr-Kmeans and Nr-DipMeans

Clustrophile 2: Guided Visual Clustering Analysis

Simcluster: clustering enumeration gene expression data on the simplex space

Cluster expansion methods from physical concepts

Parametric and non-parametric unsupervised cluster analysis

Silhouettes: A graphical aid to the interpretation and validation of cluster analysis

Robust Bregman Clustering