cluster: cluster analysis extended Rousseeuw et al

Martin Maechler

2009-01-01

Abstract:Author: Martin Maechler, based on S original by Peter Rousseeuw, Anja. Struyf@ uia. ua. ac. be and Mia. Hubert@ uia. ua. ac. be, and initial R port by Kurt. Hornik@ R-project. org

What problem does this paper attempt to address?

Clustering populations by mixed linear models

JUN ZHU,WEIR BRUCE S

1994-01-01

Abstract:Two mixed linear models are proposed for grouping populations by adissimilarity coefficent which has two parameters for squared differenceof marginal mean and varialc-3 c) np. nent of int0} ractioa. CLi; to: tr.:: canbe constructed by the mixed linear model approaches for experimentaldata with sampling errors within populations or. with some missingvalues, Unweighted pair-ro ip met: io3 (UPGM) is suoYe3ted aionmethod. Samplin,, variance of e; timat-d di; simijarity coefficient can bedetecting significance of dissimilarity of populaions within specific group. Unbiasedness and efficiency for estimation of dissimilarity coefficients areproved by Monte Carolo simulations, Worked example from cotton yielddata is given for demonstration of the use of these cluster methods.
clusterMaker2: a major update to clusterMaker, a multi-algorithm clustering app for Cytoscape

Maija Utriainen,John H Morris

DOI: https://doi.org/10.1186/s12859-023-05225-z

IF: 3.307

2023-04-05

BMC Bioinformatics

Abstract:Background: Since the initial publication of clusterMaker, the need for tools to analyze large biological datasets has only increased. New datasets are significantly larger than a decade ago, and new experimental techniques such as single-cell transcriptomics continue to drive the need for clustering or classification techniques to focus on portions of datasets of interest. While many libraries and packages exist that implement various algorithms, there remains the need for clustering packages that are easy to use, integrated with visualization of the results, and integrated with other commonly used tools for biological data analysis. clusterMaker2 has added several new algorithms, including two entirely new categories of analyses: node ranking and dimensionality reduction. Furthermore, many of the new algorithms have been implemented using the Cytoscape jobs API, which provides a mechanism for executing remote jobs from within Cytoscape. Together, these advances facilitate meaningful analyses of modern biological datasets despite their ever-increasing size and complexity. Results: The use of clusterMaker2 is exemplified by reanalyzing the yeast heat shock expression experiment that was included in our original paper; however, here we explored this dataset in significantly more detail. Combining this dataset with the yeast protein-protein interaction network from STRING, we were able to perform a variety of analyses and visualizations from within clusterMaker2, including Leiden clustering to break the entire network into smaller clusters, hierarchical clustering to look at the overall expression dataset, dimensionality reduction using UMAP to find correlations between our hierarchical visualization and the UMAP plot, fuzzy clustering, and cluster ranking. Using these techniques, we were able to explore the highest-ranking cluster and determine that it represents a strong contender for proteins working together in response to heat shock. We found a series of clusters that, when re-explored as fuzzy clusters, provide a better presentation of mitochondrial processes. Conclusions: clusterMaker2 represents a significant advance over the previously published version, and most importantly, provides an easy-to-use tool to perform clustering and to visualize clusters within the Cytoscape network context. The new algorithms should be welcome to the large population of Cytoscape users, particularly the new dimensionality reduction and fuzzy clustering techniques.
Hierarchical Cluster Analysis In Clinical Research With Heterogeneous Study Population: Highlighting Its Visualization With R

Zhongheng Zhang,Fionn Murtagh,Sven Van Poucke,Su Lin,Peng Lan

DOI: https://doi.org/10.21037/atm.2017.02.05

IF: 3.616

2017-01-01

Annals of Translational Medicine

Abstract:Big data clinical research typically involves thousands of patients and there are numerous variables available. Conventionally, these variables can be handled by multivariable regression modeling. In this article, the hierarchical cluster analysis (HCA) is introduced. This method is used to explore similarity between observations and/or clusters. The result can be visualized using heat maps and dendrograms. Sometimes, it would be interesting to add scatter plot and smooth lines into the panels of the heat map. The inherent R heatmap package does not provide this function. A series of scatter plots can be created using lattice package, and then background color of each panel is mapped to the regression coefficient by using custom-made panel functions. This is the unique feature of the lattice package. Dendrograms and color keys can be added as the legend elements of the lattice system. The latticeExtra package provides some useful functions for the work.
How Many Clusters? Which Clustering Method? Answers Via Model-Based Cluster Analysis

C. Fraley,A. E. Raftery

DOI: https://doi.org/10.1093/comjnl/41.8.578

1998-08-01

The Computer Journal

Abstract:We consider the problem of determining the structure of clustered data, without prior knowledge of the number of clusters or any other information about their composition. Data are represented by a mixture model in which each component corresponds to a different cluster. Models with varying geometric properties are obtained through Gaussian components with different parametrizations and cross-cluster constraints. Noise and outliers can be modelled by adding a Poisson process component. Partitions are determined by the expectation-maximization (EM) algorithm for maximum likelihood, with initial values from agglomerative hierarchical clustering. Models are compared using an approximation to the Bayes factor based on the Bayesian information criterion (BIC); unlike significance tests, this allows comparison of more than two models at the same time, and removes the restriction that the models compared be nested. The problems of determining the number of clusters and the clustering method are solved simultaneously by choosing the best model. Moreover, the EM result provides a measure of uncertainty about the associated classification of each data point. Examples are given, showing that this approach can give performance that is much better than standard procedures, which often fail to identify groups that are either overlapping or of varying sizes and shapes.
Cluster Analysis of Medical Research Data using R

C. V. G. Rao,M. S. Rao,Lavanya Pamulaparty

2016-04-22

Abstract:Cluster analysis divides the data into groups that are meaningful, useful or both. It is also used as a starting point for other purposes of data summarization. This paper discuss some very basic algorithms like K-means, Fuzzy C-means, Hierarchical clustering to come up with clusters, and use R data mining tool. The results are tested on the datasets namely Online News Popularity, Iris Data Set and from UCI data repository and mi RNA dataset for medical data analysis. All datasets was analyzed with different clustering algorithms and the figures we are showing is the working of them in R data mining tool. Every algorithm has its uniqueness and antithetical behavior.

Medicine,Computer Science
Model-Based Clustering, Discriminant Analysis, and Density Estimation

Chris Fraley,Adrian E Raftery

DOI: https://doi.org/10.1198/016214502760047131

IF: 4.369

2002-06-01

Journal of the American Statistical Association

Abstract:Cluster analysis is the automated search for groups of related observations in a dataset. Most clustering done in practice is based largely on heuristic but intuitively reasonable procedures, and most clustering methods available in commercial software are also of this type. However, there is little systematic guidance associated with these methods for solving important practical questions that arise in cluster analysis, such as how many clusters are there, which clustering method should be used, and how should outliers be handled. We review a general methodology for model-based clustering that provides a principled statistical approach to these issues. We also show that this can be useful for other problems in multivariate analysis, such as discriminant analysis and multivariate density estimation. We give examples from medical diagnosis, minefield detection, cluster recovery from noisy data, and spatial density estimation. Finally, we mention limitations of the methodology and discuss recent developments in model-based clustering for non-Gaussian data, high-dimensional datasets, large datasets, and Bayesian estimation.

statistics & probability
Clustering transformed compositional data using K-means, with applications in gene expression and bicycle sharing system data

Antoine Godichon-Baggioni,Cathy Maugis-Rabusseau,Andrea Rau

DOI: https://doi.org/10.1080/02664763.2018.1454894

2017-04-20

Abstract:Although there is no shortage of clustering algorithms proposed in the literature, the question of the most relevant strategy for clustering compositional data (i.e., data made up of profiles, whose rows belong to the simplex) remains largely unexplored in cases where the observed value of an observation is equal or close to zero for one or more samples. This work is motivated by the analysis of two sets of compositional data, both focused on the categorization of profiles but arising from considerably different applications: (1) identifying groups of co-expressed genes from high-throughput RNA sequencing data, in which a given gene may be completely silent in one or more experimental conditions; and (2) finding patterns in the usage of stations over the course of one week in the Velib' bicycle sharing system in Paris, France. For both of these applications, we focus on the use of appropriately chosen data transformations, including the Centered Log Ratio and a novel extension we propose called the Log Centered Log Ratio, in conjunction with the K-means algorithm. We use a nonasymptotic penalized criterion, whose penalty is calibrated with the slope heuristics, to select the number of clusters present in the data. Finally, we illustrate the performance of this clustering strategy, which is implemented in the Bioconductor package coseq, on both the gene expression and bicycle sharing system data.

Statistics Theory
clusterMaker: a multi-algorithm clustering plugin for Cytoscape

John H Morris,Leonard Apeltsin,Aaron M Newman,Jan Baumbach,Tobias Wittkop,Gang Su,Gary D Bader,Thomas E Ferrin

DOI: https://doi.org/10.1186/1471-2105-12-436

IF: 3.307

2011-11-09

BMC Bioinformatics

Abstract:Background: In the post-genomic era, the rapid increase in high-throughput data calls for computational tools capable of integrating data of diverse types and facilitating recognition of biologically meaningful patterns within them. For example, protein-protein interaction data sets have been clustered to identify stable complexes, but scientists lack easily accessible tools to facilitate combined analyses of multiple data sets from different types of experiments. Here we present clusterMaker, a Cytoscape plugin that implements several clustering algorithms and provides network, dendrogram, and heat map views of the results. The Cytoscape network is linked to all of the other views, so that a selection in one is immediately reflected in the others. clusterMaker is the first Cytoscape plugin to implement such a wide variety of clustering algorithms and visualizations, including the only implementations of hierarchical clustering, dendrogram plus heat map visualization (tree view), k-means, k-medoid, SCPS, AutoSOME, and native (Java) MCL. Results: Results are presented in the form of three scenarios of use: analysis of protein expression data using a recently published mouse interactome and a mouse microarray data set of nearly one hundred diverse cell/tissue types; the identification of protein complexes in the yeast Saccharomyces cerevisiae; and the cluster analysis of the vicinal oxygen chelate (VOC) enzyme superfamily. For scenario one, we explore functionally enriched mouse interactomes specific to particular cellular phenotypes and apply fuzzy clustering. For scenario two, we explore the prefoldin complex in detail using both physical and genetic interaction clusters. For scenario three, we explore the possible annotation of a protein as a methylmalonyl-CoA epimerase within the VOC superfamily. Cytoscape session files for all three scenarios are provided in the Additional Files section. Conclusions: The Cytoscape plugin clusterMaker provides a number of clustering algorithms and visualizations that can be used independently or in combination for analysis and visualization of biological data sets, and for confirming or generating hypotheses about biological function. Several of these visualizations and algorithms are only available to Cytoscape users through the clusterMaker plugin. clusterMaker is available via the Cytoscape plugin manager.
rCOSA: A Software Package for Clustering Objects on Subsets of Attributes

Maarten M. Kampert,Jacqueline J. Meulman,Jerome H. Friedman

DOI: https://doi.org/10.48550/arXiv.1612.00259

2016-11-30

Abstract:\texttt{rCOSA} is a software package interfaced to the R language. It implements statistical techniques for clustering objects on subsets of attributes in multivariate data. The main output of COSA is a dissimilarity matrix that one can subsequently analyze with a variety of proximity analysis methods. Our package extends the original COSA software (Friedman and Meulman, 2004) by adding functions for hierarchical clustering methods, least squares multidimensional scaling, partitional clustering, and data visualization. In the many publications that cite the COSA paper by Friedman and Meulman (2004), the COSA program is actually used only a small number of times. This can be attributed to the fact that thse original implementation is not very easy to install and use. Moreover, the available software is out-of-date. Here, we introduce an up-to-date software package and a clear guidance for this advanced technique. The software package and related links are available for free at: \url{<a class="link-external link-https" href="https://github.com/mkampert/rCOSA" rel="external noopener nofollow">this https URL</a>}

Computation
A toolbox for fuzzy clustering using the R programming language

Maria Brigida Ferraro,Paolo Giordani

DOI: https://doi.org/10.1016/j.fss.2015.05.001

IF: 4.462

2015-11-01

Fuzzy Sets and Systems

Abstract:Fuzzy clustering is used extensively in several domains of research. In the literature, starting from the well-known fuzzy k-means (fkm) clustering algorithm, an increasing number of papers devoted to fkm and its extensions can be found. Nevertheless, a lack of the related software for implementing these algorithms can be observed preventing their use in practice. Even the standard fkm is not necessarily available in the most common software. For this purpose, a new toolbox for fuzzy clustering using the R programming language is presented by examples. The toolbox, called fclust, contains a suit of fuzzy clustering algorithms, fuzzy cluster validity indices and visualization tools for fuzzy clustering results.

mathematics, applied,statistics & probability,computer science, theory & methods
A unified framework for model-based clustering, linear regression and multiple cluster structure detection

Giuliano Galimberti,Annamaria Manisi,Gabriele Soffritti

DOI: https://doi.org/10.48550/arXiv.1510.03245

2015-10-12

Methodology

Abstract:A general framework for dealing with both linear regression and clustering problems is described. It includes Gaussian clusterwise linear regression analysis with random covariates and cluster analysis via Gaussian mixture models with variable selection. It also admits a novel approach for detecting multiple clusterings from possibly correlated sub-vectors of variables, based on a model defined as the product of conditionally independent Gaussian mixture models. A necessary condition for the identifiability of such a model is provided. The usefulness and effectiveness of the described methodology are illustrated using simulated and real datasets.
Clusters that are not there: An R tutorial and a Shiny app to quantify a priori inferential risks when using clustering methods

Enrico Toffalini,Filippo Gambarota,Ambra Perugini,Paolo Girardi,Valentina Tobia,Gianmarco Altoè,David Giofrè,Psicostat Core Team,Tommaso Feraco

DOI: https://doi.org/10.1002/ijop.13246

2024-09-21

International Journal of Psychology

Abstract:Clustering methods are increasingly used in social science research. Generally, researchers use them to infer the existence of qualitatively different types of individuals within a larger population, thus unveiling previously "hidden" heterogeneity. Depending on the clustering technique, however, valid inference requires some conditions and assumptions. Common risks include not only failing to detect existing clusters due to a lack of power but also revealing clusters that do not exist in the population. Simple data simulations suggest that under conditions of sample size, number, correlation and skewness of indicators that are frequently encountered in applied psychological research, commonly used clustering methods are at a high risk of detecting clusters that are not there. Generally, this is due to some violations of assumptions that are not usually considered critical in psychology. The present article illustrates a simple R tutorial and a Shiny app (for those who are not familiar with R) that allow researchers to quantify a priori inferential risks when performing clustering methods on their own data. Doing so is suggested as a much‐needed preliminary sanity check, because conditions that inflate the number of detected clusters are very common in applied psychological research scenarios.

psychology, multidisciplinary
Cluster analysis: A modern statistical review

Adam Jaeger,David Banks

DOI: https://doi.org/10.1002/wics.1597

2022-08-29

Wiley Interdisciplinary Reviews: Computational Statistics

Abstract:Dendrograms made using different dissimilarity metrics in the cluster analyses Cluster analysis is a big, sprawling field. This review paper cannot hope to fully survey the territory. Instead, it focuses on hierarchical agglomerative clustering, k‐means clustering, mixture models, and then several related topics of which any cluster analysis practitioner should be aware. Even then, this review cannot do justice to the chosen topics. There is a lot of literature, and often it is somewhat ad hoc. That is generally the nature of cluster analysis—each application requires a bespoke analysis. Nonetheless, clustering has proven itself to be incredibly useful as an exploratory data analysis tool in biology, advertising, recommender systems, and genomics. This article is categorized under: Statistical Learning and Exploratory Methods of the Data Sciences > Clustering and Classification
Tutorial: a priori estimation of sample size, effect size, and statistical power for cluster analysis, latent class analysis, and multivariate mixture models

Edwin S Dalmaijer

2023-09-02

Abstract:Before embarking on data collection, researchers typically compute how many individual observations they should do. This is vital for doing studies with sufficient statistical power, and often a cornerstone in study pre-registrations and grant applications. For traditional statistical tests, one would typically determine an acceptable level of statistical power, (gu)estimate effect size, and then use both values to compute the required sample size. However, for analyses that identify subgroups, statistical power is harder to establish. Once sample size reaches a sufficient threshold, effect size is primarily determined by the number of measured features and the underlying subgroup separation. As a consequence, a priory computations of statistical power are notoriously complex. In this tutorial, I will provide a roadmap to determining sample size and effect size for analyses that identify subgroups. First, I introduce a procedure that allows researchers to formalise their expectations about effect sizes in their domain of choice, and use this to compute the minimally required number of measured variables. Next, I outline how to establish the minimum sample size in subgroup analyses. Finally, I use simulations to provide a reference table for the most popular subgroup analyses: k-means, Ward agglomerative hierarchical clustering, c-means fuzzy clustering, latent class analysis, latent profile analysis, and Gaussian mixture modelling. The table shows the minimum numbers of observations per expected subgroup (sample size) and features (measured variables) to achieve acceptable statistical power, and can be readily used in study design.

Methodology,Machine Learning,Other Quantitative Biology
RDCI: A novel method of cluster analysis and applications thereof in sample molecular simulations

Theophanes Raptis,Vasilios Raptis

DOI: https://doi.org/10.48550/arXiv.1306.3460

2013-06-15

Abstract:A novel method, termed Reduced Dimensionality Cluster Identification, RDCI, is presented, for the identification and quantitative description of clusters formed by N objects in three dimensional space. The method consists of finding a path, as short as possible, connecting the objects, and then tracking down the size s, of a subgroup i-n, i-n+1, ..., i+n, of 2n+1 < N particles for i varying from n+1 to N-n. Clusters are located where local minima of s(i) occur whereas local maxima serve as delimiters partitioning the path in subsets containing the clusters. Minimal post-processing allows for the removal of outliers on the basis of user-defined criteria and the identification of clearly defined clusters. The advantage of the method is that it requires no predetermined input or criteria of "clusterness" such as number of objects or size of aggregates. Among the numerous possible applications of the method, results are herein reported, from Molecular Dynamics simulations of a binary mixture of Lennard-Jones fluids and of a model polymeric system with a gas-like substance dispersed in it. The method is shown to allow the extraction of meaningful quantitative information regarding the tendency of molecules to cluster or de-cluster and the dynamics of the clustering processes.

Other Condensed Matter,Computational Physics
Assessing Representation Learning and Clustering Algorithms for Computer-Assisted Image Annotation-Simulating and Benchmarking MorphoCluster

Simon-Martin Schröder,Rainer Kiko

DOI: https://doi.org/10.3390/s22072775

2022-04-04

Abstract:Image annotation is a time-consuming and costly task. Previously, we published MorphoCluster as a novel image annotation tool to address problems of conventional, classifier-based image annotation approaches: their limited efficiency, training set bias and lack of novelty detection. MorphoCluster uses clustering and similarity search to enable efficient, computer-assisted image annotation. In this work, we provide a deeper analysis of this approach. We simulate the actions of a MorphoCluster user to avoid extensive manual annotation runs. This simulation is used to test supervised, unsupervised and transfer representation learning approaches. Furthermore, shrunken k-means and partially labeled k-means, two new clustering algorithms that are tailored specifically for the MorphoCluster approach, are compared to the previously used HDBSCAN*. We find that labeled training data improve the image representations, that unsupervised learning beats transfer learning and that all three clustering algorithms are viable options, depending on whether completeness, efficiency or runtime is the priority. The simulation results support our earlier finding that MorphoCluster is very efficient and precise. Within the simulation, more than five objects per simulated click are being annotated with 95% precision.
Clustering -- Basic concepts and methods

Jan-Oliver Felix Kapp-Joswig,Bettina G. Keller

DOI: https://doi.org/10.48550/arXiv.2212.01248

IF: 5.414

2022-12-01

Machine Learning

Abstract:We review clustering as an analysis tool and the underlying concepts from an introductory perspective. What is clustering and how can clusterings be realised programmatically? How can data be represented and prepared for a clustering task? And how can clustering results be validated? Connectivity-based versus prototype-based approaches are reflected in the context of several popular methods: single-linkage, spectral embedding, k-means, and Gaussian mixtures are discussed as well as the density-based protocols (H)DBSCAN, Jarvis-Patrick, CommonNN, and density-peaks.
A Reasonable Rough Approximation For Clustering Web Users

Duoqian Miao,Min Chen,Zhihua Wei,Qiguo Duan

DOI: https://doi.org/10.1007/978-3-540-77028-2_25

2007-01-01

Abstract:Due to the uncertainty in accessing Web pages, analysis of Web logs faces some challenges. Several rough k-means cluster algorithms have been proposed and successfully applied to Web usage mining. However, they did not explain why rough approximations of these cluster algorithms were introduced. This paper analyzes the characteristics of the data in the boundary areas of clusters, and then a rough k-means cluster algorithm based on a reasonable rough approximation (RKMrra) is proposed. Finally RKMrra is applied to Web access logs. In the experiments RKMrra compares to Lingras and West algorithm and Peters algorithm with respect to five characteristics. The results show that RKMrra discovers meaningful clusters of Web users and its rough approximation is more reasonable.
Clustering Multivariate Data using Factor Analytic Bayesian Mixtures with an Unknown Number of Components

Panagiotis Papastamoulis

DOI: https://doi.org/10.1007/s11222-019-09891-z

2019-06-02

Abstract:Recent work on overfitting Bayesian mixtures of distributions offers a powerful framework for clustering multivariate data using a latent Gaussian model which resembles the factor analysis model. The flexibility provided by overfitting mixture models yields a simple and efficient way in order to estimate the unknown number of clusters and model parameters by Markov chain Monte Carlo (MCMC) sampling. The present study extends this approach by considering a set of eight parameterizations, giving rise to parsimonious representations of the covariance matrix per cluster. A Gibbs sampler combined with a prior parallel tempering scheme is implemented in order to approximately sample from the posterior distribution of the overfitting mixture. The parameterization and number of factors is selected according to the Bayesian Information Criterion. Identifiability issues related to label switching are dealt by post-processing the simulated output with the Equivalence Classes Representatives algorithm. The contributed method and software are demonstrated and compared to similar models estimated using the Expectation-Maximization algorithm on simulated and real datasets. The software is available online at <a class="link-external link-https" href="https://CRAN.R-project.org/package=fabMix" rel="external noopener nofollow">this https URL</a>.

Methodology,Computation
Cluster-robust inference: A guide to empirical practice

James G. MacKinnon,Morten Ørregaard Nielsen,Matthew D. Webb

DOI: https://doi.org/10.1016/j.jeconom.2022.04.001

IF: 3.363

2022-05-12

Journal of Econometrics

Abstract:Methods for cluster-robust inference are routinely used in economics and many other disciplines. However, it is only recently that theoretical foundations for the use of these methods in many empirically relevant situations have been developed. In this paper, we use these theoretical results to provide a guide to empirical practice. We do not attempt to present a comprehensive survey of the (very large) literature. Instead, we bridge theory and practice by providing a thorough guide on what to do and why, based on recently available econometric theory and simulation evidence. To practice what we preach, we include an empirical analysis of the effects of the minimum wage on labor supply of teenagers using individual data.

economics,social sciences, mathematical methods,mathematics, interdisciplinary applications

cluster: cluster analysis extended Rousseeuw et al

Clustering populations by mixed linear models

clusterMaker2: a major update to clusterMaker, a multi-algorithm clustering app for Cytoscape

Hierarchical Cluster Analysis In Clinical Research With Heterogeneous Study Population: Highlighting Its Visualization With R

How Many Clusters? Which Clustering Method? Answers Via Model-Based Cluster Analysis

Cluster Analysis of Medical Research Data using R

Model-Based Clustering, Discriminant Analysis, and Density Estimation

Clustering transformed compositional data using K-means, with applications in gene expression and bicycle sharing system data

clusterMaker: a multi-algorithm clustering plugin for Cytoscape

rCOSA: A Software Package for Clustering Objects on Subsets of Attributes

A toolbox for fuzzy clustering using the R programming language

A unified framework for model-based clustering, linear regression and multiple cluster structure detection

Clusters that are not there: An R tutorial and a Shiny app to quantify a priori inferential risks when using clustering methods

Cluster analysis: A modern statistical review

Tutorial: a priori estimation of sample size, effect size, and statistical power for cluster analysis, latent class analysis, and multivariate mixture models

RDCI: A novel method of cluster analysis and applications thereof in sample molecular simulations

Assessing Representation Learning and Clustering Algorithms for Computer-Assisted Image Annotation-Simulating and Benchmarking MorphoCluster

Clustering -- Basic concepts and methods

A Reasonable Rough Approximation For Clustering Web Users

Clustering Multivariate Data using Factor Analytic Bayesian Mixtures with an Unknown Number of Components

Cluster-robust inference: A guide to empirical practice