Comparing the performance of biomedical clustering methods

Christian Wiwie,Jan Baumbach,Richard Röttger

DOI: https://doi.org/10.1038/nmeth.3583

IF: 48

2015-09-21

Nature Methods

Abstract:This analysis compares 13 clustering methods on 24 data sets using the freely accessible ClustEval platform and presents guidelines for how to choose tools and set parameters for clustering biomedical data

biochemical research methods

What problem does this paper attempt to address?

Comparison of Methods for Biological Sequence Clustering.

Ze-Gang Wei,Xu Chen,Xiao-Dan Zhang,Hao Zhang,Xing-Guo Fan,Hong-Yan Gao,Fei Liu,Yu Qian

DOI: https://doi.org/10.1109/tcbb.2023.3253138

2023-01-01

IEEE/ACM Transactions on Computational Biology and Bioinformatics

Abstract:Recent advances in sequencing technology have considerably promoted genomics research by providing high-throughput sequencing economically. This great advancement has resulted in a huge amount of sequencing data. Clustering analysis is powerful to study and probe the large-scale sequence data. A number of available clustering methods have been developed in the last decade. Despite numerous comparison studies being published, we noticed that they have two main limitations: only traditional alignment-based clustering methods are compared and the evaluation metrics heavily rely on labeled sequence data. In this study, we present a comprehensive benchmark study for sequence clustering methods. Specifically, i) alignment-based clustering algorithms including classical (e.g., CD-HIT, UCLUST, VSEARCH) and recently proposed methods (e.g., MMseq2, Linclust, edClust) are assessed; ii) two alignment-free methods (e.g., LZW-Kernel and Mash) are included to compare with alignment-based methods; and iii) different evaluation measures based on the true labels (supervised metrics) and the input data itself (unsupervised metrics) are applied to quantify their clustering results. The aims of this study are to help biological analyzers in choosing one reasonable clustering algorithm for processing their collected sequences, and furthermore, motivate algorithm designers to develop more efficient sequence clustering approaches.
Clustering cancer gene expression data: a comparative study

Marcilio CP de Souto,Ivan G Costa,Daniel SA de Araujo,Teresa B Ludermir,Alexander Schliep

DOI: https://doi.org/10.1186/1471-2105-9-497

IF: 3.307

2008-11-27

BMC Bioinformatics

Abstract:BackgroundThe use of clustering methods for the discovery of cancer subtypes has drawn a great deal of attention in the scientific community. While bioinformaticians have proposed new clustering methods that take advantage of characteristics of the gene expression data, the medical community has a preference for using "classic" clustering methods. There have been no studies thus far performing a large-scale evaluation of different clustering methods in this context.Results/ConclusionWe present the first large-scale analysis of seven different clustering methods and four proximity measures for the analysis of 35 cancer gene expression data sets. Our results reveal that the finite mixture of Gaussians, followed closely by k-means, exhibited the best performance in terms of recovering the true structure of the data sets. These methods also exhibited, on average, the smallest difference between the actual number of classes in the data sets and the best number of clusters as indicated by our validation criteria. Furthermore, hierarchical methods, which have been widely used by the medical community, exhibited a poorer recovery performance than that of the other methods evaluated. Moreover, as a stable basis for the assessment and comparison of different clustering methods for cancer gene expression data, this study provides a common group of data sets (benchmark data sets) to be shared among researchers and used for comparisons with new methods. The data sets analyzed in this study are available at http://algorithmics.molgen.mpg.de/Supplements/CompCancer/.

biochemical research methods,biotechnology & applied microbiology,mathematical & computational biology
A comparison framework and guideline of clustering methods for mass cytometry data

Xiao Liu,Weichen Song,Brandon Y. Wong,Ting Zhang,Shunying Yu,Guan Ning Lin,Xianting Ding

DOI: https://doi.org/10.1186/s13059-019-1917-7

IF: 17.906

2019-12-01

Genome Biology

Abstract:Abstract Background With the expanding applications of mass cytometry in medical research, a wide variety of clustering methods, both semi-supervised and unsupervised, have been developed for data analysis. Selecting the optimal clustering method can accelerate the identification of meaningful cell populations. Result To address this issue, we compared three classes of performance measures, “precision” as external evaluation, “coherence” as internal evaluation, and stability, of nine methods based on six independent benchmark datasets. Seven unsupervised methods (Accense, Xshift, PhenoGraph, FlowSOM, flowMeans, DEPECHE, and kmeans) and two semi-supervised methods (Automated Cell-type Discovery and Classification and linear discriminant analysis (LDA)) are tested on six mass cytometry datasets. We compute and compare all defined performance measures against random subsampling, varying sample sizes, and the number of clusters for each method. LDA reproduces the manual labels most precisely but does not rank top in internal evaluation. PhenoGraph and FlowSOM perform better than other unsupervised tools in precision, coherence, and stability. PhenoGraph and Xshift are more robust when detecting refined sub-clusters, whereas DEPECHE and FlowSOM tend to group similar clusters into meta-clusters. The performances of PhenoGraph, Xshift, and flowMeans are impacted by increased sample size, but FlowSOM is relatively stable as sample size increases. Conclusion All the evaluations including precision, coherence, stability, and clustering resolution should be taken into synthetic consideration when choosing an appropriate tool for cytometry data analysis. Thus, we provide decision guidelines based on these characteristics for the general reader to more easily choose the most suitable clustering tools.

genetics & heredity,biotechnology & applied microbiology
Selection of single cell clustering methodologies through rank aggregation of multiple performance measures

Owen Visser,Somnath Datta

2024-11-07

Abstract:As single-cell gene expression data analysis continues to grow, the need for reliable clustering methods has become increasingly important. The prevalence of heuristic means for method choice could lead to inaccurate reports if comprehensive evaluation of the methods is omitted. Typical comparisons of methods fail to address the complexity presented by the data, transformations, or internal parameters. Previous work in the field of microarray data provided measures to evaluate the stability characteristic of clustering algorithms. Additional work on aggregation in the same era presented a way to compare multiple methodologies using several performance measures. In this paper, we provide adaptations to these measures and employ two aggregation schemes to create ranked lists of method and parameter choices for six unique datasets. Our findings demonstrate that an ensemble of validation measures, combined with ranking based on measures' dataset specific preferences, provides an objective way to select clustering methodologies, taking into account characteristic evaluation from each measure.

Quantitative Methods
A systematic performance evaluation of clustering methods for single-cell RNA-seq data

Angelo Duò,Mark D. Robinson,Charlotte Soneson

DOI: https://doi.org/10.12688/f1000research.15666.3

2020-11-16

F1000Research

Abstract:Subpopulation identification, usually via some form of unsupervised clustering, is a fundamental step in the analysis of many single-cell RNA-seq data sets. This has motivated the development and application of a broad range of clustering methods, based on various underlying algorithms. Here, we provide a systematic and extensible performance evaluation of 14 clustering algorithms implemented in R, including both methods developed explicitly for scRNA-seq data and more general-purpose methods. The methods were evaluated using nine publicly available scRNA-seq data sets as well as three simulations with varying degree of cluster separability. The same feature selection approaches were used for all methods, allowing us to focus on the investigation of the performance of the clustering algorithms themselves. We evaluated the ability of recovering known subpopulations, the stability and the run time and scalability of the methods. Additionally, we investigated whether the performance could be improved by generating consensus partitions from multiple individual clustering methods. We found substantial differences in the performance, run time and stability between the methods, with SC3 and Seurat showing the most favorable results. Additionally, we found that consensus clustering typically did not improve the performance compared to the best of the combined methods, but that several of the top-performing methods already perform some type of consensus clustering. All the code used for the evaluation is available on GitHub ( https://github.com/markrobinsonuzh/scRNAseq_clustering_comparison ). In addition, an R package providing access to data and clustering results, thereby facilitating inclusion of new methods and data sets, is available from Bioconductor ( https://bioconductor.org/packages/DuoClustering2018 ).
An empirical comparison and characterisation of nine popular clustering methods

Christian Hennig

DOI: https://doi.org/10.48550/arXiv.2102.03645

2021-02-07

Abstract:Nine popular clustering methods are applied to 42 real data sets. The aim is to give a detailed characterisation of the methods by means of several cluster validation indexes that measure various individual aspects of the resulting clusters such as small within-cluster distances, separation of clusters, closeness to a Gaussian distribution etc. as introduced in Hennig (2019). 30 of the data sets come with a "true" clustering. On these data sets the similarity of the clusterings from the nine methods to the "true" clusterings is explored. Furthermore, a mixed effects regression relates the observable individual aspects of the clusters to the similarity with the "true" clusterings, which in real clustering problems is unobservable. The study gives new insight not only into the ability of the methods to discover "true" clusterings, but also into properties of clusterings that can be expected from the methods, which is crucial for the choice of a method in a real situation without a given "true" clustering.

Methodology
Benchmarking Spatial Clustering Methods with Spatially Resolved Transcriptomics Data

Zhiyuan Yuan,Fangyuan Zhao,Senlin Lin,Yu Zhao,Jianhua Yao,Yan Cui,Xiao-Yong Zhang,Yi Zhao

DOI: https://doi.org/10.1038/s41592-024-02215-8

IF: 48

2024-01-01

Nature Methods

Abstract:Spatial clustering, which shares an analogy with single-cell clustering, has expanded the scope of tissue physiology studies from cell-centroid to structure-centroid with spatially resolved transcriptomics (SRT) data. Computational methods have undergone remarkable development in recent years, but a comprehensive benchmark study is still lacking. Here we present a benchmark study of 13 computational methods on 34 SRT data (7 datasets). The performance was evaluated on the basis of accuracy, spatial continuity, marker genes detection, scalability, and robustness. We found existing methods were complementary in terms of their performance and functionality, and we provide guidance for selecting appropriate methods for given scenarios. On testing additional 22 challenging datasets, we identified challenges in identifying noncontinuous spatial domains and limitations of existing methods, highlighting their inadequacies in handling recent large-scale tasks. Furthermore, with 145 simulated data, we examined the robustness of these methods against four different factors, and assessed the impact of pre- and postprocessing approaches. Our study offers a comprehensive evaluation of existing spatial clustering methods with SRT data, paving the way for future advancements in this rapidly evolving field.
Comparison of clustering tools in R for medium-sized 10x Genomics single-cell RNA-sequencing data

Saskia Freytag,Luyi Tian,Ingrid Lönnstedt,Milica Ng,Melanie Bahlo

DOI: https://doi.org/10.12688/f1000research.15809.2

2018-12-19

F1000Research

Abstract:Background: The commercially available 10x Genomics protocol to generate droplet-based single cell RNA-seq (scRNA-seq) data is enjoying growing popularity among researchers. Fundamental to the analysis of such scRNA-seq data is the ability to cluster similar or same cells into non-overlapping groups. Many competing methods have been proposed for this task, but there is currently little guidance with regards to which method to use. Methods: Here we use one gold standard 10x Genomics dataset, generated from the mixture of three cell lines, as well as multiple silver standard 10x Genomics datasets generated from peripheral blood mononuclear cells to examine not only the accuracy but also running time and robustness of a dozen methods. Results: We found that Seurat outperformed other methods, although performance seems to be dependent on many factors, including the complexity of the studied system. Furthermore, we found that solutions produced by different methods have little in common with each other. Conclusions: In light of this we conclude that the choice of clustering tool crucially determines interpretation of scRNA-seq data generated by 10x Genomics. Hence practitioners and consumers should remain vigilant about the outcome of 10x Genomics scRNA-seq analysis.
Single-cell RNA-seq Data Clustering: A Survey with Performance Comparison Study

Ruiyi Li,Jihong Guan,Shuigeng Zhou

DOI: https://doi.org/10.1142/s0219720020400053

2020-01-01

Journal of Bioinformatics and Computational Biology

Abstract:Clustering analysis has been widely applied to single-cell RNA-sequencing (scRNA-seq) data to discover cell types and cell states. Algorithms developed in recent years have greatly helped the understanding of cellular heterogeneity and the underlying mechanisms of biological processes. However, these algorithms often use different techniques, were evaluated on different datasets and compared with some of their counterparts usually using different performance metrics. Consequently, there lacks an accurate and complete picture of their merits and demerits, which makes it difficult for users to select proper algorithms for analyzing their data. To fill this gap, we first do a review on the major existing scRNA-seq data clustering methods, and then conduct a comprehensive performance comparison among them from multiple perspectives. We consider 13 state of the art scRNA-seq data clustering algorithms, and collect 12 publicly available real scRNA-seq datasets from the existing works to evaluate and compare these algorithms. Our comparative study shows that the existing methods are very diverse in performance. Even the top-performance algorithms do not perform well on all datasets, especially those with complex structures. This suggests that further research is required to explore more stable, accurate, and efficient clustering algorithms for scRNA-seq data.
Statistical power for cluster analysis

Edwin S. Dalmaijer,Camilla L. Nord,Duncan E. Astle

DOI: https://doi.org/10.1186/s12859-022-04675-1

IF: 3.307

2022-06-02

BMC Bioinformatics

Abstract:Cluster algorithms are gaining in popularity in biomedical research due to their compelling ability to identify discrete subgroups in data, and their increasing accessibility in mainstream software. While guidelines exist for algorithm selection and outcome evaluation, there are no firmly established ways of computing a priori statistical power for cluster analysis. Here, we estimated power and classification accuracy for common analysis pipelines through simulation. We systematically varied subgroup size, number, separation (effect size), and covariance structure. We then subjected generated datasets to dimensionality reduction approaches (none, multi-dimensional scaling, or uniform manifold approximation and projection) and cluster algorithms (k-means, agglomerative hierarchical clustering with Ward or average linkage and Euclidean or cosine distance, HDBSCAN). Finally, we directly compared the statistical power of discrete (k-means), "fuzzy" (c-means), and finite mixture modelling approaches (which include latent class analysis and latent profile analysis).

biochemical research methods,biotechnology & applied microbiology,mathematical & computational biology
Benchmark and Parameter Sensitivity Analysis of Single-Cell RNA Sequencing Clustering Methods

Monika Krzak,Yordan Raykov,Alexis Boukouvalas,Luisa Cutillo,Claudia Angelini

DOI: https://doi.org/10.3389/fgene.2019.01253

IF: 3.7

2019-12-11

Frontiers in Genetics

Abstract:Single-cell RNA-seq (scRNAseq) is a powerful tool to study heterogeneity of cells. Recently, several clustering based methods have been proposed to identify distinct cell populations. These methods are based on different statistical models and usually require to perform several additional steps, such as preprocessing or dimension reduction, before applying the clustering algorithm. Individual steps are often controlled by method-specific parameters, permitting the method to be used in different modes on the same datasets, depending on the user choices. The large number of possibilities that these methods provide can intimidate non-expert users, since the available choices are not always clearly documented. In addition, to date, no large studies have invistigated the role and the impact that these choices can have in different experimental contexts. This work aims to provide new insights into the advantages and drawbacks of scRNAseq clustering methods and describe the ranges of possibilities that are offered to users. In particular, we provide an extensive evaluation of several methods with respect to different modes of usage and parameter settings by applying them to real and simulated datasets that vary in terms of dimensionality, number of cell populations or levels of noise. Remarkably, the results presented here show that great variability in the performance of the models is strongly attributed to the choice of the user-specific parameter settings. We describe several tendencies in the performance attributed to their modes of usage and different types of datasets, and identify which methods are strongly affected by data dimensionality in terms of computational time. Finally, we highlight some open challenges in scRNAseq data clustering, such as those related to the identification of the number of clusters.

genetics & heredity
Performance evaluation results of evolutionary clustering algorithm star for clustering heterogeneous datasets

Bryar A. Hassan,TarikA. Rashid,Seyedali Mirjalili

DOI: https://doi.org/10.1016/j.dib.2021.107044

2021-04-30

Abstract:This article presents the data used to evaluate the performance of evolutionary clustering algorithm star (ECA*) compared to five traditional and modern clustering algorithms. Two experimental methods are employed to examine the performance of ECA* against genetic algorithm for clustering++ (GENCLUST++), learning vector quantisation (LVQ) , expectation maximisation (EM) , K-means++ (KM++) and K-means (KM). These algorithms are applied to 32 heterogenous and multi-featured datasets to determine which one performs well on the three tests. For one, ther paper examines the efficiency of ECA* in contradiction of its corresponding algorithms using clustering evaluation measures. These validation criteria are objective function and cluster quality measures. For another, it suggests a performance rating framework to measurethe the performance sensitivity of these algorithms on varos dataset features (cluster dimensionality, number of clusters, cluster overlap, cluster shape and cluster structure). The contributions of these experiments are two-folds: (i) ECA* exceeds its counterpart aloriths in ability to find out the right cluster number; (ii) ECA* is less sensitive towards dataset features compared to its competitive techniques. Nonetheless, the results of the experiments performed demonstrate some limitations in the ECA*: (i) ECA* is not fully applied based on the premise that no prior knowledge exists; (ii) Adapting and utilising ECA* on several real applications has not been achieved yet.

Neural and Evolutionary Computing,Artificial Intelligence,Machine Learning
Extended multivariate comparison of 68 cluster validity indices. A review

Roberto Todeschini,Davide Ballabio,Veronica Termopoli,Viviana Consonni

DOI: https://doi.org/10.1016/j.chemolab.2024.105117

IF: 4.175

2024-03-27

Chemometrics and Intelligent Laboratory Systems

Abstract:Clustering is an unsupervised machine learning methodology widely used in several sciences to find groups of similar patterns in complex data. The results generated by clustering algorithms generally depend on user-defined input parameters such as the number of expected clusters, which can have a great impact on the homogeneity of the identified clusters. Clustering validity indices (CVIs) are an effective method for determining the optimal number of clusters that best fit the natural partition of a dataset. They do not require any underlying assumption nor a priori knowledge about the true dataset structure. Since 1965, many cluster validity indices have been proposed in the literature and used in several different applications. In this paper, the performance of 68 cluster validity indices was evaluated on 21 real-life research and simulated datasets. CVIs were compared on the same partition for each dataset, which was searched for by the k -means clustering algorithm. Multivariate chemometric methods were applied to disclose mutual relationships among the indices and to select those that are more effective in terms of accuracy and reliability.

automation & control systems,computer science, artificial intelligence,instruments & instrumentation,statistics & probability,mathematics, interdisciplinary applications,chemistry, analytical
Simulation-derived best practices for clustering clinical data

Caitlin E Coombes,Xin Liu,Zachary B Abrams,Kevin R Coombes,Guy Brock,Caitlin E. Coombes,Zachary B. Abrams,Kevin R. Coombes

DOI: https://doi.org/10.1016/j.jbi.2021.103788

IF: 8

2021-06-01

Journal of Biomedical Informatics

Abstract:<h3 class="u-h4 u-margin-m-top u-margin-xs-bottom">Introduction</h3><p>Clustering analyses in clinical contexts hold promise to improve the understanding of patient phenotype and disease course in chronic and acute clinical medicine. However, work remains to ensure that solutions are rigorous, valid, and reproducible. In this paper, we evaluate best practices for dissimilarity matrix calculation and clustering on mixed-type, clinical data.</p><h3 class="u-h4 u-margin-m-top u-margin-xs-bottom">Methods</h3><p>We simulate clinical data to represent problems in clinical trials, cohort studies, and EHR data, including single-type datasets (binary, continuous, categorical) and 4 data mixtures. We test 5 single distance metrics (Jaccard, Hamming, Gower, Manhattan, Euclidean) and 3 mixed distance metrics (DAISY, Supersom, and Mercator) with 3 clustering algorithms (hierarchical (HC), <em>k-</em>medoids, self-organizing maps (SOM)). We quantitatively and visually validate by Adjusted Rand Index (ARI) and silhouette width (SW).</p><h3 class="u-h4 u-margin-m-top u-margin-xs-bottom">Results</h3><p>HC outperformed <em>k-</em>medoids and SOM by ARI across data types. DAISY produced the highest mean ARI for mixed data types for all mixtures except unbalanced mixtures dominated by continuous data. Compared to the Hamming distance, a real data application of DAISY with HC uncovered superior, separable clusters.</p><h3 class="u-h4 u-margin-m-top u-margin-xs-bottom">Discussion</h3><p>Selecting an appropriate mixed-type metric allows the investigator to obtain optimal separation of patient clusters and get maximum use of their data. Superior metrics for mixed-type data handle multiple data types using multiple, type-focused distances. Better subclassification of disease opens avenues for targeted treatments, precision medicine, clinical decision support, and improved patient outcomes.</p>

medical informatics,computer science, interdisciplinary applications
A framework for benchmarking clustering algorithms

Marek Gagolewski

DOI: https://doi.org/10.1016/j.softx.2022.101270

2023-10-26

Abstract:The evaluation of clustering algorithms can involve running them on a variety of benchmark problems, and comparing their outputs to the reference, ground-truth groupings provided by experts. Unfortunately, many research papers and graduate theses consider only a small number of datasets. Also, the fact that there can be many equally valid ways to cluster a given problem set is rarely taken into account. In order to overcome these limitations, we have developed a framework whose aim is to introduce a consistent methodology for testing clustering algorithms. Furthermore, we have aggregated, polished, and standardised many clustering benchmark dataset collections referred to across the machine learning and data mining literature, and included new datasets of different dimensionalities, sizes, and cluster types. An interactive datasets explorer, the documentation of the Python API, a description of the ways to interact with the framework from other programming languages such as R or MATLAB, and other details are all provided at <<a class="link-external link-https" href="https://clustering-benchmarks.gagolewski.com" rel="external noopener nofollow">this https URL</a>>.

Machine Learning
Comparing clusterings and numbers of clusters by aggregation of calibrated clustering validity indexes

Serhat Emre Akhanli,Christian Hennig

DOI: https://doi.org/10.48550/arXiv.2002.01822

2020-06-24

Abstract:A key issue in cluster analysis is the choice of an appropriate clustering method and the determination of the best number of clusters. Different clusterings are optimal on the same data set according to different criteria, and the choice of such criteria depends on the context and aim of clustering. Therefore, researchers need to consider what data analytic characteristics the clusters they are aiming at are supposed to have, among others within-cluster homogeneity, between-clusters separation, and stability. Here, a set of internal clustering validity indexes measuring different aspects of clustering quality is proposed, including some indexes from the literature. Users can choose the indexes that are relevant in the application at hand. In order to measure the overall quality of a clustering (for comparing clusterings from different methods and/or different numbers of clusters), the index values are calibrated for aggregation. Calibration is relative to a set of random clusterings on the same data. Two specific aggregated indexes are proposed and compared with existing indexes on simulated and real data.

Methodology
Cleaning by clustering: methodology for addressing data quality issues in biomedical metadata

Wei Hu,Amrapali Zaveri,Honglei Qiu,Michel Dumontier

DOI: https://doi.org/10.1186/s12859-017-1832-4

IF: 3.307

2017-01-01

BMC Bioinformatics

Abstract:Background The ability to efficiently search and filter datasets depends on access to high quality metadata. While most biomedical repositories require data submitters to provide a minimal set of metadata, some such as the Gene Expression Omnibus (GEO) allows users to specify additional metadata in the form of textual key-value pairs (e.g. sex: female). However, since there is no structured vocabulary to guide the submitter regarding the metadata terms to use, consequently, the 44,000,000+ key-value pairs in GEO suffer from numerous quality issues including redundancy, heterogeneity, inconsistency, and incompleteness. Such issues hinder the ability of scientists to hone in on datasets that meet their requirements and point to a need for accurate, structured and complete description of the data. Methods In this study, we propose a clustering-based approach to address data quality issues in biomedical, specifically gene expression, metadata. First, we present three different kinds of similarity measures to compare metadata keys. Second, we design a scalable agglomerative clustering algorithm to cluster similar keys together. Results Our agglomerative cluster algorithm identified metadata keys that were similar, based on (i) name, (ii) core concept and (iii) value similarities, to each other and grouped them together. We evaluated our method using a manually created gold standard in which 359 keys were grouped into 27 clusters based on six types of characteristics: (i) age, (ii) cell line, (iii) disease, (iv) strain, (v) tissue and (vi) treatment. As a result, the algorithm generated 18 clusters containing 355 keys (four clusters with only one key were excluded). In the 18 clusters, there were keys that were identified correctly to be related to that cluster, but there were 13 keys which were not related to that cluster. We compared our approach with four other published methods. Our approach significantly outperformed them for most metadata keys and achieved the best average F-Score (0.63). Conclusion Our algorithm identified keys that were similar to each other and grouped them together. Our intuition that underpins cleaning by clustering is that, dividing keys into different clusters resolves the scalability issues for data observation and cleaning, and keys in the same cluster with duplicates and errors can easily be found. Our algorithm can also be applied to other biomedical data types.
Benchmarking digital PCR partition classification methods with empirical and simulated duplex data

Yao Chen,Ward De Spiegelaere,Wim Trypsteen,David Gleerup,Jo Vandesompele,Antoon Lievens,Matthijs Vynck,Olivier Thas

DOI: https://doi.org/10.1093/bib/bbae120

IF: 9.5

2024-03-27

Briefings in Bioinformatics

Abstract:Abstract Digital PCR (dPCR) is a highly accurate technique for the quantification of target nucleic acid(s). It has shown great potential in clinical applications, like tumor liquid biopsy and validation of biomarkers. Accurate classification of partitions based on end-point fluorescence intensities is crucial to avoid biased estimators of the concentration of the target molecules. We have evaluated many clustering methods, from general-purpose methods to specific methods for dPCR and flowcytometry, on both simulated and real-life data. Clustering method performance was evaluated by simulating various scenarios. Based on our extensive comparison of clustering methods, we describe the limits of these methods, and formulate guidelines for choosing an appropriate method. In addition, we have developed a novel method for simulating realistic dPCR data. The method is based on a mixture distribution of a Poisson point process and a skew-$t$ distribution, which enables the generation of irregularities of cluster shapes and randomness of partitions between clusters (‘rain’) as commonly observed in dPCR data. Users can fine-tune the model parameters and generate labeled datasets, using their own data as a template. Besides, the database of experimental dPCR data augmented with the labeled simulated data can serve as training and testing data for new clustering methods. The simulation method is available as an R Shiny app.

biochemical research methods,mathematical & computational biology
Comparison and Evaluation of Network Clustering Algorithms Applied to Genetic Interaction Networks

Lin Hou,Lin Wang,Arthur Berg,Minping Qian,Yunping Zhu,Fangting Li,Minghua Deng

DOI: https://doi.org/10.2741/532

2012-01-01

Frontiers in Bioscience

Abstract:The goal of network clustering algorithms detect dense clusters in a network, and provide a first step towards the understanding of large scale biological networks. With numerous recent advances in biotechnologies, large-scale genetic interactions are widely available, but there is a limited understanding of which clustering algorithms may be most effective. In order to address this problem, we conducted a systematic study to compare and evaluate six clustering algorithms in analyzing genetic interaction networks, and investigated influencing factors in choosing algorithms. The algorithms considered in this comparison include hierarchical clustering, topological overlap matrix, bi-clustering, Markov clustering, Bayesian discriminant analysis based community detection, and variational Bayes approach to modularity. Both experimentally identified and synthetically constructed networks were used in this comparison. The accuracy of the algorithms is measured by the Jaccard index in comparing predicted gene modules with benchmark gene sets. The results suggest that the choice differs according to the network topology and evaluation criteria. Hierarchical clustering showed to be best at predicting protein complexes; Bayesian discriminant analysis based community detection proved best under epistatic miniarray profile (EMAP) datasets; the variational Bayes approach to modularity was noticeably better than the other algorithms in the genome-scale networks.
Clustering in life sciences.

Ying Zhao,George Karypis

DOI: https://doi.org/10.1385/1-59259-364-X:183

2003-01-01

Abstract:Clustering is the task of organizing a set of objects into meaningful groups. These groups can be disjoint, overlapping, or organized in some hierarchical fashion. The key element of clustering is the notion that the discovered groups are meaningful. This definition is intentionally vague, as what constitutes meaningful is to a large extent, application dependent. In some applications this may translate to groups in which the pairwise similarity between their objects is maximized, and the pairwise similarity between objects of different groups is minimized. In some other applications this may translate to groups that contain objects that share some key characteristics, even though their overall similarity is not the highest. Clustering is an exploratory tool for analyzing large datasets, and has been used extensively in numerous application areas. Clustering has a wide range of applications in life sciences and over the years has been used in many areas ranging from the analysis of clinical information, phylogeny, genomics, and proteomics. For example, clustering algorithms applied to gene expression data can be used to identify co-regulated genes and provide a genetic fingerprint for various diseases. Clustering algorithms applied on the entire database of known proteins can be used to automatically organize the different proteins into close- and distant-related families, and identify subsequences that are mostly preserved across proteins (52, 22, 55, 68, 49). Similarly, clustering algorithms applied to the tertiary structural datasets can be used to perform a similar organization and provide insights in the rate of change between sequence and structure (20, 65). The primary goal of this chapter is to provide an overview of the various issues involved in clustering large datasets, describe the merits and underlying assumptions of some of the commonly used clustering approaches, and provide insights on how to cluster datasets arising in various areas within life-sciences. Toward this end, the chapter is organized in broadly three parts. The first part (Sections 2- 4) describes the various types of clustering algorithms developed over the years, the various methods for computing the similarity between objects arising in life sciences, and methods for assessing the quality of the clusters. The second part (Section 5) focuses on the problem of clustering data arising from microarray experiments and describes some of the commonly used approaches. Finally, the third part (Section 6) provides a brief introduction to CLUTO, a general purpose toolkit for clustering various datasets, with an emphasis on its applications to problems and analysis requirements within life sciences.

Comparing the performance of biomedical clustering methods

Comparison of Methods for Biological Sequence Clustering.

Clustering cancer gene expression data: a comparative study

A comparison framework and guideline of clustering methods for mass cytometry data

Selection of single cell clustering methodologies through rank aggregation of multiple performance measures

A systematic performance evaluation of clustering methods for single-cell RNA-seq data

An empirical comparison and characterisation of nine popular clustering methods

Benchmarking Spatial Clustering Methods with Spatially Resolved Transcriptomics Data

Comparison of clustering tools in R for medium-sized 10x Genomics single-cell RNA-sequencing data

Single-cell RNA-seq Data Clustering: A Survey with Performance Comparison Study

Statistical power for cluster analysis

Benchmark and Parameter Sensitivity Analysis of Single-Cell RNA Sequencing Clustering Methods

Performance evaluation results of evolutionary clustering algorithm star for clustering heterogeneous datasets

Extended multivariate comparison of 68 cluster validity indices. A review

Simulation-derived best practices for clustering clinical data

A framework for benchmarking clustering algorithms

Comparing clusterings and numbers of clusters by aggregation of calibrated clustering validity indexes

Cleaning by clustering: methodology for addressing data quality issues in biomedical metadata

Benchmarking digital PCR partition classification methods with empirical and simulated duplex data

Comparison and Evaluation of Network Clustering Algorithms Applied to Genetic Interaction Networks

Clustering in life sciences.