Abstract:Because of the large variety of cluster validation indices (CVIs), choosing the most suitable index is challenging. We assessed several CVIs using artificial binary data sets. Only a few CVI performed as expected with noisy data. Tau and silhouette widths proved to be the best geometric CVIs both for equal and unequal cluster sizes. Among non‐geometric indices, Crispness and OptimClass performed best. Aims Different clustering methods often classify the same data set differently. Selecting the "best" clustering solution from alternatives is possible with cluster validation indices. Because of the large variety of cluster validation indices (CVIs), choosing the most suitable index concerning the data set and clustering algorithms is challenging. We aim to assess different internal clustering validation indices. Methods Artificial binary data sets with equal‐ and unequal‐sized well‐separated a priori clusters were simulated and three levels of noise were then added. Twenty replications of each of the six types of data sets (two group sizes × three levels of noise) were created and analyzed by three clustering algorithms with Jaccard dissimilarity. Twenty‐seven clustering validation indices are evaluated including both geometric and non‐geometric indices. Results Although, in theory, all CVIs could differentiate between good and wrong classifications, only a few perform as expected with noisy data. Tau and silhouette widths proved to be the best geometric CVIs both for equal and unequal cluster sizes. Among non‐geometric indices, crispness and OptimClass performed best. Conclusion We recommend using these best‐performing CVIs. We suggest plotting the CVI value against the number of clusters because the lack of a sharp peak means that the position of the maximum is uncertain.

Comparison of internal evaluation criteria in hierarchical clustering of categorical data

Comparison of Similarity Measures for Categorical Data in Hierarchical Clustering

A Comparative Study Of Several Cluster Number Selection Criteria

An Evaluation Method for Clustering Quality and Its Application

Comparison of Spectral Clustering, K-clustering and Hierarchical Clustering on E-Nose Datasets: Application to the Recognition of Material Freshness, Adulteration Levels and Pretreatment Approaches for Tomato Juices

Evaluating and Validating Cluster Results

Efficient synthetical clustering validity indexes for hierarchical clustering

A new approach for evaluating internal cluster validation indices

Quantitative evaluation of internal cluster validation indices using binary data sets

Cluster Merging and Splitting in Hierarchical Clustering Algorithms

An Information-Theoretic External Cluster-Validity Measure

On the Use of Relative Validity Indices for Comparing Clustering Approaches

A review of systematic selection of clustering algorithms and their evaluation

Empirical and Theoretical Comparisons of Selected Criterion Functions for Document Clustering

Analysis and improvement of evaluation indexes for clustering results

Internal Purity: A Differential Entropy Based Internal Validation Index for Crisp and Fuzzy Clustering Validation

Comparing clusterings and numbers of clusters by aggregation of calibrated clustering validity indexes

Normalised clustering accuracy: An asymmetric external cluster validity measure

Extended multivariate comparison of 68 cluster validity indices. A review

Choosing models in model-based clustering and discriminant analysis

An experimental comparison of label selection methods for hierarchical document clusters