Abstract:Because of the large variety of cluster validation indices (CVIs), choosing the most suitable index is challenging. We assessed several CVIs using artificial binary data sets. Only a few CVI performed as expected with noisy data. Tau and silhouette widths proved to be the best geometric CVIs both for equal and unequal cluster sizes. Among non‐geometric indices, Crispness and OptimClass performed best. Aims Different clustering methods often classify the same data set differently. Selecting the "best" clustering solution from alternatives is possible with cluster validation indices. Because of the large variety of cluster validation indices (CVIs), choosing the most suitable index concerning the data set and clustering algorithms is challenging. We aim to assess different internal clustering validation indices. Methods Artificial binary data sets with equal‐ and unequal‐sized well‐separated a priori clusters were simulated and three levels of noise were then added. Twenty replications of each of the six types of data sets (two group sizes × three levels of noise) were created and analyzed by three clustering algorithms with Jaccard dissimilarity. Twenty‐seven clustering validation indices are evaluated including both geometric and non‐geometric indices. Results Although, in theory, all CVIs could differentiate between good and wrong classifications, only a few perform as expected with noisy data. Tau and silhouette widths proved to be the best geometric CVIs both for equal and unequal cluster sizes. Among non‐geometric indices, crispness and OptimClass performed best. Conclusion We recommend using these best‐performing CVIs. We suggest plotting the CVI value against the number of clusters because the lack of a sharp peak means that the position of the maximum is uncertain.

Choosing models in model-based clustering and discriminant analysis

Selecting the number of clusters, clustering models, and algorithms. A unifying approach based on the quadratic discriminant score

Comparing Model Selection and Regularization Approaches to Variable Selection in Model-Based Clustering

Investigation on Several Model Selection Criteria for Determining the Number of Cluster

Improving model choice in classification: an approach based on clustering of covariance matrices

Evaluating the performance of AIC and BIC for selecting spatial econometric models

A Comparative Study Of Several Cluster Number Selection Criteria

Model-Based Clustering, Discriminant Analysis, and Density Estimation

Variable selection in model-based clustering and discriminant analysis with a regularization approach

Model-based cluster and discriminant analysis with the MIXMOD software

Quantitative evaluation of internal cluster validation indices using binary data sets

On Bic'S Selection Consistency for Discriminant Analysis

Gaussian model-based partitioning using iterated local search

Choosing the Number of Topics in LDA Models -- A Monte Carlo Comparison of Selection Criteria

Information Geometric Model Selection Criterion and Its Application in Cognition

Theoretical Analysis and Comparison of Several Criteria on Linear Model Dimension Reduction

Simultaneous Clustering and Model Selection for Multinomial Distribution: A Comparative Study

Model Selection with Misspecified Spatial Covariance Structure

Models and Selection Criteria for Regression and Classification

Efficient Selection Between Hierarchical Cognitive Models: Cross-validation With Variational Bayes

Model-based Clustering with Sparse Covariance Matrices