Abstract:Cluster analysis is a crucial data mining technology widely used in image segmentation, language processing, and pattern recognition. Most existing clustering algorithms cannot identify complex shapes in manifold data sets and data sets with varying-density distribution, especially when clusters with significant differences in density are close to each other. Hierarchical clustering algorithms can identify data sets of arbitrary shapes. However, hierarchical clustering algorithms not only cannot cluster datasets with significant density variations but also have a high time cost. So in this paper, we propose a novel hierarchical clustering algorithm based on density-distance cores, called HCDC. It first selects the density-distance representative points for each point from the set of candidate representative points. Then it selects density-distance cores from all density-distance representatives. And it replaces the whole data set with density-distance cores and uses a new distance between them to apply hierarchical clustering. To avoid the influence of noise points in the dataset when finding density-distance cores, we also propose the noise point detection method and verify the feasibility of this method. In this paper, we compare our proposed algorithm with existing classical and novel algorithms on synthetic and real datasets. Experiments show that our algorithm clusters better than existing algorithms on complex-shaped datasets and datasets with different densities. On datasets with sparse and dense clusters close to each other, the ARI score of HCDC is more than 0.1 higher than that of LDP-MST. In particular, on the grid dataset, HCDC's ARI score is 0.997 higher than LDP-MST. On DS3 and DS8, HCDC's ARI score is more than 0.14 higher than the second-best algorithm, RNN-DBSCAN. Moreover, on the zoo dataset, HCDC's ARI score is 0.15 and 0.6 higher than RNN-DBSCAN and LDP-MST, respectively. On the olivetti face dataset, HCDC is the only algorithm with an NMI score above 0.9 on photo1 and photo2 datasets.

DCSI -- An improved measure of cluster separability based on separation and connectedness

A Distance-based Separability Measure for Internal Cluster Validation

An Internal Cluster Validity Index Using a Distance-based Separability Measure

Density Peak Clustering with connectivity estimation

A Novel Intrinsic Measure of Data Separability

A New Separation Measure for Improving the Effectiveness of Validity Indices

Data Separability for Neural Network Classifiers and the Development of a Separability Index

A Statistical Information-Based Clustering Approach in Distance Space

A note on the separability index

HCDC: A novel hierarchical clustering algorithm based on density-distance cores for data sets with varying density

An improved density-sensitive semi-supervised clustering algorithm

Constraint Based Dimension Correlation and Distance Divergence for Clustering High-Dimensional Data

Interpretable Clustering with the Distinguishability Criterion

A Dual Distance Based Spatial Clustering Method

Clustering by Defining and Merging Candidates of Cluster Centers Via Independence and Affinity.

Density-based statistical merging clustering algorithm

Clustering by measuring local direction centrality for data with heterogeneous density and weak connectivity

Clustering Deviation Index (CDI): a robust and accurate internal measure for evaluating scRNA-seq data clustering

Density-Sensitive Evolutionary Clustering

A Distance Scaling Method to Improve Density-Based Clustering.

Density Peaks Clustering Algorithm Based on Density Stratification and Subcluster Fusion