A Kind of Similarity Degree for Non-Precise Data with Application to Clustering Analysis

Chang-Lin MEI,Wen-xiu ZHANG

DOI: https://doi.org/10.3969/j.issn.1001-7402.2001.02.001

2001-01-01

Abstract:In this paper, a kind of similarity degree for non-precise data is proposed and based on this similarity degree, some clustering methods for non-precise data are investigated.

What problem does this paper attempt to address?

A kind of practical fuzzy clustering

Min Yao,Weihong Wang

DOI: https://doi.org/10.1109/WCICA.2002.1021464

2002-01-01

Abstract:Fuzzy clustering is one of the most important fuzzy techniques. This paper adopts a relational degree as similarity measurement between objects, and thus forms relational clustering which may find wide applications in social systems and economic systems. At the same time, this paper presents a kind of recomposed fuzzy clustering which can be used to execute the task of partitioning and evaluation for a lot of objects in many systems effectively.
A New Measurement of Similarity and Related Clustering Algorithm

HUANG Jian-peng,LU Li-qiang

DOI: https://doi.org/10.15943/j.cnki.fdxb-jns.2006.02.007

2006-01-01

Abstract:Distance and neighbor are of common measurements for the similarity of clusters.After the experimental analysis for the performance of the algorithms based on Euclidean distance and KNN,the definition of SNN(shared k-nearest neighbor) is presented.A new measurement of similarity based on SNN and KNN is introduced and its implementation algorithm is also included.With the clustering results of the simple data,the complex one as well as the network intrusion dataset of KDD Cup'99,the algorithm based on SNN is proven to produce more accurate and more natural clusters for the datasets of different density,size,shape than the k-means.Moreover the algorithm is not sensitive to the choice of parameters.
Similarity Calculation Based on Feature Weight Evaluation

Ming LIU,Chong WU,Yuan-Chao LIU,Cheng-Jie SUN

DOI: https://doi.org/10.11897/SP.J.1016.2015.01420

2015-01-01

Abstract:Along with high-speed advance of information technology,the unsupervised character-istic of clustering makes itself an effective implement for data analysis.To acquire high clustering performance,the effective and precise similarity calculation plays a prime and necessary role for clustering algorithms.Owing to the fact that different features have diverse contributions to describe similarity among data,it is necessary to assess feature’s contribution by means of some transcendental knowledge (e.g.constrained data provided by users),and import it in similarity measurement to acquire more precise calculating results.Unfortunately,conventional weight evaluating methods all fail to consider two challenges:(1)high possibility of asymmetrical distri-bution of constrained data in feature space;(2 )high possibility of inconsistency contained by constrained data.Previous two issues disable conventional weight evaluating methods to acquire high precision,and even make them unable to work.Hence,this paper proposes a novel constraint based weight evaluating method to deal with them.For the former one,constrained data are partitioned into several equivalent classes,and distributing parameters are assigned to them to balance their distributions.For the latter one,constrained data are connected to form an undirected graph,and belief values are thereby computed to measure and reduce their possibilities to be inconsistent. Finally,these two parameters are integrated in weight evaluating function to form an accurate similarity measurement.Experimental results demonstrate that,this weight evaluating method can combine constrained data to obtain diverse contributions of different features to similarity calculation,and can be applied in any clustering algorithm to improve its precision.
A Statistical Information-Based Clustering Approach in Distance Space

Yue Shi-hong,Li Ping,Guo Ji-dong,Zhou Shui-geng

DOI: https://doi.org/10.1007/bf02842480

2005-01-01

Journal of Zhejiang University SCIENCE A

Abstract:Clustering, as a powerful data mining technique for discovering interesting data distributions and patterns in the underlying database, is used in many fields, such as statistical data analysis, pattern recognition, image processing, and other business applications. Density-based Spatial Clustering of Applications with Noise (DBSCAN) (Esteret al., 1996) is a good performance clustering method for dealing with spatial data although it leaves many problems to be solved. For example, DBSCAN requires a necessary user-specified threshold while its computation is extremely time-consuming by current method such as OPTICS, etc. (Ankerstet al., 1999), and the performance of DBSCAN under different norms has yet to be examined. In this paper, we first developed a method based on statistical information of distance space in database to determine the necessary threshold. Then our examination of the DBSCAN performance under different norms showed that there was determinable relation between them. Finally, we used two artificial databases to verify the effectiveness and efficiency of the proposed methods.
Discriminative Similarity for Data Clustering

Yingzhen Yang,Ping Li

DOI: https://doi.org/10.48550/arXiv.2109.08675

IF: 5.414

2021-09-17

Machine Learning

Abstract:Similarity-based clustering methods separate data into clusters according to the pairwise similarity between the data, and the pairwise similarity is crucial for their performance. In this paper, we propose {\em Clustering by Discriminative Similarity (CDS)}, a novel method which learns discriminative similarity for data clustering. CDS learns an unsupervised similarity-based classifier from each data partition, and searches for the optimal partition of the data by minimizing the generalization error of the learnt classifiers associated with the data partitions. By generalization analysis via Rademacher complexity, the generalization error bound for the unsupervised similarity-based classifier is expressed as the sum of discriminative similarity between the data from different classes. It is proved that the derived discriminative similarity can also be induced by the integrated squared error bound for kernel density classification. In order to evaluate the performance of the proposed discriminative similarity, we propose a new clustering method using a kernel as the similarity function, CDS via unsupervised kernel classification (CDSK), with its effectiveness demonstrated by experimental results.
A rough set based clustering algorithm and the information theoretical approach to refine clusters

Qingdong Wang,HuaPing Dai,Youxian Sun

DOI: https://doi.org/10.1109/WCICA.2004.1342320

2004-01-01

Abstract:In many clustering processes, the presence of more information does not usually generate a corresponding increase in the performance of clustering. The presence of irrelevant information decreases the effectiveness of the clustering algorithm. We propose a solution to improve the quality of clustering that is an attribute-weighted clustering algorithm based on rough set theory and the information theoretical refinement process. Firstly, we give every attribute the same weight value, and use rough set based clustering algorithm to get the initial classes. Then we weigh every attribute by Shannon's Entropy Theory, substitute mutual entropy values for the weight of every attribute, and compute with attribute-weighted rough set clustering algorithm again to refine and improve the clustering result. We have tested our algorithm on data sets from UCI repository. The experimental results show that our algorithm can obtain better results in classification rate and purity of classes than other traditional clustering methods.
Discriminative Similarity for Clustering and Semi-Supervised Learning

Yingzhen Yang,Feng Liang,Nebojsa Jojic,Shuicheng Yan,Jiashi Feng,Thomas S. Huang

DOI: https://doi.org/10.48550/arXiv.1709.01231

IF: 5.414

2017-09-05

Machine Learning

Abstract:Similarity-based clustering and semi-supervised learning methods separate the data into clusters or classes according to the pairwise similarity between the data, and the pairwise similarity is crucial for their performance. In this paper, we propose a novel discriminative similarity learning framework which learns discriminative similarity for either data clustering or semi-supervised learning. The proposed framework learns classifier from each hypothetical labeling, and searches for the optimal labeling by minimizing the generalization error of the learned classifiers associated with the hypothetical labeling. Kernel classifier is employed in our framework. By generalization analysis via Rademacher complexity, the generalization error bound for the kernel classifier learned from hypothetical labeling is expressed as the sum of pairwise similarity between the data from different classes, parameterized by the weights of the kernel classifier. Such pairwise similarity serves as the discriminative similarity for the purpose of clustering and semi-supervised learning, and discriminative similarity with similar form can also be induced by the integrated squared error bound for kernel density classification. Based on the discriminative similarity induced by the kernel classifier, we propose new clustering and semi-supervised learning methods.
Clustering Uncertain Data Based on Probability Distribution Similarity

Bin Jiang,Jian Pei,Yufei Tao,Xuemin Lin

DOI: https://doi.org/10.1109/TKDE.2011.221

IF: 9.235

2013-01-01

IEEE Transactions on Knowledge and Data Engineering

Abstract:Clustering on uncertain data, one of the essential tasks in mining uncertain data, posts significant challenges on both modeling similarity between uncertain objects and developing efficient computational methods. The previous methods extend traditional partitioning clustering methods like $(k)$-means and density-based clustering methods like DBSCAN to uncertain data, thus rely on geometric distances between objects. Such methods cannot handle uncertain objects that are geometrically indistinguishable, such as products with the same mean but very different variances in customer ratings. Surprisingly, probability distributions, which are essential characteristics of uncertain objects, have not been considered in measuring similarity between uncertain objects. In this paper, we systematically model uncertain objects in both continuous and discrete domains, where an uncertain object is modeled as a continuous and discrete random variable, respectively. We use the well-known Kullback-Leibler divergence to measure similarity between uncertain objects in both the continuous and discrete cases, and integrate it into partitioning and density-based clustering methods to cluster uncertain objects. Nevertheless, a naïve implementation is very costly. Particularly, computing exact KL divergence in the continuous case is very costly or even infeasible. To tackle the problem, we estimate KL divergence in the continuous case by kernel density estimation and employ the fast Gauss transform technique to further speed up the computation. Our extensive experiment results verify the effectiveness, efficiency, and scalability of our approaches.
Cluster and Association Analysis of Natural Languages Based on Inclined Similarity Measures

Zhenning CHEN,Zhenyu CHEN

2017-01-01

Abstract:Cluster analysis is the task of grouping a set of objects by associations of these objects.The diameters of cluster and association analysis are similarity measures,which often involves the absolute similarity of the symmetry property.But most rules found in natural languages are inclined and have asymmetrical forms.We describes the asymmetrical associationby a parameter of Probability Entailment,i.e.the conditional probability,to represent the asymmetrical associations among features.And then we define the Domination Relation,the Tight Relation,the Control Center,and the Midway island.A strategy for cluster based on inclined similarity measures is presented to deal with issues likethe false isolated points,data sparsity and family iconicity.
A Similarity Measurement with Entropy-Based Weighting for Clustering Mixed Numerical and Categorical Datasets

Xia Que,Siyuan Jiang,Jiaoyun Yang,Ning An

DOI: https://doi.org/10.3390/a14060184

2021-01-01

Algorithms

Abstract:Many mixed datasets with both numerical and categorical attributes have been collected in various fields, including medicine, biology, etc. Designing appropriate similarity measurements plays an important role in clustering these datasets. Many traditional measurements treat various attributes equally when measuring the similarity. However, different attributes may contribute differently as the amount of information they contained could vary a lot. In this paper, we propose a similarity measurement with entropy-based weighting for clustering mixed datasets. The numerical data are first transformed into categorical data by an automatic categorization technique. Then, an entropy-based weighting strategy is applied to denote the different importances of various attributes. We incorporate the proposed measurement into an iterative clustering algorithm, and extensive experiments show that this algorithm outperforms OCIL and K-Prototype methods with 2.13% and 4.28% improvements, respectively, in terms of accuracy on six mixed datasets from UCI.
A New Similarity Combining Reconstruction Coefficient with Pairwise Distance for Agglomerative Clustering

Zhiling Cai,Xiaofei Yang,Tianyi Huang,William Zhu

DOI: https://doi.org/10.1016/j.ins.2019.08.048

IF: 8.1

2020-01-01

Information Sciences

Abstract:Agglomerative clustering is a mainstream clustering method that can produce an informative hierarchical structure of clusters. Existing similarities in agglomerative clustering are typically based on the pairwise distance. Although this type of similarity captures the local structure of data well, it is sensitive to noise and outliers because it considers only the distance between data points. In this paper, we propose a new similarity called RCPD by combining the reconstruction coefficient, which is robust to noise and outliers, with the pairwise distance for agglomerative clustering. Our new similarity takes advantage of both the distance between data points and the linear representation among data points. Thus, RCPD not only captures the local structure of data well but is also robust to noise and outliers. The experimental results on 11 real-world benchmark datasets show that our new clustering method consistently outperforms many state-of-the-art clustering approaches.
A New Similarity in Clustering Through Users' Interest and Social Relationship

Jianxiong Guo,Zhehao Zhu,Yucen Gao,Xiaofeng Gao

DOI: https://doi.org/10.1016/j.tcs.2024.114833

IF: 1.002

2024-01-01

Theoretical Computer Science

Abstract:Clustering is a basic technology in data mining, and similarity measurement plays a crucial role in it. The existing clustering algorithms, especially those for social networks, pay more attention to users' properties while ignoring the global measurement across social relationships. In this paper, a new clustering algorithm is proposed, which not only considers the distance of users' properties but also considers users' social influence. Social influence can be further divided into mutual influence and self influence. With mutual influence, we can deal with users' interests and measure their similarities by introducing areas and activities, thus better weighing the influence between them in an indirect way. Separately, we formulate a new propagation model, PR-Threshold++, by merging the PageRank algorithm and Linear Threshold model, to model the self influence. Based on that, we design a novel similarity by exploiting users' distance, mutual influence, and self influence. Finally, we adjust K-medoids according to our similarity and use real-world datasets to evaluate their performance in intensive simulations.
A Comparison Study on Similarity and Dissimilarity Measures in Clustering Continuous Data

Ali Seyed Shirkhorshidi,Saeed Aghabozorgi,Teh Ying Wah

DOI: https://doi.org/10.1371/journal.pone.0144059

IF: 3.7

2015-12-11

PLoS ONE

Abstract:Similarity or distance measures are core components used by distance-based clustering algorithms to cluster similar data points into the same clusters, while dissimilar or distant data points are placed into different clusters. The performance of similarity measures is mostly addressed in two or three-dimensional spaces, beyond which, to the best of our knowledge, there is no empirical study that has revealed the behavior of similarity measures when dealing with high-dimensional datasets. To fill this gap, a technical framework is proposed in this study to analyze, compare and benchmark the influence of different similarity measures on the results of distance-based clustering algorithms. For reproducibility purposes, fifteen publicly available datasets were used for this study, and consequently, future distance measures can be evaluated and compared with the results of the measures discussed in this work. These datasets were classified as low and high-dimensional categories to study the performance of each measure against each category. This research should help the research community to identify suitable distance measures for datasets and also to facilitate a comparison and evaluation of the newly proposed similarity or distance measures with traditional ones.

multidisciplinary sciences
New approaches for clustering high dimensional data

Wei Wang,Jinze Liu

2006-01-01

Abstract:Clustering is one of the most effective methods for analyzing datasets that contain a large number of objects with numerous attributes. Clustering seeks to identify groups, or clusters, of similar objects. In low dimensional space, the similarity between objects is often evaluated by summing the difference across all of their attributes. High dimensional data, however, may contain irrelevant attributes which mask the existence of clusters. The discovery of groups of objects that are highly similar within some subsets of relevant attributes becomes an important but challenging task. My thesis focuses on various models and algorithms for this task. We first present a flexible clustering model, namely OP-Cluster (Order Preserving Cluster). Under this model, two objects are similar on a subset of attributes if the values of these two objects induce the same relative ordering of these attributes. OP-Clustering algorithm has demonstrated to be useful to identify co-regulated genes in gene expression data. We also propose a semi-supervised approach to discover biologically meaningful OP-Clusters by incorporating existing gene function classifications into the clustering process. This semi-supervised algorithm yields only OP-clusters that are significantly enriched by genes from specific functional categories. Real datasets are often noisy. We propose a noise-tolerant clustering algorithm for mining frequently occurring itemsets. This algorithm is called approximate frequent itemsets (AFI). Both the theoretical and experimental results demonstrate that our AFI mining algorithm has higher recoverability of real clusters than any other existing itemset mining approaches. Pair-wise dissimilarities are often derived from original data to reduce the complexities of high dimensional data. Traditional clustering algorithms taking pair-wise dissimilarities as input often generate disjoint clusters from pair-wise dissimilarities. It is well known that the classification model represented by disjoint clusters is inconsistent with many real classifications, such gene function classifications. We develop a Poclustering algorithm, which generates overlapping clusters from pair-wise dissimilarities. We prove that by allowing overlapping clusters, Poclustering fully preserves the information of any dissimilarity matrices while traditional partitioning algorithms may cause significant information loss.
An Evaluation Method for Clustering Quality and Its Application

HAN Xi-wu,ZHAO Tie-jun

DOI: https://doi.org/10.3321/j.issn:0367-6234.2009.11.052

2009-01-01

Abstract:Based on intra-cluster similarity and inter-cluster similarity,this paper proposes a clustering quality evaluation method,which determines the similarity between the clustering result and the natural structure of the concerned population. A simple experiment and some analyses indicate that the evaluation method is basically in accordance with human’s intuition of classification,and can be used to evaluate clustering results,or as complementary techniques to other methods,or to modify manual standards for clustering test.
Clustering Algorithm Based on Dual-Index Nearest Neighbor Similarity Measure and Its Application in Gene Expression Data Analysis

zongjin li,ChangXin song,Jiyu Yang,Zeyu Jia,Chengying Yan,Liqin Tian,Xiaoming Wu

DOI: https://doi.org/10.21203/rs.3.rs-2641728/v1

2023-01-01

Abstract:Abstract Background The critical step in analyzing gene expression data is to divide genes into co-expression modules using module detection methods. Clustering algorithms are the most commonly employed technique for gene module detection. To obtain gene modules with great biological significance, the choice of an appropriate similarity measure methodology is vital. However, commonly used similarity measurement may not fully capture the complexities of biological systems. Hence, exploring more informative similarity measures before partitioning gene co-expression modules remains important. Results In this paper, we proposed a Dual-Index Nearest Neighbor Similarity Measure (DINNSM) algorithm to address the above issue. The algorithm first calculates the similarity matrix between genes using Pearson correlation or Spearman correlation. Then, nearest neighbor measurements are constructed based on the similarity matrix. Finally, the similarity matrix is reconstructed. We tested the six similarity measurement methods (Pearson correlation, Spearman correlation, Euclidean distance, maximum information coefficient, distance correlation, and DINNSM) by using four clustering algorithms: K-means, Hierarchical, FCM, and WGCNA on three independent gene expression datasets. The cluster evaluation was based on four indices: the Silhouette index, Calinski-Harabaz index, Adjust-Biological homogeneity index, and Davies-Bouldin index. The results showed that DINNSM is accurate and can get biologically meaningful gene co-expression modules. Conclusions DINNSM is better at revealing the complex biological relationships between genes and helps to obtain more accurate and biologically meaningful gene co-expression modules.
Dissimilarity based on direction information and its application

Chunzhong Li,Yubo Yuan,Zongben Xu

DOI: https://doi.org/10.1109/CISP.2011.6099997

2011-01-01

Abstract:Similarity (dissimilarity) is of critical importance for data analysis, especially for clustering problem. The classical dissimilarity is related to distance (norm of the difference between each pair of data points), but it ignores the direction information from one data point to another. In this paper, we proposed a new dissimilarity based on direction consistence, which considers not only the distance information but also the direction information. It has some advantages and can be used in clustering to give a good performance. © 2011 IEEE.
Similarity-Based Three-Way Clustering by Using Dimensionality Reduction

Anlong Li,Yiping Meng,Pingxin Wang

DOI: https://doi.org/10.3390/math12131951

IF: 2.4

2024-06-24

Mathematics

Abstract:Three-way clustering uses core region and fringe region to describe a cluster, which divide the dataset into three parts. The division helps identify the central core and outer sparse regions of a cluster. One of the main challenges in three-way clustering is the meaningful construction of the two sets. Aimed at handling high-dimensional data and improving the stability of clustering, this paper proposes a novel three-way clustering method. The proposed method uses dimensionality reduction techniques to reduce data dimensions and eliminate noise. Based on the reduced dataset, random sampling and feature extraction are performed multiple times to introduce randomness and diversity, enhancing the algorithm's robustness. Ensemble strategies are applied on these subsets, and the k-means algorithm is utilized to obtain multiple clustering results. Based on these results, we obtain co-association frequency between different samples and fused clustering result using the single-linkage method of hierarchical clustering. In order to describe the core region and fringe region of each cluster, the similar class of each sample is defined by co-association frequency. The lower and upper approximations of each cluster are obtained based on similar class. The samples in the lower approximation of each cluster belong to the core region of the cluster. The differences between lower and upper approximations of each cluster are defined as fringe region. Therefore, a three-way explanation of each cluster is naturally formed. By employing various UC Irvine Machine Learning Repository (UCI) datasets and comparing different clustering metrics such as Normalized Mutual Information (NMI), Adjusted Rand Index (ARI), and Accuracy (ACC), the experimental results show that the proposed strategy is effective in improving the structure of clustering results.

mathematics
Efficient Similarity Joins for Near-Duplicate Detection

Chuan Xiao,Wei Wang,Xuemin Lin,Jeffrey Xu Yu,Guoren Wang

DOI: https://doi.org/10.1145/2000824.2000825

IF: 1.6289

2011-01-01

ACM Transactions on Database Systems

Abstract:With the increasing amount of data and the need to integrate data from multiple data sources, one of the challenging issues is to identify near-duplicate records efficiently. In this article, we focus on efficient algorithms to find a pair of records such that their similarities are no less than a given threshold. Several existing algorithms rely on the prefix filtering principle to avoid computing similarity values for all possible pairs of records. We propose new filtering techniques by exploiting the token ordering information; they are integrated into the existing methods and drastically reduce the candidate sizes and hence improve the efficiency. We have also studied the implementation of our proposed algorithm in stand-alone and RDBMS-based settings. Experimental results show our proposed algorithms can outperform previous algorithms on several real datasets.
A novel intuitionistic fuzzy similarity measure with applications in decision-making, pattern recognition, and clustering problems

Rakesh Kumar,Satish Kumar

DOI: https://doi.org/10.1007/s41066-023-00366-1

2023-02-22

Granular Computing

Abstract:The distance and similarity measures are two interrelated information measures that can be effectively used to quantify the degree of deviation and degree of similarity between the pairs of objects. Among several interesting measures on intuitionistic fuzzy sets, the similarity measure is a fundamental and prominent tool for handling the imperfect and ambiguous information. The researchers have suggested several similarity measures, but some of them produce conflicting results in practical significance and violate the basic axioms of similarity. In this communication, we propose a novel similarity measure and visualize it, we also discuss the axioms of similarity and non-linearity property from graphical perspective. The characteristics of the suggested measure are demonstrated with the help of various numerical experiments and it is analyzed that the developed measure can overcome the unreasonable cases of the existing measures. We also discuss the effectiveness of the suggested measure over various existing measures in context of linguistic hedges. An algorithm for pattern recognition based on the proposed measure is developed and demonstrated with numerical experiments that stated measure suppresses the limitations of prevailing measures. Further, the efficiency of the suggested measure is examined in clustering analysis by matching the objects on the different confidence levels. Finally, a new decision-making algorithm based on the suggested measure is developed and a comparative study with existing approach is performed to establish its validity.

A Kind of Similarity Degree for Non-Precise Data with Application to Clustering Analysis

A kind of practical fuzzy clustering

A New Measurement of Similarity and Related Clustering Algorithm

Similarity Calculation Based on Feature Weight Evaluation

A Statistical Information-Based Clustering Approach in Distance Space

Discriminative Similarity for Data Clustering

A rough set based clustering algorithm and the information theoretical approach to refine clusters

Discriminative Similarity for Clustering and Semi-Supervised Learning

Clustering Uncertain Data Based on Probability Distribution Similarity

Cluster and Association Analysis of Natural Languages Based on Inclined Similarity Measures

A Similarity Measurement with Entropy-Based Weighting for Clustering Mixed Numerical and Categorical Datasets

A New Similarity Combining Reconstruction Coefficient with Pairwise Distance for Agglomerative Clustering

A New Similarity in Clustering Through Users' Interest and Social Relationship

A Comparison Study on Similarity and Dissimilarity Measures in Clustering Continuous Data

New approaches for clustering high dimensional data

An Evaluation Method for Clustering Quality and Its Application

Clustering Algorithm Based on Dual-Index Nearest Neighbor Similarity Measure and Its Application in Gene Expression Data Analysis

Dissimilarity based on direction information and its application

Similarity-Based Three-Way Clustering by Using Dimensionality Reduction

Efficient Similarity Joins for Near-Duplicate Detection

A novel intuitionistic fuzzy similarity measure with applications in decision-making, pattern recognition, and clustering problems