A Parallel K-means Algorithm for High Dimensional Text Data

Xiaolei Shan,Yanming Shen,Yuxin Wang

DOI: https://doi.org/10.1109/icce-china.2018.8448719

2018-01-01

Abstract:This paper proposed a Parallel K-means Algorithm for High Dimensional sparse Text data (PKHT). By using GPU (Graphic Processing Unit) and MPI (Message-passing Interface), the proposed algorithm achieves a up to 11x lower running time

What problem does this paper attempt to address?

Parallelization of K-Means Clustering Algorithm for Data Mining

Hao Jiang,Liyan Yu

DOI: https://doi.org/10.1051/itmconf/20171203048

2017-01-01

ITM Web of Conferences

Abstract:In this paper, we studied the parallelization of K-Means clustering algorithm, proposed a parallel scheme, designed a corresponding algorithm, and implemented the algorithm in GPU environment. The experimental result shows that the GPU-based parallelization algorithm has a good acceleration effect compared with the CPU-based serialization algorithm.
Dvt-Pkm: An Improved Gpu Based Parallel K-Means Algorithm

Bo Yan,Ye Zhang,Zijiang Yang,Hongyi Su,Hong Zheng

DOI: https://doi.org/10.1007/978-3-319-09339-0_60

2014-01-01

Abstract:K-Means clustering algorithm is a typical partition-based clustering algorithm. Its two major disadvantages lie in the facts that the algorithm is sensitive to initial cluster centers and the outliers exert significant influence on the clustering results. In addition, K-Means algorithm traverses and computes all the data multiple times. Thus, the algorithm is not efficient when dealing with large data sets. In order to overcome the above limitations, this paper proposes to exclude the outliers using the minimum number of points in the d-dimensional hypersphere area. Then k cluster centers can be obtained by adjusting the threshold making use of density idea. Finally, K-Means algorithm will be integrated with Compute Unified Device Architecture (CUDA). The time efficiency is improved considerably through taking advantage of computing power of Graphic Processing Unit (GPU). We use the ratio of distance between classes to distance within classes and speedup as the evaluation criteria. The experiments indicate that the proposed algorithm significantly improves the stability and running efficiency of K-Means algorithm.
Parallel K-means Algorithm for Massive Texts on Spark

Peng LIU,Jiayu TENG,Enjie DING,Lei MENG

DOI: https://doi.org/10.3969/j.issn.1003-0077.2017.04.021

2017-01-01

Abstract:Due to sharp increase of internet texts ,the processing of k-means on such data is incredibly lengthened . Some classic parallel architectures ,such as Hadoop ,have not improved the execution efficiency of K-means ,because the frequent iteration in such algorithms is hard to be efficiently handled .This paper proposed a parallelization algo-rithm of k-means based on Spark .It makes full use of in-memory-computing RDD model of Spark so as to well meet the frequent iteration requirement of k-means .Experimental results show that k-means executes much more effi-ciently in Spark than in Hadoop on the same datasets and the same computing environments .
A Parallel K-Means Clustering Algorithm with MPI

Jing Zhang,Gongqing Wu,Xuegang Hu,Shiying Li,Shuilong Hao

DOI: https://doi.org/10.1109/paap.2011.17

2013-01-01

Journal of Computers

Abstract:Clustering is one of the most popular methods for data analysis, which is prevalent in many disciplines such as image segmentation, bioinformatics, pattern recognition and statistics etc. The most popular and simplest clustering algorithm is K-means because of its easy implementation, simplicity, efficiency and empirical success. However, the real-world applications produce huge volumes of data, thus, how to efficiently handle of these data in an important mining task has been a challenging and significant issue. In addition, MPI (Message Passing Interface) as a programming model of message passing presents high performances, scalability and portability. Motivated by this, a parallel K-means clustering algorithm with MPI, called MKmeans, is proposed in this paper. The algorithm enables applying the clustering algorithm effectively in the parallel environment. Experimental study demonstrates that MKmeans is relatively stable and portable, and it performs with low overhead of time on large volumes of data sets.
Subspace Clustering by Directly Solving Discriminative K-means

Chenhui Gao,Wenzhi Chen,Feiping Nie,Weizhong Yu,Feihu Yan

DOI: https://doi.org/10.1016/j.knosys.2022.109452

IF: 8.139

2022-01-01

Knowledge-Based Systems

Abstract:Applications in many domains such as text mining and natural language processing need to deal with high-dimensional data. High-dimensional data may present better clustering characteristics on a selected low-dimensional subspace. Subspace clustering is to project the data onto a low-dimensional subspace before clustering. Traditional subspace clustering methods employ eigenvalue decomposition to find the projection of the input data and perform K-means or kernel K-means to obtain the clustering matrix. This kind of methods is not only inefficient, but also adopts a two-step method to generate an approximate solution. Although Discriminative K-means (DisKmeans) integrates dimensionality reduction and clustering into a joint framework and solves the optimization problem by kernel K-means, such method needs to find the centroids in the kernel space and class labels iteratively and has a square time complexity. Accordingly, in this paper, we propose an algorithm, namely Fast DisKmeans (FDKM), to obtain the cluster indicator matrix in a direct way. Moreover, our proposed method has a linear time complexity, which is a significant reduction compared with the squared time complexity of DisKmeans. We also demonstrate that solving the object function of DisKmeans is equivalent to representing the cluster assignment matrix by a low-dimensional linear mapping of the data. Based on this observation, we propose the second algorithm, namely Iterative Fast DisKmeans (IFDKM), which also has a linear time complexity. A series of experiments were conducted on several datasets, and the experimental results showed the superior performance of FDKM and IFDKM.
Cluster Center Initialization Parallel Algorithm for K-Means Algorithm

Bo Yan,Ye Zhang,Hong Yi Su,Hong Zheng

DOI: https://doi.org/10.4028/www.scientific.net/amr.989-994.2169

2014-01-01

Advanced Materials Research

Abstract:K-Means algorithm is a one of the most famous unsupervised clustering algorithm. It has many disadvantages, such as sensitivity to the initial clustering centers and computes all the data points multiple times when facing the increasing data volume. In order to overcome the above limitations, this paper proposes to make use of density idea to find k cluster centers by adjusting the threshold. Finally, we design and implementation of the K-Means algorithm on the modern Graphic Processing Unit (GPU). The ratio of distance between classes to distance within classes and speedup are used as evaluation criteria. The experiments indicate that the proposed algorithm significantly improves the stability and efficiency of K-Means algorithm.
K-Means Parallel Acceleration for Sparse Data Dimensions on Flink

Zihao Zeng,Kenli Li,Mingxing Duan,Chubo Liu,Xiangke Liao

DOI: https://doi.org/10.1109/hpcc/smartcity/dss.2019.00284

2019-01-01

Abstract:The K-means algorithm is a clustering algorithm which widely used in various applications, and it's running time is dramatically increased as the data size expanded. When the volume of data exceeds the range that can be carried by a single machine, the parallel operation of the algorithm must be implemented by using a distributed computing framework. Generally, during the parallel operation of the task, there are differences among the running time of each task due to the data skew, and the running progress of the entire job is determined by the task with the longest running time. In this paper, we propose an optimal data partitioning method for the application of the k-means algorithm on the sparsely dimensioned dataset to eliminate the data skew problem and further accelerate the parallel execution of the algorithm. Experimental evaluation on large-scale text datasets demonstrate the effectiveness of our partitioning approach on Flink.
Study of Multilevel Parallel Algorithm of KPCA for Hyperspectral Images.

Rulin Xu,Chang Gao,Jingfei Jiang

DOI: https://doi.org/10.1007/978-981-13-2712-4_8

2018-01-01

Abstract:Hyperspectral remote sensing image data has been widely used in a variety of applications due to its continuous spectrum and high spectral resolution. However, reducing huge dimensions with high data relevance is time-consuming, and parallel processing is required to accelerate this process. In the previous work, the KPCA (Kernel Principal Component Analysis), a nonlinear dimensionality reduction method was studied, and a parallel KPCA algorithm was proposed based on heterogeneous system with a single GPU, and achieved the desired experimental results. However, as data scale grows, the proposed solution would consume all the available memory on a single node and encounter performance bottleneck. Therefore, to tackle the limitation of insufficient memory caused by the reduction of large-scale hyperspectral data dimension, in this paper the intra-node parallelization using multi-core CPUs and many-core GPUs are exploited to improve the parallel hierarchy of distributed-storage KPCA. Finally, we designed and implemented a multilevel hybrid parallel KPCA algorithm that achieves 2.75–9.27 times speedup compared to the traditional coarse-grained parallel KPCA method on MPI.
Research on Efficient K_Means Parallel Algorithm Based on Hadoop Distributed Architecture

Lin Qian,Lin Wang,Zhu Mei,Jun Yu,Guangxin Zhu,Debing Song,Mingjie Xu

DOI: https://doi.org/10.1088/1757-899x/452/4/042066

2018-01-01

IOP Conference Series Materials Science and Engineering

Abstract:Focusing on the problems of K-means algorithm that has high time complexity, slow convergence, lower clustering accuracy, slow operating speed, an efficient K-means parallel algorithm based on Hadoop system and MapReduce framework is proposed. Firstly, the algorithm uses K selective sorting algorithm to improve the sampling efficiency; Secondly, the iterative center is updated by using the weight replacement policy; finally, the initial center point is obtained based on the sample pretreatment strategy. Experimental results show that the proposed algorithm not only has good convergence, accuracy and speedup, but also can improve performance of the algorithm.
The Study Of Parallel K-Means Algorithm

Yufang Zhang,Zhongyang Xiong,Jiali Mao,Ling Ou

DOI: https://doi.org/10.1109/WCICA.2006.1714203

2006-01-01

Abstract:Clustering analysis plays an important role in scientific research and commercial application. K-means algorithm is a widely used partition method in clustering. As the dataset's scale increases rapidly, it is difficult to use K-means to deal with massive amount of data. A parallel strategy is incorporated into clustering method and a parallel K-means algorithm is proposed. For enhancing the efficiency of parallel K-means, dynamic load balance is introduced. Data parallel strategy and Master/Slave model are adopted. The experiments demonstrate that the parallel K-means has higher efficiency and universal use.
Research on Parallelization of GPU-based K-Nearest Neighbor Algorithm

Hao Jiang,Yulin Wu

DOI: https://doi.org/10.1088/1742-6596/910/1/012006

2017-01-01

Journal of Physics Conference Series

Abstract:Based on the analysis of the K-Nearest Neighbor Algorithm, the feasibility of parallelization is studied from the steps of the algorithm, the operation efficiency and the data structure of each step, and the part of parallel execution is determined. A K-Nearest Neighbor Algorithm parallelization scheme is designed and the parallel G-KNN algorithm is implemented in the CUDA environment. The experimental results show that the K-Nearest Neighbor Algorithm has a significant improvement in efficiency after parallelization, especially on large-scale data.
Efficient Parallel Implementation of a Density Peaks Clustering Algorithm on Graphics Processing Unit

Ke-shi Ge,Hua-you Su,Dong-sheng Li,Xi-cheng Lu

DOI: https://doi.org/10.1631/fitee.1601786

IF: 2.526

2017-01-01

Frontiers of Information Technology & Electronic Engineering

Abstract:The density peak (DP) algorithm has been widely used in scientific research due to its novel and effective peak density-based clustering approach. However, the DP algorithm uses each pair of data points several times when determining cluster centers, yielding high computational complexity. In this paper, we focus on accelerating the time-consuming density peaks algorithm with a graphics processing unit (GPU). We analyze the principle of the algorithm to locate its computational bottlenecks, and evaluate its potential for parallelism. In light of our analysis, we propose an efficient parallel DP algorithm targeting on a GPU architecture and implement this parallel method with compute unified device architecture (CUDA), called the ‘CUDA-DP platform’. Specifically, we use shared memory to improve data locality, which reduces the amount of global memory access. To exploit the coalescing accessing mechanism of GPU, we convert the data structure of the CUDA-DP program from array of structures to structure of arrays. In addition, we introduce a binary search-and-sampling method to avoid sorting a large array. The results of the experiment show that CUDA-DP can achieve a 45-fold acceleration when compared to the central processing unit based density peaks implementation.
A Novel Density Based Clustering Algorithm and Its Parallelization.

Xiaokang Li,Binbin Yu,Yinghua Zhou,Guangzhong Sun

DOI: https://doi.org/10.1109/pdcat.2014.9

2014-01-01

Abstract:K-Means, a simple but effective clustering algorithm, is widely used in data mining, machine learning and computer vision community. K-Means algorithm consists of initialization of cluster centers and iteration. The initial cluster centers have a great impact on cluster result and algorithm efficiency. More appropriate initial centers of k-Means can get closer to the optimum solution, and even much quicker convergence. In this paper, we propose a novel clustering algorithm, Kmms, which is the abbreviation of k-Means and Mean Shift. It is a density based algorithm. Experiments show our algorithm not only costs less initialization time compared with other density based algorithms, but also achieves better clustering quality and higher efficiency. And compared with the popular k-Means++ algorithm, our method gets comparable accuracy, mostly even better. Furthermore, we parallelize Kmms algorithm based on OPenMP from both initialization and iteration step and prove the convergence of the algorithm.
K-Means on Commodity GPUs with CUDA

Bai Hong-tao,He Li-li,Ouyang Dan-tong,Li Zhan-shan,Li He

DOI: https://doi.org/10.1109/CSIE.2009.491

2009-01-01

Abstract:K-means algorithm is one of the most famous unsupervised clustering algorithms. Many theoretical improvements for the performance of original algorithms have been put forward, while almost all of them are based on Single Instruction Single Data(SISD) architecture processors (CPUs), which partly ignored the inherent paralleled characteristic of the algorithms. In this paper, a novel Single Instruction Multiple Data (SIMD) architecture processors (GPUs)based k-means algorithm is proposed. In this algorithm, in order to accelerate compute-intensive portions of traditional k-means, both data objects assignment and k centroids recalculation are offloaded to the GPU in parallel. We have implemented this GPU-based k-means on the newest generation GPU with Compute Unified Device Architecture(CUDA). The numerical experiments demonstrated that the speed of GPU-based k-means could reach as high as 40 times of the CPU-based k-means.
A Parallel GPU-Based Approach to Clustering Very Fast Data Streams

Pengtao Huang,Xiu Li,Bo Yuan

DOI: https://doi.org/10.1145/2806416.2806545

2015-01-01

Abstract:Clustering data streams has become a hot topic in the era of big data. Driven by the ever increasing volume, velocity and variety of data, more efficient algorithms for clustering large-scale complex data streams are needed. In this paper, we present a parallel algorithm called PaStream, which is based on advanced Graphics Processing Unit (GPU) and follows the online-offline framework of CluStream. Our approach can achieve hundreds of times speedup on high-speed and high-dimensional data streams compared with CluStream. It can also discover clusters with arbitrary shapes and handle outliers properly. The efficiency and scalability of PaStream are demonstrated through comprehensive experiments on synthetic and standard benchmark datasets with various problem factors.
Design and Optimization of Nonlinear Dimensionality Reduction Algorithm for Hyperspectral Images on Heterogeneous System

Haifang Zhou,Chang Gao,Xunyun Liu

DOI: https://doi.org/10.1109/iThings-GreenCom-CPSCom-SmartData.2017.164

2017-01-01

Abstract:Hyperspectral remote sensing image data has been widely used in a variety of applications due to its continuous spectrum and high spectral resolution. However, reducing huge dimensions with high data relevance is time-consuming, and parallel processing is required to accelerate this process. In this paper, we studied the KPCA (Kernel Principal Component Analysis), a nonlinear dimensionality reduction method, and proposed a parallel KPCA algorithm (KPCA_G) based on the CPU/GPU heterogeneous system. The experimental results prove that our KPCA_G algorithm achieves up to 173x speedup over the original serial KPCA. Moreover, to tackle the limitation of insufficient memory caused by the reduction of large-scale hyperspectral data dimension, we exploit the intra-node parallelization using multi-core CPUs and many-core GPUs to improve the parallel hierarchy of distributed-storage KPCA. Finally, we designed and implemented a multilevel hybrid parallel KPCA algorithm that achieves 2.56 similar to 9.03 times speedup compared to the traditional coarse-grained parallel KPCA method on MPI.
Parallel WMD Algorithm Based on GPU Acceleration

李肯立,王昊天,阳王东,胡蓉,罗辉章

DOI: https://doi.org/10.11896/jsjkx.210600213

2021-01-01

Computer Science

Abstract:Word Mover's Distance (WMD) is a method of measuring text similarity.It defines the difference between two texts as the minimum distance between the word embedding vectors of the text.WMD uses the vocabulary to represent the text as a normalized bag-of-words vector.Since the words of the text occupies a small proportion in the corpus,the document vector gene-rated by the bag-of-words model is very sparse.Multiple documents can form a high-dimensional sparse matrix,such a sparse matrix will generate a lot of unnecessary operations.By calculating the WMD of a single source document for multiple target documents at once,the calculation process can be highly parallelized.Aiming at the sparsity of text vectors,this paper proposes a GPU-based parallel Sinkhorn-WMD algorithm,which uses compressed format to store target text to improve memory utilization,and reduces intermediate calculations based on the sparse structure.The pre-trained word embedding vector is used to calculate the word distance matrix,the WMD algorithm is improved,and the optimization algorithm is verified on two public news data sets.The experimental results show that the parallel algorithm on NVIDIA TITAN RTX can achieve a speedup of up to 67.43× compared with the CPU serial algorithm.
FT K-means: A High-Performance K-means on GPU with Fault Tolerance

Shixun Wu,Yitong Ding,Yujia Zhai,Jinyang Liu,Jiajun Huang,Zizhe Jian,Huangliang Dai,Sheng Di,Bryan M. Wong,Zizhong Chen,Franck Cappello

2024-08-08

Abstract:K-means is a widely used algorithm in clustering, however, its efficiency is primarily constrained by the computational cost of distance computing. Existing implementations suffer from suboptimal utilization of computational units and lack resilience against soft errors. To address these challenges, we introduce FT K-means, a high-performance GPU-accelerated implementation of K-means with online fault tolerance. We first present a stepwise optimization strategy that achieves competitive performance compared to NVIDIA's cuML library. We further improve FT K-means with a template-based code generation framework that supports different data types and adapts to different input shapes. A novel warp-level tensor-core error correction scheme is proposed to address the failure of existing fault tolerance methods due to memory asynchronization during copy operations. Our experimental evaluations on NVIDIA T4 GPU and A100 GPU demonstrate that FT K-means without fault tolerance outperforms cuML's K-means implementation, showing a performance increase of 10\%-300\% in scenarios involving irregular data shapes. Moreover, the fault tolerance feature of FT K-means introduces only an overhead of 11\%, maintaining robust performance even with tens of errors injected per second.

Distributed, Parallel, and Cluster Computing,Machine Learning
Fabric Image Layering Based on Parallel K-means

Lei Wang,Wei Zhao,Tong Sang,Zeng Zeng,Yangfan Li

DOI: https://doi.org/10.1109/HPCC-DSS-SmartCity-DependSys57074.2022.00193

2022-12-01

Abstract:K-means is a commonly used image layering algorithm. Since the K-means algorithm needs to calculate the Euclidean distance between data points, the computational complexity is high and the time consumption is large, which is difficult to meet the needs of industrial production. Therefore, it is very important to accelerate K-means. Many calculations in the K-means algorithm are independent, andCUDA can solve this problem better. This paper proposes a stream-based K-means parallel algorithm, which can not only realize the parallel calculation inside K-means, but also realize the search of multiple $K$ at the same time. Besides, according to the image data and the Euclidean distance calculation mechanism, a simple size penalty coefficient with extremely low computational complexity is proposed to select the $K$ value. Experiments show that the CUDA K-means method proposed in this paper is 27 times higher than that of CPU K-means, and the proposed penalty coefficient can find the inflection point in the loss of multiple $K$ values.

Computer Science
A KNN Optimization Based on GPU Parallel Computing Method

Bo Liu,Jianhou Gan,Bin Wen

DOI: https://doi.org/10.1109/csei47661.2019.8938843

2019-01-01

Abstract:KNN (K-nearest neighbor) is a simple and practical classification algorithm, but it is less efficient when dealing with massive high-dimensional data. Parallel computing is an effective way to accelerate big data calculations. In order to improve the efficiency of KNN, an optimization method based on GPU parallel computing is proposed in this paper. In the distance calculation stage, the parallelism is increased to the numerical value, and the independent components are calculated in parallel before the thread collaborative calculation. In the distance sorting stage, a method of judging order is proposed. This method based on shared memory gives Odd-even sorting the parallel ability to determine whether the sequence is ordered. The experimental results obtained in this research show that the proposed method can obviously improve the execution efficiency of KNN algorithm in high-dimensional samples and partial ordered distribution.

A Parallel K-means Algorithm for High Dimensional Text Data

Parallelization of K-Means Clustering Algorithm for Data Mining

Dvt-Pkm: An Improved Gpu Based Parallel K-Means Algorithm

Parallel K-means Algorithm for Massive Texts on Spark

A Parallel K-Means Clustering Algorithm with MPI

Subspace Clustering by Directly Solving Discriminative K-means

Cluster Center Initialization Parallel Algorithm for K-Means Algorithm

K-Means Parallel Acceleration for Sparse Data Dimensions on Flink

Study of Multilevel Parallel Algorithm of KPCA for Hyperspectral Images.

Research on Efficient K_Means Parallel Algorithm Based on Hadoop Distributed Architecture

The Study Of Parallel K-Means Algorithm

Research on Parallelization of GPU-based K-Nearest Neighbor Algorithm

Efficient Parallel Implementation of a Density Peaks Clustering Algorithm on Graphics Processing Unit

A Novel Density Based Clustering Algorithm and Its Parallelization.

K-Means on Commodity GPUs with CUDA

A Parallel GPU-Based Approach to Clustering Very Fast Data Streams

Design and Optimization of Nonlinear Dimensionality Reduction Algorithm for Hyperspectral Images on Heterogeneous System

Parallel WMD Algorithm Based on GPU Acceleration

FT K-means: A High-Performance K-means on GPU with Fault Tolerance

Fabric Image Layering Based on Parallel K-means

A KNN Optimization Based on GPU Parallel Computing Method