Abstract:The trend of analyzing big data in artificial intelligence demands highly-scalable machine learning algorithms, among which clustering is a fundamental and arguably the most widely applied method. To extend the applications of regular vector-based clustering algorithms, the Discrete Distribution (D2) clustering algorithm has been developed, aiming at clustering data represented by bags of weighted vectors which are well adopted data signatures in many emerging information retrieval and multimedia learning applications. However, the high computational complexity of D2-clustering limits its impact in solving massive learning problems. Here we present the parallel D2-clustering (PD2-clustering) algorithm with substantially improved scalability. We developed a hierarchical multipass algorithm structure for parallel computing in order to achieve a balance between the individual-node computation and the integration process of the algorithm. Experiments and extensive comparisons between PD2-clustering and other clustering algorithms are conducted on synthetic datasets. The results show that the proposed parallel algorithm achieves significant speed-up with minor accuracy loss. We apply PD2-clustering to image concept learning. In addition, by extending D2-clustering to symbolic data, we apply PD2-clustering to protein sequence clustering. For both applications, we demonstrate the high competitiveness of our new algorithm in comparison with other state-of-the-art methods.

Parallel Hierarchical Clustering Algorithm Based on Preprocessed Data

Parallel spectral clustering algorithm

A Parallel Hierarchical Clustering Algorithm for PCs Cluster System.

Parallel Massive Clustering of Discrete Distributions

Efficient hierarchical clustering of large high dimensional datasets.

A Mechanism of Remote Sensing Image for Parallel Processing Base on Splitting Blocks

Paralinear Distance and Its Algorithm for Hierarchical Clustering of High-dimensional Discrete Variables

A Parallel Varied Density-Based Clustering Algorithm with Optimized Data Partition

CLUSTER-BASED OCEAN REMOTE SENSING IMAGE FUSION PARALLEL COMPUTING STRATEGY

Multi-core based Parallel N-path labeling HKM clustering algorithm.

Parallel Filtered Graphs for Hierarchical Clustering

Fast Parallel Algorithms for Euclidean Minimum Spanning Tree and Hierarchical Spatial Clustering

PARCLE: a Parallel Clustering Algorithm for Cluster System

A Coding Hierarchy Computing Based Clustering Algorithm

An efficient parallel method for canopy clusteing with multi-core platform

ParSymG: a Parallel Clustering Approach for Unsupervised Classification of Remotely Sensed Imagery

Design of PPU framework for processing ordered data blocks in the cluster environment

Study of Fast Parallel Clustering Partition Algorithm for Large Data Sets

PHC: a Fast Partition and Hierarchy-Based Clustering Algorithm

Scalable Hierarchical Agglomerative Clustering

Cluster-PC Based Parallel Distributed Data Clustering and Its Applications