Abstract:The main target of this paper is to design a density-based clustering algorithm using the weighted grid and information entropy based on MapReduce, noted as DBWGIE-MR, to deal with the problems of unreasonable division of data gridding, low accuracy of clustering results and low efficiency of parallelization in big data clustering algorithm based on density. This algorithm is implemented in three stages: data partitioning, local clustering, and global clustering. For each stage, we propose several strategies to improve the algorithm. In the first stage, based on the spatial distribution of data points, we propose an adaptive division strategy (ADG) to divide the grid adaptively. In the second stage, we design a weighted grid construction strategy (NE) which can strengthen the relevance between grids to improve the accuracy of clustering. Meanwhile, based on the weighted grid and information entropy, we design a density calculation strategy (WGIE) to calculate the density of the grid. And last, to improve the parallel efficiency, core clusters computing algorithm based on MapReduce (COMCORE-MR) are proposed to parallel compute the core clusters of the clustering algorithm. In the third stage, based on disjoint-set, we propose a core cluster merging algorithm (MECORE) to speed-up ratio the convergence of merged local clusters. Furthermore, based on MapReduce, a core clusters parallel merging algorithm (MECORE-MR) is proposed to get the clustering algorithm results faster, which improves the core clusters merging efficiency of the density-based clustering algorithm. We conduct the experiments on four synthetic clusters. Compared with H-DBSCAN, DBSCAN-MR and MR-VDBSCAN, the experimental results show that the DBWGIE-MR algorithm has higher stability and accuracy, and it takes less time in parallel clustering.

Parallel Density-Based Clustering Algorithm by Using Weighted Grid and Information Entropy

DBWGIE-MR: A Density-Based Clustering Algorithm by Using the Weighted Grid and Information Entropy Based on MapReduce

Distributed Affinity Propagation Clustering Based on MapReduce

A Parallel Varied Density-Based Clustering Algorithm with Optimized Data Partition

A Grid-Based Density Peaks Clustering Algorithm

PGMCLU: A Novel Parallel Grid-Based Clustering Algorithm for Multi-Density Datasets

MRG-DBSCAN: an Improved DBSCAN Clustering Method Based on Map Reduce and Grid

Research on K-medoids clustering algorithm based on data density and its parallel processing based on MapReduce

5New density clustering algorithm based on MapReduce

Grid-Based Clustering Algorithm Based On Intersecting Partition And Density Estimation

Grid-based Clustering Algorithm for Multi-Density

Clustering Algorithm Based on Grid Density and Distance Information Characteristics

DPCG: an Efficient Density Peaks Clustering Algorithm Based on Grid

An Improved K-means Algorithm Based on Mapreduce and Grid

An Efficient Density-Based Clustering for Multi-Dimensional Database

An Efficient Clustering Algorithm Based on Grid Density and its Application in Human Mobility Analysis.

CABGD: an Improved Clustering Algorithm Based on Grid-Density

Efficient Parallel Clustering Algorithm Based on Density

An Efficient Density-based Clustering Algorithm for Higher-Dimensional Data

An effective intersected grid clustering algorithm based on density estimation

Parallel Gravitational Clustering Based on Grid Partitioning for Large-Scale Data