Abstract:Many important applications continuously generate data, such as financial transaction administration, satellite monitoring, network flow monitoring, and web information processing. The data mining results are always evolving with the newly generated data. Obviously, for the clustering task, it is better to incrementally update the new clustering results based on the old data rather than to recluster all of the data from scratch. The incremental clustering approach is an essential way to solve the problem of clustering with growing Big Data. This paper proposes a boundary-profile-based incremental clustering (BPIC) method to find arbitrarily shaped clusters with dynamically growing datasets. This method represents the existing clustering results with a collection of boundary profiles and discards the inner points of clusters rather than keep all data. It greatly saves both time and space storage costs. To identify the boundary profile, this paper presents a boundary-vector-based boundary point detection (BV-BPD) algorithm that summarizes the structure of the existing clusters. The BPIC method processes each new point in an online fashion and updates the clustering results in a batch mode. When a new point arrives, the BPIC method either immediately labels it or temporarily puts it into a bucket according to the relationship between the new data and the boundary profiles. A bucket is employed to distinguish the noise from the potential seeds of new clusters and alleviate the effects of data order. When the bucket is full, the BPIC method will cluster the data within it and update the clustering results. Thus, the BPIC method is insensitive to noise and the order of new data, which is critical for the robustness of the incremental clustering process. In the experiments, the performance of the boundary point detection algorithm BV-BPD is compared with the state-of-the-art method. The results show that the BV-BPD is better than the state-of-the-art method. Additionally, the performance of BPIC and other two incremental clustering methods are investigated in terms of clustering quality, time and space efficiency. The experimental results indicate that the BPIC method is able to get a qualified clustering result on a large dataset with higher time and space efficiency.

Incremental Clustering: The Case for Extra Clusters

An Incremental Clustering Algorithm Based On Swarm Intelligence Theory

Computational analysis of incremental clustering approaches for Large Data

Incremental Subspace Clustering over Multiple Data Streams

An Efficient Incremental Algorithm for Clustering Based on Density

Incremental CFS Clustering on Large Data.

Incremental clustering based on affinity propagation

Incremental Spectral Clustering by Efficiently Updating the Eigen-System.

An Incremental Clustering Algorithm Based on CFS

How to Perform Incremental Clustering - A SOM Based View

Incremental Clustering Using Information Bottleneck Theory

Key Grids based Batch-Incremental CLIQUE Clustering Algorithm Considering Cluster Structure Changes

An incremental clustering algorithm based on semantic concepts

An Incremental Clustering With Attribute Unbalance Considered For Categorical Data

An Incremental Clustering Algorithm Based On Sample Selection

An Incremental Algorithm for Clustering Search Results

A Tree-Based Incremental Overlapping Clustering Method Using the Three-Way Decision Theory

An Incremental Clustering Method Based On The Boundary Profile

Incremental Spectral Clustering with Application to Monitoring of Evolving Blog Communities

KIHCDP: An Incremental Hierarchical Clustering Approach for IoT Data Using Dirichlet Process

Clustering-based incremental learning for imbalanced data classification