Abstract:Internet of Things (IoT) devices are constantly producing vast amounts of data, necessitating efficient storage and processing to extract useful information. However, the models used to extract relevant information from IoT data are often hindered by the lack of useful data and the ever-changing distribution of this data. This paper introduces an incremental data clustering technique on a continuous stream of data through a Dirichlet process-based approach that is adept at handling the formation of clusters in streaming data. The complete approach is twofold; firstly, it starts with an estimated distribution of data and allocates an incoming data point to the estimated data distribution. Secondly, it refines the estimated data distribution after the allocation of the current point and over the subsequent arrival of data points. The influx of data leads to greater challenges in determining clusters for incoming points and preserving the current clusters for improved decision-making. In this context, our proposed approach deals with the increasing amount of data using a selective elimination technique on both existing and incoming data. To assess the performance of the proposed approach, benchmark experiments have been performed using benchmark datasets. The results of the experiments demonstrate that the proposed model has a gain ranging from 2% to 4% as compared to the existing state-of-the-art and recent adaptive clustering approaches in terms of clustering accuracy with incremental data addition and variable clustering parameters. The proposed method shows a high gain in terms of running time ranging from 2% to 20% as compared to the existing approaches depending on the data reduction parameter. Furthermore, research findings through this work indicate that it is possible to set a trade-off between accuracy and running time by adjusting the elimination parameter depending on the requirements of the considered application.

Computational analysis of incremental clustering approaches for Large Data

Incremental CFS Clustering on Large Data.

Incremental Clustering: The Case for Extra Clusters

An Incremental Cfs Algorithm for Clustering Large Data in Industrial Internet of Things

KIHCDP: An Incremental Hierarchical Clustering Approach for IoT Data Using Dirichlet Process

An Efficient Incremental Algorithm for Clustering Based on Density

A Statistical Information-Based Clustering Approach in Distance Space

Clustering Large Attributed Information Networks: an Efficient Incremental Computing Approach.

Optimized Big Data K-means Clustering Using MapReduce

Efficient Finer-Grained Incremental Processing with MapReduce for Big Data

An Incremental Clustering Algorithm Based On Swarm Intelligence Theory

New Proximity Estimate for Incremental Update of Non-uniformly Distributed Clusters

An Incremental Clustering Algorithm Based on CFS

A Distributed Processing Framework of Incremental Text Clustering under the Background of Big Data

Adaptive Density-Based Spatial Clustering for Massive Data Analysis.

Incremental Subspace Clustering over Multiple Data Streams

Analysis of Different Approaches of Parallel Block Processing for K-Means Clustering Algorithm

Key Grids based Batch-Incremental CLIQUE Clustering Algorithm Considering Cluster Structure Changes

Scalable Hierarchical Agglomerative Clustering

Comparative Analysis of Optimization Strategies for K-means Clustering in Big Data Contexts: A Review

An efficient K-means algorithm for Massive Data