Abstract:The widespread use of Internet-of-Things (IoT) technologies, smartphones, and social media services generates huge amounts of data streaming at high velocity. Automatic interpretation of these rapidly arriving data streams is required for the timely detection of interesting events that usually emerge in the form of clusters. This article proposes a new relative of the visual assessment of the cluster tendency (VAT) model, which produces a record of structural evolution in the data stream by building a cluster heat map of the entire processing history in the stream. The existing VAT-based algorithms for streaming data, called inc-VAT/inc-iVAT and dec-VAT/dec-iVAT, are not suitable for high-velocity and high-volume streaming data because of high memory requirements and slower processing speed as the accumulated data increases. The scalable iVAT (siVAT) algorithm can handle big batch data, but for streaming data, it needs to be (re)applied everytime a new datapoint arrives, which is not feasible due to the associated computation complexities. To address this problem, we propose an incremental siVAT algorithm, called inc-siVAT, which deals with the streaming data in chunks. It first extracts a small size smart sample using an intelligent sampling scheme, called maximin random sampling (MMRS), then incrementally updates the smart sample points on the fly, using our novel incremental MMRS (inc-MMRS) algorithm, to reflect changes in the data stream after each chunk is processed, and finally, produces an incrementally built iVAT image of the updated smart sample, using the inc-VAT/inc-iVAT and dec-VAT/dec-iVAT algorithms. These images can be used to visualize the evolving cluster structure and for anomaly detection in streaming data. Our method is illustrated with one synthetic and four real datasets, two of which evolve significantly over time. Our numerical experiments demonstrate the algorithm's ability to successfully identify anomalies and visualize-changing cluster structure in streaming data.

Incremental Multi-Dimension Scaling Visualization Mining Method for Data Stream

An Incremental Dimensionality Reduction Method for Visualizing Streaming Multidimensional Data

Using Visualization to Improve Clustering Analysis on Heterogeneous Information Network.

A Feature-Based Data Mining Method for Fluid Dynamics

A Parallel Framework for Streaming Dimensionality Reduction

Dynamic Incremental SVM Learning Algorithm for Mining Data Streams

A scalable supervised algorithm for dimensionality reduction on streaming data

A spectral method for assessing and combining multiple data visualizations

Scalable Multivariate Volume Visualization and Analysis based on Dimension Projection and Parallel Coordinates.

Incremental Subspace Clustering over Multiple Data Streams

A three-dimensional display for big data sets

Visualization of Financial Data Based on SOM and Gravitational Field Clustering

The Research of Visualization Data Mining in Industrial Data

Visual Structural Assessment and Anomaly Detection for High-Velocity Data Streams

A New Approach to Discover Interlacing Data Structures in High-Dimensional Space

A Quick and Feature based Visualization Algorithm for Large-scale Flow Data.

Scale-dependent Spatial Data Mining

Incremental frequent itemsets mining based on frequent pattern tree and multi-scale

A News Video Mining Method Based On Statistical Analysis And Visualization

The Research Of Visualization Data Mining Based On Manifold Learning

Scalable Volume Visualization for Big Scientific Data Modeled by Functional Approximation