Abstract:Network traffic anomaly detection is critical for advanced network applications. However, network traffic monitoring data arrive in a streaming fashion and could be infinite, which makes the offline algorithms that attempt to store the entire stream monitoring data for analysis not scalable. To well utilize the strong ability of tensor model, we use a tensor to represent the prior non-anomalous traffic matrices and propose a novel unsupervised anomaly detection framework that can be used to detect anomalies in a streaming fashion by making only one pass over the data while utilizing limited storage. In the framework, we propose a succinct tensor sketch to maintain, in a streaming model, the subspace that can well represent all prior non-anomalous data detected. Using the subspace, anomalies in each new incoming traffic monitoring data can be quickly detected based on a simple outlier score calculation. Further, we prove that the tensor sketch is mergeable. Exploiting this property, we propose a distributed anomaly detection framework in which the distributed node only needs to upload its succinct tensor sketch instead of the raw monitoring data to the central node to calculate the global subspace of the whole network, which greatly saves the transmission cost. We theoretically prove that our tensor sketch based anomaly detection algorithm compares favorably with the offline approach which calculates the subspace based on expensive global Singular Value Decomposition (SVD). The experimental results demonstrate the effectiveness and efficiency of our approach over other popular online anomaly detection algorithms.

On Anomalous Hotspot Discovery in Graph Streams

Socialanalysis: A Real-Time Query And Mining System From Social Media Data Streams

Outlier detection in graph streams

Fast Memory-efficient Anomaly Detection in Streaming Heterogeneous Graphs

Anomaly Detection of Network Streams Via Dense Subgraph Discovery

Voila: Visual Anomaly Detection and Monitoring with Streaming Spatiotemporal Data

Subtle Anomaly Detection in Dynamic Networks Using Graph Spectra

Anomaly Detection over Streaming Graphs with Finger-Based Higher-Order Graph Sketch

Sketch-Based Anomaly Detection in Streaming Graphs

Node Anomaly Detection for Homogeneous Distributed Environments

Conditional Heavy Hitter Monitoring and Application of Heterogeneous Graph Streams Based on Sketches

Burst Hotspots Dynamic Detection and Tracking on Large-Scale Text Stream

A Sketch Based Game Theoretic Approach to Detect Anomalous Dense Sub-Communities in Large Data Streams

A Framework for Spotting Anomaly

A Novel Hotspot Location Algorithm for P2P Streaming Distribution Networking.

On-Line Network Traffic Anomaly Detection Based on Tensor Sketch

A distributed data streaming algorithm for network-wide traffic anomaly detection

Continuous, Online Anomaly Region Detection and Tracking in Networks.

Streaming Anomaly Detection Using Randomized Matrix Sketching

Dynamic Community Detection in Weighted Graph Streams.

Spatiotemporal Anomaly Visualization for Large-scale Dynamic Networks