Abstract:Internet has become an essential part of the daily life for billions of users worldwide, who are using a large variety of network services and applications everyday. However, there have been serious security problems and network failures that are hard to resolve, for example, Botnet attacks, polymorphic worm/virus spreading, DDoS, and flash crowds. To address many of these problems, we need to have a network-wide view of the traffic dynamics, and more importantly, be able to detect traffic anomaly in a timely manner. Existing network measurement and monitoring solutions often suffer scalability problems caused by overly large processing, space, or communication overhead. In this paper, we propose to develop sketch-based algorithms for network-wide anomaly detection that are able to detect both high-profile and coordinated low-profile traffic anomalies as an outlier in the regular traffic patterns. Our approach is based on the spatial analysis by using traffic measurements from multiple monitors. Spatial analysis methods have been proved to be effective in detecting network-wide traffic anomalies that are not detectable at a single monitor. To our knowledge, Principle Component Analysis (PCA) is the best-known spatial detection method for the coordinated low-profile traffic anomalies. However, existing PCA-based solutions have scalability problems in that they require O(m 2 n) running time and O(mn) space to analyze traffic measurements from m aggregated traffic flows within a sliding window of the length n, which makes it often infeasible to be deployed for monitoring large-scale high-speed networks. We propose two novel sketch-based algorithms for PCA-based traffic anomaly detection in a distributed fashion. Our algorithm can archive O(w log n) running time and O(w log 2 n) space at local monitors, and O(m 2 log n) running time and O(m log n) space at Network Operation Center, where w denotes the maximum number of traffic flows at a single local monitor. Additionally, our algorithm can protect the privacy of traffic measurements for Internet Service Providers.

A Real-Time Detection Approach to Network Traffic Anomalies in Communication Networks

A Haar Transform-Based Detection Approach to Network Traffic Anomalies in Power Telecommunication Access Networks

A Time-Frequency Detecting Method for Network Traffic Anomalies

Network Anomaly Detection Based on Statistical Approach and Time Series Analysis

Research on the detection of abnormal traffic for multi-channel network

Spatio-Temporal Network Traffic Estimation and Anomaly Detection Based on Convolutional Neural Network in Vehicular Ad-Hoc Networks

Detection Network Anomalies Based on Packet and Flow Analysis

Analyzing and Visualizing Anomalies and Events in Time Series of Network Traffic

A Transform Domain-Based Anomaly Detection Approach To Network-Wide Traffic

A Novel Change-Point Detection Approach for Monitoring High-Dimensional Traffics in Distributed Systems

Detection of network traffic anomaly based on instantaneous parameters analysis

Sketch-based Network-wide Traffic Anomaly Detection

Fast Network Traffic Anomaly Detection Based on Iteration

Network Traffic Anomaly Detection Based on Sliding Window

An Adaptive Profile-Based Approach for Detecting Anomalous Traffic in Backbone.

A scheme for fast network traffic anomaly detection

Anomaly internet network traffic detection by kernel principle component classifier

A distributed data streaming algorithm for network-wide traffic anomaly detection

A Feature Analysis Approach to Network Traffic in Communication Networks

Algorithm of Traffic Anomaly Detection Based on Steady Model

A BasisEvolution Framework for Network Traffic Anomaly Detection