Abstract:Packet sampling methods such as Cisco's NetFlow are widely employed by large networks to reduce the amount of traffic data measured. A key problem with packet sampling is that it is inherently a lossy process, discarding (potentially useful) information. In this paper, we empirically evaluate the impact of sampling on anomaly detection metrics. Starting with unsampled flow records collected during the Blaster worm outbreak, we reconstruct the underlying packet trace and simulate packet sampling at increasing rates. We then use our knowledge of the Blaster anomaly to build a baseline of normal traffic (without Blaster), against which we can measure the anomaly size at various sampling rates. This approach allows us to evaluate the impact of packet sampling on anomaly detection without being restricted to (or biased by) a particular anomaly detection method.We find that packet sampling does not disturb the anomaly size when measured in volume metrics such as the number of bytes and number of packets, but grossly biases the number of flows. However, we find that recently proposed entropy-based summarizations of packet and flow counts are affected less by sampling, and expose the Blaster worm outbreak even at higher sampling rates. Our findings suggest that entropy summarizations are more resilient to sampling than volume metrics. Thus, while not perfect, sampling still preserves sufficient distributional structure, which when harnessed by tools like entropy, can expose hard-to-detect scanning anomalies.

Novel Traffic Sampling Method for Anomaly Detection

Accurate and Efficient Traffic Monitoring Using Adaptive Non-Linear Sampling Method

Time Stratified Adaptive Packet Sampling Based on Traffic Load

Analysis Method of Multi-Source Flow Characteristics and Its Application in Anomaly Detection

Network anomaly detection and classification via opportunistic sampling

Anomaly Detection Based on Traffic Information Structure

Network traffic monitoring based on mining frequent patterns

Impact of packet sampling on anomaly detection metrics

A2C: Anti-Attack Counters for Traffic Measurement

Survey of Research on Anomaly Detection of Wavelet Analysis Based on Network Traffic

Network Traffic Anomaly Detection Method Based on Multi scale Residual Feature

Monitoring Abnormal Network Traffic Based on Blind Source Separation Approach

An Improved TRW Algorithm Based on Packet Sampling

Research on the detection of abnormal traffic for multi-channel network

High-speed anomaly traffic detection based on staged frequency domain features

Methods of real-time screening and reconstruction for dynamic traffic abnormal data

Monitoring abnormal traffic flows based on independent component analysis

A method of on-line fast identification for the chaos in traffic flow

Symmetry Degree Measurement and Its Applications to Anomaly Detection

Time-Driven vs packet-driven: a deep study on traffic sampling

Traffic Flow Anomaly Detection Based on Wavelet Denoising and Support Vector Regression