Abstract:Intrusion detection is an important technique in the defense-in-depth network security framework. Most current intrusion detection models lack the ability to process massive audit data streams for real-time anomaly detection. In this paper, we present an effective anomaly intrusion detection model based on Principal Component Analysis (PCA). The model is more suitable for high speed processing of massive data streams in real-time from various data sources by considering the frequency property of audit events than by use of the transition property or the correlation property. It can serve as a general framework that a practical Intrusion Detection Systems (IDS) can be implemented in various computing environments. In this method, a multi-pronged anomaly detection model is used to monitor various computer system and network behaviors. Three sources of data, system call data from the University of New Mexico (lpr) and from KLINNS Lab of Xi'an Jiaotong University (ftp), shell command data from AT&T Research laboratory, and network data from MIT Lincoln Lab, are used to validate the model and the method. The frequencies of individual system calls generated by one process and of individual commands embedded in one command block as well as features extracted in one network connection are transformed into an input data vector. Our method is employed to reduce the high dimensional data vectors and thus the detection is handled in a lower dimension with high efficiency and low use of system resources. The distance between a vector and its reconstruction in the reduced subspace is used for anomaly detection. Empirical results show that our model is promising in terms of detection accuracy and computational efficiency, and thus amenable for real-time intrusion detection.

A distributed data streaming algorithm for network-wide traffic anomaly detection

Detecting Distributed Network Traffic Anomaly with Network-Wide Correlation Analysis

PCA-subspace Method - is It Good Enough for Network-Wide Anomaly Detection.

Processing of Massive Audit Data Streams for Real-Time Anomaly Intrusion Detection

Distributed Online One-Class Support Vector Machine for Anomaly Detection over Networks

PCA Based Approach for Monitoring the Spatial-and-temporal Characteristics of P2P Traffic

The abnormal traffic detection scheme based on PCA and SSH

Correlated Anomaly Detection from Large Streaming Data

Anomaly Detection on Collective Moving Patterns: Manifold Learning Based Analysis of Traffic Streams.

Network Traffic Anomaly Detection Based on Sliding Window

Research on the detection of abnormal traffic for multi-channel network

Multi‐scale Anomaly Detection for High‐speed Network Traffic

Statistical Analysis for Origin-Destination Traffic Anomalies

Distributed Stealthy Traffic Anomaly Detection Based on Wavelet Packet Analysis

Robust PCA-Based Abnormal Traffic Flow Pattern Isolation and Loop Detector Fault Detection

Distributed Anomaly Detection in Edge Streams using Frequency based Sketch Datastructures

An efficient method for online detecting abnormal cascading pattern in distributed networks

Structural Analysis of Network Traffic Matrix via Relaxed Principal Component Pursuit

Anomaly Detection of Large Scale Network Based on Data Streams

Hankel-structured Tensor Robust PCA for Multivariate Traffic Time Series Anomaly Detection

Monitoring abnormal traffic flows based on independent component analysis