Abstract:Mapping origin-destination (OD) network traffic is pivotal for network management and proactive security tasks. However, lack of sufficient flow-level measurements as well as potential anomalies pose major challenges towards this goal. Leveraging the spatiotemporal correlation of nominal traffic, and the sparse nature of anomalies, this paper brings forth a novel framework to map out nominal and anomalous traffic, which treats jointly important network monitoring tasks including traffic estimation, anomaly detection, and traffic interpolation. To this end, a convex program is first formulated with nuclear and $\ell_1$-norm regularization to effect sparsity and low rank for the nominal and anomalous traffic with only the link counts and a {\it small} subset of OD-flow counts. Analysis and simulations confirm that the proposed estimator can {\em exactly} recover sufficiently low-dimensional nominal traffic and sporadic anomalies so long as the routing paths are sufficiently "spread-out" across the network, and an adequate amount of flow counts are randomly sampled. The results offer valuable insights about data acquisition strategies and network scenaria giving rise to accurate traffic estimation. For practical networks where the aforementioned conditions are possibly violated, the inherent spatiotemporal traffic patterns are taken into account by adopting a Bayesian approach along with a bilinear characterization of the nuclear and $\ell_1$ norms. The resultant nonconvex program involves quadratic regularizers with correlation matrices, learned systematically from (cyclo)stationary historical data. Alternating-minimization based algorithms with provable convergence are also developed to procure the estimates. Insightful tests with synthetic and real Internet data corroborate the effectiveness of the novel schemes.

Structure Regularized Traffic Monitoring Model for Traffic Matrix Estimation and Anomaly Detection

Structure Regularized Traffic Monitoring for Traffic Matrix Estimation and Anomaly Detection by Link-Load Measurements.

SLRTA: A sparse and low-rank tensor-based approach to internet traffic anomaly detection

STAP: A Spatio-Temporal Correlative Estimating Model for Improving Quality of Traffic Data

Traffic Dynamics Exploration and Incident Detection Using Spatiotemporal Graphical Modeling

Structural Analysis of Network Traffic Matrix via Relaxed Principal Component Pursuit

Traffic System Anomaly Detection Using Spatiotemporal Pattern Networks

An AI-Augmented Kalman Filter Approach to Monitoring Network Traffic Matrix

An Accurate Approach for Traffic Matrix Estimation in Large-Scale Backbone Networks.

Efficient and Robust Freeway Traffic Speed Estimation under Oblique Grid using Vehicle Trajectory Data

Monitoring the Spatial-Temporal Effect of Internet Traffic Based on Random Matrix Theory

Traffic Sensor Health Monitoring Using Spatiotemporal Graphical Modeling

Traffic Model Analysis for Anomaly Detection

Traffic Matrix Estimation Using Square Root Filtering/Smoothing Algorithm

Urban Traffic Pattern Analysis and Applications Based on Spatio-Temporal Non-Negative Matrix Factorization

Estimating Traffic and Anomaly Maps via Network Tomography

Spatiotemporal Regularized Tucker Decomposition Approach for Traffic Data Imputation

Anomaly Detection Based on Traffic Information Structure

Detecting Urban Traffic Anomalies Using Traffic-Monitoring Data

Monitoring of Link Traffic Patterns Based on Random Matrix Theory

Structured Nonnegative Matrix Factorization for Traffic Flow Estimation of Large Cloud Networks