Abstract:Web applications are crucial infrastructures in the modern society, which have high demand of reliability and security. However, their frontend can be manipulable by the clients (e.g., the frontend code can be modified to bypass some validation steps), which incurs the runtime anomaly when operating the web service. Existing state-of-the-art anomaly detectors largely learn a deep learning model from the collected logs to predict abnormal logs with a probability. While effective in general, those approaches can suffer from (1) inaccuracy caused by subtle difference between the normal and abnormal/attack logs and (2) additional efforts for root cause analysis. In this work, we propose WebNorm, an anomaly detection approach to detect and explain the attack-caused anomalies on web applications in a unified way. Our rationale lies in learning the behaviorial normalities of a running web application as invariants. The normalities are designed regarding data normality (e.g., what information must be consistent across different events), flow normality (e.g., what events must happen under certain circumstances), and common-sense normality (e.g., what is the normal range of some parameters). The violation of the invariants indicates both the alarm and its explanation. WebNorm first monitors the normal behaviors of subject application and captures its information flows between entities such as frontend, service, and database. Then, it learns the behaviorial normalities in terms of logical rules so that it can detect and explain behaviorial anomaly by the inconsistency between the learned normalities and the runtime application behaviors. We model the invariants as first-order logics, transferrable to executable Python scripts to generate alarm with explainable root cause. Our extensive experiment shows that, on detecting the tamper attacks on the web applications as TrainTicket and NiceFish. WebNorm improves the precision and the recall of the baselines such as LogAnomaly, LogRobust, DeepLog, NeuralLog, PLELog, ReplicaWatcher by more than 56.1% and 35.1% respectively, serving as a new state-of-the-art anomaly detection solution.

Detecting and Explaining Anomalies Caused by Web Tamper Attacks Via Building Consistency-based Normality

Anomaly-Based Web Attack Detection: A Deep Learning Approach

Anomaly-Based Web Attack Detection

Web-Based Application Anomaly Detection Based On Efficient Frequent Pattern Mining

A Novel Anomaly Detection Approach for Mitigating Web-Based Attacks Against Clouds

Natural Language Processing-based Model for Log Anomaly Detection

WSAD: An Unsupervised Web Session Anomaly Detection Method

Interactive Context-Aware Anomaly Detection Guided by User Feedback

TargetVue: Visual Analysis of Anomalous User Behaviors in Online Communication Systems

Anomaly Detection in the Open World: Normality Shift Detection, Explanation, and Adaptation.

LogAnomaly: Unsupervised Detection of Sequential and Quantitative Anomalies in Unstructured Logs

Log-based Anomaly Detection based on EVT Theory with feedback

Abnormal Traffic Detection: Traffic Feature Extraction and DAE-GAN With Efficient Data Augmentation

Robust Log-Based Anomaly Detection on Unstable Log Data

Data mining methods for anomaly detection of HTTP request exploitations

Comprehensive Analysis and Evaluation of Anomalous User Activity in Web Server Logs

A Transform Domain-Based Anomaly Detection Approach To Network-Wide Traffic

Study of Intrusion Detecton Mondels for Web Applications

Anomaly Detection Based on Web Users' Browsing Behaviors

A Survey on Explainable Anomaly Detection

AnomalyNCD: Towards Novel Anomaly Class Discovery in Industrial Scenarios