Abstract:Web applications are crucial infrastructures in the modern society, which have high demand of reliability and security. However, their frontend can be manipulable by the clients (e.g., the frontend code can be modified to bypass some validation steps), which incurs the runtime anomaly when operating the web service. Existing state-of-the-art anomaly detectors largely learn a deep learning model from the collected logs to predict abnormal logs with a probability. While effective in general, those approaches can suffer from (1) inaccuracy caused by subtle difference between the normal and abnormal/attack logs and (2) additional efforts for root cause analysis. In this work, we propose WebNorm, an anomaly detection approach to detect and explain the attack-caused anomalies on web applications in a unified way. Our rationale lies in learning the behaviorial normalities of a running web application as invariants. The normalities are designed regarding data normality (e.g., what information must be consistent across different events), flow normality (e.g., what events must happen under certain circumstances), and common-sense normality (e.g., what is the normal range of some parameters). The violation of the invariants indicates both the alarm and its explanation. WebNorm first monitors the normal behaviors of subject application and captures its information flows between entities such as frontend, service, and database. Then, it learns the behaviorial normalities in terms of logical rules so that it can detect and explain behaviorial anomaly by the inconsistency between the learned normalities and the runtime application behaviors. We model the invariants as first-order logics, transferrable to executable Python scripts to generate alarm with explainable root cause. Our extensive experiment shows that, on detecting the tamper attacks on the web applications as TrainTicket and NiceFish. WebNorm improves the precision and the recall of the baselines such as LogAnomaly, LogRobust, DeepLog, NeuralLog, PLELog, ReplicaWatcher by more than 56.1% and 35.1% respectively, serving as a new state-of-the-art anomaly detection solution.

Anomaly Detection in Certificate Transparency Logs

Isolation-Based Anomaly Detection

A Classifier-Based Approach to Multi-Class Anomaly Detection for Astronomical Transients

: Towards Fine-Grained Unknown Class Detection Against the Open-Set Attack Spectrum with Variable Legitimate Traffic

On the Detection of Fake Certificates Via Attribute Correlation

A Novel Anomaly Detection Algorithm Based On Trident Tree

Anomaly Detection in Network Management System Based on Isolation Forest

Anomaly Detection in Blockchain Transactions: A Comparative Study of Isolation Forest, K-Means Clustering, and LSTM Models

Deep Optimal Isolation Forest with Genetic Algorithm for Anomaly Detection

CTF: Anomaly Detection in High-Dimensional Time Series with Coarse-to-Fine Model Transfer

OptIForest: Optimal Isolation Forest for Anomaly Detection

A Classifier-Based Approach to Multi-Class Anomaly Detection Applied to Astronomical Time-Series

A Novel Privacy-Preserving Distributed Anomaly Detection Method

Anomaly Detection in Networks with Application to Financial Transaction Networks

Anomaly detection in microservice environments using distributed tracing data analysis and NLP

Anomaly detection in cross-country money transfer temporal networks

Detecting and Explaining Anomalies Caused by Web Tamper Attacks Via Building Consistency-based Normality

Isolation-based Anomaly Detection Using Nearest-Neighbor Ensembles

LSHiForest: A Generic Framework for Fast Tree Isolation Based Ensemble Anomaly Analysis.

Anomaly Detection Based on Isolation Mechanisms: A Survey

Comprehensive Analysis and Evaluation of Anomalous User Activity in Web Server Logs