Abstract:In the current era of information technology, blockchain is widely used in various fields, and the monitoring of the security and status of the blockchain system is of great concern. Online anomaly detection for the real-time stream data plays vital role in monitoring strategy to find abnormal events and status of blockchain system. However, as the high requirements of real-time and online scenario, online anomaly detection faces many problems such as limited training data, distribution drift, and limited update frequency. In this paper, we propose an adaptive stream outlier detection method (ASOD) to overcome the limitations. It first designs a K-nearest neighbor Gaussian mixture model (KNN-GMM) and utilizes online learning strategy. So, it is suitable for online scenarios and does not rely on large training data. The K-nearest neighbor optimization limits the influence of new data locally rather than globally, thus improving the stability. Then, ASOD applies the mechanism of dynamic maintenance of Gaussian components and the strategy of dynamic context control to achieve self-adaptation to the distribution drift. And finally, ASOD adopts a dimensionless distance metric based on Mahalanobis distance and proposes an automatic threshold method to accomplish anomaly detection. In addition, the KNN-GMM provides the life cycle and the anomaly index for continuous tracking and analysis, which facilities the cause analysis and further interpretation and traceability. From the experimental results, it can be seen that ASOD achieves near-optimal F1 and recall on the NAB dataset with an improvement of 6% and 20.3% over the average, compared to baselines with sufficient training data. ASOD has the lowest F1 variance among the five best methods, indicating that it is effective and stable for online anomaly detection on stream data.

Log-Based Anomaly Detection With Robust Feature Extraction and Online Learning

Robust Log-Based Anomaly Detection on Unstable Log Data

LogOnline: A Semi-Supervised Log-Based Anomaly Detector Aided with Online Learning Mechanism.

Robust and Transferable Log-based Anomaly Detection.

OMLog: Online Log Anomaly Detection for Evolving System with Meta-learning

Log-based Anomaly Detection based on EVT Theory with feedback

A Robust Wide & Deep Learning Framework for Log-Based Anomaly Detection

Enhancing Robustness of On-Line Learning Models on Highly Noisy Data

Robust Anomaly Detection in Feature-Evolving Time Series

An Empirical Investigation of Practical Log Anomaly Detection for Online Service Systems.

RAD: On-line Anomaly Detection for Highly Unreliable Data

Robust Anomaly Detection for Time-series Data

ADA: Adaptive Deep Log Anomaly Detector

Research on anomaly detection and real-time reliability evaluation with the log of cloud platform

An Extreme Learning Machine for Unsupervised Online Anomaly Detection in Multivariate Time Series

LogEncoder: Log-Based Contrastive Representation Learning for Anomaly Detection

Robust and Transferable Anomaly Detection in Log Data using Pre-Trained Language Models

Robust and Rapid Adaption for Concept Drift in Software System Anomaly Detection.

FastLogAD: Log Anomaly Detection with Mask-Guided Pseudo Anomaly Generation and Discrimination

ASOD: an Adaptive Stream Outlier Detection Method Using Online Strategy

A Survey of Deep Anomaly Detection for System Logs