Abstract:Maintaining the high availability of online systems requires reliable and fast online anomaly detection and fault localization. However, existing anomaly detection methods either suffer high training costs and low generalization capabilities or are designed and evaluated using offline data with limited efficacy in online usage. Furthermore, these methods' fault localization capabilities are often inadequate due to external observability constraints. Therefore, designing a new approach to address these limitations effectively is essential. To address the aforementioned limitations, this paper proposes a novel non-intrusive in situ anomaly detection and fault lo-calization framework, Detective-Dee. The proposed framework leverages a compressed sensing method for anomaly detection, which exhibits strong generalization capabilities and eliminates extensive training. Detective-Dee further improves its performance by incorporating three optimization techniques: concurrent sub-stitution sampling, Look-Up-Table-based similarity calculation, and substitution window-based threshold selection to improve parallelism and reduce computational and comparison overheads. Additionally, the framework adopts an innovative non-intrusive fault localization strategy based on anomaly detection triggering. This approach utilizes the dynamic instrumentation capabilities of eBPF, combined with extracting vulnerable function and function call chains through source code analysis, to improve the online anomaly detection capability and achieve robust fault localization with low overhead. To validate the effectiveness of Detective-Dee, we developed a prototype system and conducted a comprehensive evaluation. The results demonstrate that, compared to the state-of-the-art anomaly detection method, Detective-Dee exhibits a 4x improve-ment in anomaly detection speed while maintaining higher online and comparable offline detection ability. Furthermore, under 33 real-world fault cases across eight popular distributed systems, Detective-Dee successfully detects 31 cases and accurately locates 26 cases with less than 1% overhead, outperforming the state-of-the-art method.

Datactive: Data Fault Localization for Object Detection Systems

Dynamic Data Fault Localization for Deep Neural Networks

Automated Detection of Defects with Low Semantic Information in X-ray Images Based on Deep Learning

Detective-Dee: A Non-Intrusive in Situ Anomaly Detection and Fault Localization Framework

An Effective Data-Driven Approach for Localizing Deep Learning Faults

DeepFD: Automated Fault Diagnosis and Localization for Deep Learning Programs

A real-time anchor-free defect detector with global and local feature enhancement for surface defect detection

DSD-DA: Distillation-based Source Debiasing for Domain Adaptive Object Detection

Prototype-guided domain adaptive one-stage object detector for defect detection

PointDet: an Object Detection Framework Based on Human Local Features in the Task of Identifying Violations

Foregroundness-Aware Task Disentanglement and Self-Paced Curriculum Learning for Domain Adaptive Object Detection.

Localization-Aware Active Learning for Object Detection

Distribution Discrepancy and Feature Heterogeneity for Active 3D Object Detection

Spatial Self-Distillation for Object Detection with Inaccurate Bounding Boxes

Evaluation Tool to Diagnose Faults and Discrepancy in Semi-Automated or Manual Annotations in Ultrasound Cine Loops (Videos)

Yolo-sd: simulated feature fusion for few-shot industrial defect detection based on YOLOv8 and stable diffusion

Dynamic Knowledge Distillation with Noise Elimination for RGB-D Salient Object Detection

Surface defect detection of industrial components based on vision

Foreground separation knowledge distillation for object detection

AFDet: Toward More Accurate and Faster Object Detection in Remote Sensing Images

PDD-Net: A Precise Defect Detection Network Based on Point Set Representation