Abstract:There has been a large number of studies in interpretable and explainable ML for cybersecurity, in particular, for intrusion detection. Many of these studies have significant amount of overlapping and repeated evaluations and analysis. At the same time, these studies overlook crucial model, data, learning process, and utility related issues and many times completely disregard them. These issues include the use of overly complex and opaque ML models, unaccounted data imbalances and correlated features, inconsistent influential features across different explanation methods, the inconsistencies stemming from the constituents of a learning process, and the implausible utility of explanations. In this work, we empirically demonstrate these issues, analyze them and propose practical solutions in the context of feature-based model explanations. Specifically, we advise avoiding complex opaque models such as Deep Neural Networks and instead using interpretable ML models such as Decision Trees as the available intrusion datasets are not difficult for such interpretable models to classify successfully. Then, we bring attention to the binary classification metrics such as Matthews Correlation Coefficient (which are well-suited for imbalanced datasets. Moreover, we find that feature-based model explanations are most often inconsistent across different settings. In this respect, to further gauge the extent of inconsistencies, we introduce the notion of cross explanations which corroborates that the features that are determined to be impactful by one explanation method most often differ from those by another method. Furthermore, we show that strongly correlated data features and the constituents of a learning process, such as hyper-parameters and the optimization routine, become yet another source of inconsistent explanations. Finally, we discuss the utility of feature-based explanations.

AI/ML for Network Security: The Emperor has no Clothes

Which Neural Network Makes More Explainable Decisions? an Approach Towards Measuring Explainability

Evaluating The Explainability of State-of-the-Art Machine Learning-based Online Network Intrusion Detection Systems

"How do I fool you?": Manipulating User Trust via Misleading Black Box Explanations

A Critical Assessment of Interpretable and Explainable Machine Learning for Intrusion Detection

Explainable Artificial Intelligence (XAI) to Enhance Trust Management in Intrusion Detection Systems Using Decision Tree Model

Explaining Tree Model Decisions in Natural Language for Network Intrusion Detection

Evaluating Standard Feature Sets Towards Increased Generalisability and Explainability of ML-based Network Intrusion Detection

Explaining Network Intrusion Detection System Using Explainable AI Framework

A deep learning anomaly detection framework with explainability and robustness

SecurityNet: Assessing Machine Learning Vulnerabilities on Public Models

Explainability using Decision Trees and Monte Carlo Simulations

Improving IoT Security With Explainable AI: Quantitative Evaluation of Explainability for IoT Botnet Detection

Explaining Decisions in ML Models: a Parameterized Complexity Analysis

AUTOLYCUS: Exploiting Explainable AI (XAI) for Model Extraction Attacks against Interpretable Models

TRUST XAI: Model-Agnostic Explanations for AI With a Case Study on IIoT Security

Explainable and Transferable Adversarial Attack for ML-Based Network Intrusion Detectors

Interpretability in Safety-Critical FinancialTrading Systems

Towards the Science of Security and Privacy in Machine Learning

Distance-Restricted Explanations: Theoretical Underpinnings & Efficient Implementation