Abstract:Anomaly detection is a challenging problem that has been researched within a variety of application domains. In network intrusion detection, anomaly based techniques are particularly attractive because of their ability to identify previously unknown attacks without the need to be programmed with the specific signatures of every possible attack. There is a significant body of work in anomaly based intrusion detection applying statistical analysis, data-mining, and machine learning disciplines. However despite more than two decades of active research, there is a striking lack of anomaly based systems in commercial use today. Many of the currently proposed anomaly based systems do not adequately address a series of challenges making them unsuitable for operational deployment. In existing approaches, every step of the anomaly detection process requires expert manual intervention. This dependence makes developing practical systems extremely challenging. In this thesis, we integrate the strengths of machine learning and quality-of-service mitigation techniques for network anomaly detection, and build an operationally practical framework for anomaly- based network intrusion detection. We propose methods for self-adaptive, self-tuning, self-optimizing, and automatically responsive network anomaly detection. In specific, we propose and develop methods for adaptive input normalization adjusting scaling parameters online based on evolving values in observed traffic patterns, adaptive algorithms for flow-based network anomaly detection that respond to feedback to account for concept drift, and evolving methods for aggregated alert correlation that consolidate individual alarms into network events. We propose and design a model for dictating optimal performance in an anomaly detection system and reinforcement learning algorithms for automated tuning and optimization and a confidence forwarding model to support automated response. Furthermore, we develop a fair bandwidth sharing and delay differentiation mechanism for scalable automated response that insulates network resources from malicious traffic while minimizing collateral damage. We develop a prototype network anomaly detection system that integrates the proposed and developed techniques. We evaluate developed approaches using the 1999 Knowledge Discovery and Data-mining Cup and MAWI Lab datasets, but also we create a new dataset based on a combination of live network traces and controlled simulated data injects. Results demonstrate the effectiveness and capability of automated means.

Recent Advances in Machine Learning-based Anomaly Detection for Industrial Control Networks

Cross-domain Anomaly Detection for Power Industrial Control System

Machine Learning in Industrial Control System Security: A Survey

Evaluation of Machine Learning-based Anomaly Detection Algorithms on an Industrial Modbus/TCP Data Set

Machine learning for intrusion detection in industrial control systems: challenges and lessons from experimental evaluation

Instruction Detection in SCADA/Modbus Network Based on Machine Learning

Machine Learning for Intrusion Detection in Industrial Control Systems: Applications, Challenges, and Recommendations

Machine Learning in Network Anomaly Detection: A Survey

Enhanced Anomaly Detection in Industrial Control Systems aided by Machine Learning

Anomaly Detection for Industrial Control Networks using Machine Learning with the help from the Inter-Arrival Curves

Anomaly events classification and detection system in critical industrial internet of things infrastructure using machine learning algorithms

Industrial Anomaly Detection and Attack Classification Method Based on Convolutional Neural Network.

Automated network anomaly detection with learning, control and mitigation

Using machine learning to detect network intrusions in industrial control systems: a survey

Function-Aware Anomaly Detection Based on Wavelet Neural Network for Industrial Control Communication

Machine learning in industrial control system (ICS) security: current landscape, opportunities and challenges

Anomaly Detection in Industrial Machinery using IoT Devices and Machine Learning: a Systematic Mapping

Online Anomaly Detection of Industrial IoT Based on Hybrid Machine Learning Architecture

Compared Insights on Machine-Learning Anomaly Detection for Process Control Feature

Hybrid Statistical-Machine Learning for Real-Time Anomaly Detection in Industrial Cyber–Physical Systems

Machine Learning-Based Threat Identification of Industrial Internet