Abstract:In an increasingly digitalized world, cybersecurity has emerged as a critical component of safeguarding sensitive information and infrastructure from malicious threats. The threat actors are often in line or even one step ahead of the defense, causing the increasing reliance of security teams on artificial intelligence while trying to detect zero-day attacks. However, most of the cybersecurity solutions based on artificial intelligence that can be found in the literature are trained and tested on reference datasets that are at least five or more years old, which gives a vague insight into their security performances. Moreover, they often tend to be designed as isolated, self-focused components. The aim of this paper is to design and implement a modular network intrusion detection architecture capable of simulating cyberattacks based on real-world scenarios while evaluating its defense capabilities. The architecture is designed as a full pipeline from real-time network data collection and transformation to threat-information presentation and visualization, with a pre-trained artificial intelligence module at its core. Well-known components like CICFlowMeter, Prometheus, and Grafana are used and modified to fit our data preparation and core modules to form the proposed architecture for real-world network traffic security monitoring. For the sake of cyberattack simulation, the proposed architecture is situated within a virtual environment, surrounded by the Kali Linux-based penetration simulation agent on one side and a vulnerable agent on the other. The intrusion detection artificial intelligence module is trained on the CICIDS-2017 dataset, and it is demonstrated using the proposed architecture that, despite being trained on an outdated dataset, the trained module is still effective in detecting sophisticated modern attacks. Two case studies are given to illustrate how modular architectures and virtual environments can be valuable tools to assess the security properties of artificial intelligence-based solutions through simulation in real-world scenarios.

Real-Time Cyber Analytics Data Collection Framework

Socialanalysis: A Real-Time Query And Mining System From Social Media Data Streams

Real-Time-Data Analytics in Raw Materials Handling

Parameterization and Performance Analysis of a Scalable, near Real-Time Packet Capturing Platform

A Dynamic Ensemble Learning Framework for Data Stream Analysis and Real-Time Threat Detection

Enhancing Cyber Security through Predictive Analytics: Real-Time Threat Detection and Response

A Real-Time Streaming System for Customized Network Traffic Capture

Toward a monitoring and threat detection system based on stream processing as a virtual network function for big data

DBD-Guardian and Privacy-Aware Near Real-Time Cybersecurity Analytics

A Framework for an Event Driven Video Surveillance System

Real-time big data processing for anomaly detection: A Survey

AI-powered threat detection in surveillance systems: A real-time data processing framework

PARIS: A Practical, Adaptive Trace-Fetching and Real-Time Malicious Behavior Detection System

Real-time Text Analytics Pipeline Using Open-source Big Data Tools

TrustData: Trustworthy and Secured Data Collection for Event Detection in Industrial Cyber-Physical System

An Artificial Intelligence-Based Real-Time Monitoring Framework for Time Series

Cybersecurity Knowledge Graph Enables Targeted Data Collection for Cybersecurity Analysis.

A Hierarchical Security Events Correlation Model for Real-time Cyber Threat Detection and Response

Modular deep learning-based network intrusion detection architecture for real-world cyber-attack simulation

A Novel Real-Time Aggregation Method on Network Security Events.

Tracking Real-time Anomalies in Cyber-Physical Systems Through Dynamic Behavioral Analysis