Abstract:Ever since the internet became part of the everyday lives of humans providing network security has been considered of utmost importance. Over the years lot of time and energy has been devoted by people in the research community and industry to provide better, improved and secure mechanisms to ensure secure communications on the internet. Amongst the many fields of study, the most prominent and ever evolving one has been the study of network traffic for attack detection and mitigation. The advent of new technologies has led to an increase in the pace of network based attacks and therefore novel modified approaches are needed to be able to cope with these latest trends. Distributed machine learning with the development of new tools and frameworks like RDD structure in Apache Spark provides an immense scope of growth in this direction. Moreover, the dynamic nature of present day network traffic called concept drift has also necessitated studying solutions from a different angle. We, therefore, in this paper have worked on distributed machine learning based ensemble techniques to detect the presence of concept drift in network traffic and detect network based attacks. The work has been done in three parts. Firstly, two classifiers, namely, Random Forest and Logistic Regression have been used as level '0′ learners and Support Vector Machine has been used as level '1′ learner. Secondly, to handle the process of concept drift we have used a sliding window based <i>K</i>-means clustering. And thirdly ensemble based techniques for detection of attacks in the traffic. The experiments have been performed on three datasets, namely, the NSL-KDD dataset, the CIDDS-2017 dataset and generated Testbed dataset. These tests have been conducted on different machines by varying the number of executor cores to study time latency in a distributed environment. An accuracy of 93% on NSL-KDD, 98% on CIDDS-2017 and 97% on Testbed datasets for SVM based blending model have been achieved.

Anomaly Detection in Audio with Concept Drift using Adaptive Huffman Coding

Concept drift challenge in multimedia anomaly detection: A case study with facial datasets

Did You Hear That? Introducing AADG: A Framework for Generating Benchmark Data in Audio Anomaly Detection

Robust Audio Anomaly Detection

Concept Drift Detection and Adaptation with Weak Supervision on Streaming Unlabeled Data

MIMII-Gen: Generative Modeling Approach for Simulated Evaluation of Anomalous Sound Detection System

Learning to Adapt to Domain Shifts with Few-shot Samples in Anomalous Sound Detection

A Copula-Driven Unsupervised Learning Framework for Anomaly Detection with Multivariate Heterogeneous Data

CDA-PDDWE: Concept Drift-Aware Performance-Based Diversified Dynamic Weighted Ensemble for Non-stationary Environments

Multimedia Datasets for Anomaly Detection: A Review

Autoencoder-based Anomaly Detection in Streaming Data with Incremental Learning and Concept Drift Adaptation

HybridAD: A Hybrid Model-Driven Anomaly Detection Approach for Multivariate Time Series

Unsupervised Incremental Learning with Dual Concept Drift Detection for Identifying Anomalous Sequences

Drift-aware Anomaly Detection for Non-stationary Time Series

Video Anomaly Detection via Spatio-Temporal Pseudo-Anomaly Generation : A Unified Approach

Distributed anomaly detection using concept drift detection based hybrid ensemble techniques in streamed network data

Hierarchical Context Representation and Self-adaptive Thresholding for Multivariate Anomaly Detection

Unsupervised Anomaly Detection and Localization of Machine Audio: A GAN-based Approach

Spatial-based Bayesian Hidden Markov Models with Dirichlet Mixtures for Video Anomaly Detection

Exploration of Audio Quality Assessment and Anomaly Localisation Using Attention Models

ADTCD: An Adaptive Anomaly Detection Approach Toward Concept Drift in IoT