Abstract:Intrusion detection system (IDS) provides an important basis for the network defense. Due to the development of the cloud computing and social network, massive amounts of data are generated, which inevitably brings much pressure to IDS. And therefore, it becomes crucial to efficiently divide the data into different classes over big data according to data features. Moreover, we can further determine whether one is normal behavior or not based on the classes information. Although the clustering approach based on K-means for IDS has been well studied, unfortunately directly using it in big data environment may suffer from inappropriateness. On the one hand, the efficiency of data clustering needs to be improved. On the other hand, differ from the classification, there is no unified evaluation indicator for clustering issue, and thus, it is necessary to study which indicator is more suitable for evaluating the clustering results of IDS. In this paper, we propose a clustering method for IDS based on Mini Batch K-means combined with principal component analysis. First, a preprocessing method is proposed to digitize the strings and then the data set is normalized so as to improve the clustering efficiency. Second, the principal component analysis method is used to reduce the dimension of the processed data set aiming to further improve the clustering efficiency, and then mini batch K-means method is used for data clustering. More specifically, we use K-means++ to initialize the centers of cluster in order to avoid the algorithm getting into the local optimum, in addition, we choose the Calsski Harabasz indicator so that the clustering result is more easily determined. Compared with the other methods, the experimental results and the time complexity analysis show that our proposed method is effective and efficient. Above all, our proposed clustering method can be used for IDS over big data environment.

Research of Intrusion Detection System Based on Clone Clustering

Research of the Intrusion Detection System Based on Fuzzy Clustering

The Application of Cluster Analysis to Intrusion Detection System

Distributed Intrusion Detection Based On Clustering

Multi-agent based hybrid Intrusion detection system

Intrusion Detection Based on Adaptive Polyclonal Clustering

Clustering Approach Based On Mini Batch Kmeans For Intrusion Detection System Over Big Data

An Improved Kernel Clustering Algorithm Used in Computer Network Intrusion Detection.

A Genetic Clustering Method for Intrusion Detection.

Intrusion Detection Based On Unsupervised Clustering And Simulated Annealing (Idcsa)

Intrusion Detection Based on Immune Clonal Selection Algorithms

Intrusion detection system based on the beetle swarm optimization and K‐RMS clustering algorithm

A Survey on the Use of Data Clustering for Intrusion Detection System in Cybersecurity

CLUSTERING BASED ON SWARM INTELLIGENCE WITH APPLICATION TO ANOMALY INTRUSION DETECTION

Intrusion Detection Based on Immune Clustering

A multiple-level hybrid intrusion detection system based on hierarchical clustering and decision trees

A time stamped clustering method for intrusion detection

Toward Fast and Deterministic Clone Detection for Large Anonymous RFID Systems.

Clustering based on Self-Organizing Ant Colony Networks with Application to Intrusion Detection

An Adaptive Clustering Algorithm for Intrusion Detection

CLUSTERING-BASED NETWORK INTRUSION DETECTION