Abstract:Intrusion detection system (IDS) provides an important basis for the network defense. Due to the development of the cloud computing and social network, massive amounts of data are generated, which inevitably brings much pressure to IDS. And therefore, it becomes crucial to efficiently divide the data into different classes over big data according to data features. Moreover, we can further determine whether one is normal behavior or not based on the classes information. Although the clustering approach based on K-means for IDS has been well studied, unfortunately directly using it in big data environment may suffer from inappropriateness. On the one hand, the efficiency of data clustering needs to be improved. On the other hand, differ from the classification, there is no unified evaluation indicator for clustering issue, and thus, it is necessary to study which indicator is more suitable for evaluating the clustering results of IDS. In this paper, we propose a clustering method for IDS based on Mini Batch K-means combined with principal component analysis. First, a preprocessing method is proposed to digitize the strings and then the data set is normalized so as to improve the clustering efficiency. Second, the principal component analysis method is used to reduce the dimension of the processed data set aiming to further improve the clustering efficiency, and then mini batch K-means method is used for data clustering. More specifically, we use K-means++ to initialize the centers of cluster in order to avoid the algorithm getting into the local optimum, in addition, we choose the Calsski Harabasz indicator so that the clustering result is more easily determined. Compared with the other methods, the experimental results and the time complexity analysis show that our proposed method is effective and efficient. Above all, our proposed clustering method can be used for IDS over big data environment.

A Parallel Clustering Ensemble Algorithm for Intrusion Detection System

A Mixed Intrusion Detection System Utilizing K-means and Extreme Gradient Boosting

Cluster ensemble for intrusion detection systems

Research of Intrusion Detection System Based on Clone Clustering

The Application of Cluster Analysis to Intrusion Detection System

A Hybrid Spectral Clustering and Deep Neural Network Ensemble Algorithm for Intrusion Detection in Sensor Networks

Distributed Intrusion Detection Based On Clustering

Intrusion Detection Based on Parallel Intelligent Optimization Feature Extraction and Distributed Fuzzy Clustering in WSNs.

Intrusion Detection Based On Unsupervised Clustering And Simulated Annealing (Idcsa)

An Efficient Network Intrusion Detection System Based on Feature Selection and Ensemble Classifier.

Intrusion Detection Based on Adaptive Polyclonal Clustering

Research on Intrusion Detection Model using ensemble learning methods

CLUSTERING BASED ON SWARM INTELLIGENCE WITH APPLICATION TO ANOMALY INTRUSION DETECTION

Building an Efficient Intrusion Detection System Based on Feature Selection and Ensemble Classifier

Clustering Approach Based On Mini Batch Kmeans For Intrusion Detection System Over Big Data

An Improved Kernel Clustering Algorithm Used in Computer Network Intrusion Detection.

Classification and Clustering Based Ensemble Techniques for Intrusion Detection Systems: A Survey

A Survey on the Use of Data Clustering for Intrusion Detection System in Cybersecurity

An Efficient Parallel Anomaly Detection Algorithm Based on Hierarchical Clustering.

A Combined Feature Selection Method Based on Clustering in Intrusion Detection

An Adaptive Clustering Algorithm for Intrusion Detection