Big Data Analytics for Network Security and Intelligence
Xingshu CHEN,Xuemei ZENG,Wenxian WANG,Guolin SHAO
DOI: https://doi.org/10.15961/j.jsuese.201600352
2017-01-01
Abstract:With the development of IT and communication technology,the network environment is becoming more and more complicated,and the perimeters of host and network become dynamic and fuzzy due to the application of cloud computing and visualization technology.At the same time,network attacks become more frequent and advanced network threats with evasive and persistent behavior and profit-chasing are also increasing.However,due to the limit of data source and process ability and device deployment relied on physical environment,traditional network security and intelligence techniques are inefficient on the acquisition capability,analytical abilities and utilize capacity of threat intelligence,and the perception and prediction ability of the network security situation is limited,so it cannot solve the current and future network security challenges efficiently.The chances and challenges caused by big data for network security and intelligence analysis are took as this paper clue.first of all,the connotation of big data is reviewed,and the current dilemmas in network security and intelligence analysis are analyzed,and then the relationship between big data with network security and intelligence analysis is explored,and the changes of traditional security analysis brought by big data technologies are parsed.Big data security analysis,a new security method is formed after the big data technologies was applied in cyber security field.The value of big data security analysis embodies in solving practical problems in network security and intelligence analysis through the methods and technologies of big data analysis under sticking to the purpose of security analysis and the character of security data itself.On the one hand,big data processing technology,such as bulk data processing technology,streaming data processing technology,interactive data query technology,can solve the issues of data processing in the high-performance network traffic real-time restore and analysis,massive historical log data analysis and rapid retrieval,massive text data real-time processing.On the other hand,big data technologies are applied in security visual analysis,security event association and network user behavior analysis,a series research branch of big data security analysis are formed,such as big data interactive visual analysis,multi-source event correlation analysis,user entity behavior analysis,network behavior analysis and so on.Big data security analysis technologies have applied in APT attack detection,network anomaly detection,network security situation perception,network threat intelligence analysis,ere.Even though some achievements have been made in big data based network security and intelligence analysis,the current network security situation is still not optimistic.The effective detection method of advanced network threats and attacks is lacking.The detection and prediction result to unknown complex network attacks is undesirable.The measurement system for evaluation methods of network security situation awareness is needed,and large-scale network security situation awareness indicator system for key assets and entire network is incomplete yet,and the evaluation methods are no pertinence.It is difficult to acquire data from new type data sources of threat information and the standards of threat intelligence sharing are needed to be researched further.Large-scale and integrate threat intelligence center and open service platform aren't yet build.Around the above problems,it needs to be studied that the methods of advanced network threat discovery,complex network attack prediction,large-scale network security situational awareness and threat information collection and sharing technology and needs some key technical breakthroughs such as early detection of advanced network threats,concealment of continuous network communication behavior detection,big data analytics based network feature extraction technology,intelligent-based advanced network threat forecast,non-public network intelligence collection,so as to improve the big data supporting capability for network information security and enhance network information security risk perception and disposal capacity.