An Improved Multi-Pattern Matching Algorithm and Its Application on NIDS

XU Ning,LAI Hai-guang,HUANG Hao

DOI: https://doi.org/10.3969/j.issn.1001-3695.2006.02.019

2006-01-01

Abstract:An improved algorithm based on the ideas of Boyer-Moore method to search for multiple patterns is presented.The application of this algorithm in a network intrusion detection system is also discussed.The experiments demonstrate good improvement in matching time.

What problem does this paper attempt to address?

Multi-Gigabit Rate Pattern-Matching Engine Using Grouping CAM/TCAM

Yi-fan HE,Guo-yin XU,Hai-bin SHEN

DOI: https://doi.org/10.3969/j.issn.1005-9490.2007.01.042

2007-01-01

Abstract:In order to improve the performance of pattern-matching module in NIDS, an efficient engine for multi-gigabit rate, line-speed pattern-matching is proposed after analyzing a large amount of related work. By grouping and evenly storing patterns into CAM or TCAM according to their lengths and amount, as well as using payload-switching technique, this engine not only reaches multi-gigabit rate performance, but also has high storage efficiency. System performance can be improved further by using multiple process modules in parallel. In a 200 MHz Clock implementation, system throughput can reach more than 6 Gbit/s.
NIDS: Neural Network Based Intrusion Detection System

Hassan Hadi Al-Maksousy,Michele C. Weigle,Cong Wang

DOI: https://doi.org/10.1109/ths.2018.8574174

2018-01-01

Abstract:Malware detection is an important problem. We propose an efficient real-time system to detect and classify malware based on network behavior using deep neural networks. We show that the splitting of the system into two neural networks, detection and analysis, is the key to increasing accuracy. Therefore, this approach allows for the construction of a real-time monitoring system with very low CPU usage. Finally, we provide a comparison analysis of four machine learning classifiers for this task.
Research on high-performance pattern matching algorithm

WANG Zhi-wei,PING Ling-di,LU Min-feng

DOI: https://doi.org/10.3778/j.issn.1002-8331.2010.01.009

2010-01-01

Abstract:On the basis of the BM algorithm and its derivative versions:BMH,Sunday and other algorithms,a new improved algorithm is presented.The improved algorithm has three important features:(1)using two-character inspired strategy to improve the pattern moving length and its probability.The largest length is n+2;(2)using the method of dynamic segmentation to minimize the matching;(3)building the chain with the location for the same character in the pattern to take full advantage of inspiring characters and reduce the redundancy of matching.Experimental results show that the improved algorithm has higher matching efficiency.
Multi-agent based hybrid Intrusion detection system

张宝军,潘雪增,王界兵,平玲娣

DOI: https://doi.org/10.3785/j.issn.1008-973X.2009.06.004

2009-01-01

Abstract:Real-time intrusion detection under current network environment exists the following problems: first, the scale of network is large, and a great deal of information needs to be processed, which requires large throughput to the intrusion detection system (IDS); second, the network environment is complex, and the data type is multiplex, accordantly, the intrusion detection system should has high accuracy. Aiming at these problems, a model of intrusion detection system was proposed. The model uses the distributed architecture based on multi-agent system and shows good expansibility, and can self-adjust according to the scale and bandwidth of network. The model uses technologies of anomaly intrusion detection and misuse intrusion detection together, and has low false alert rate and miss alert rate. Multi-attribute data abstraction is used in the model to grasp the feature of intrusion accurately and provide strong support for intrusion identification. The classifier is constructed with radial basis function (RBF) so as to have good extension to unknown intrusions, and can do effective judgment to unknown intrusions. Experimental results show that the system has large throughput and high accuracy, thus it is suitable for current network and has good practicability.
Improved Pattern Matching Algorithm Based on Partition

HUANG Yong,PING Ling-di,PAN Xue-zeng,CHEN Jian

DOI: https://doi.org/10.16411/j.cnki.issn1006-7736.2008.01.024

2008-01-01

Abstract:ISKIP and IKMPSKIP algorithms were developed combining quick search(QS) algorithm to improve performance of SKIP and KMPSKIP algorithms using partition window.The shift information of the last character of next window was looked ahead to increase the shift distance,and the times of matching were reduced efficiently.Theoretical analysis shows that two improved algorithms are superior to the original ones in respect of average time complexities.Experimental results also demonstrate its effectiveness.Especially in case of small patterns,ISKIP algorithm takes only 65%～85% of time of BMH algorithm.
Intrusion Detection And Simulation For High-Speed Networks

Fei Yu,XiaoPing Dai,Yue Shen,Huang Huang,Miaoliang Zhu

DOI: https://doi.org/10.1109/ICSSSM.2005.1500110

2005-01-01

Abstract:The problem in network security presently is how to get real-time intrusion detection and alert. In the paper, we design Intrusion Detection System architecture and arithmetic again, meanwhile put forward an improved pattern matching arithmetic based on protocol analysis and theorization machine on Frete. New architecture can use network processor to collect and analyze data in network bottom, which enhance the speed and efficiency in Detection System.
A robust approach for matching mixed case-sensitive and case-insensitive patterns

Hongbin Lu,Kai Zheng,Bin Liu,Changhua Sun

DOI: https://doi.org/10.1109/ICNS.2007.16

2007-01-01

Abstract:As one of the key methods as well as a bottleneck for Network Intrusion Detection Systems (NIDSes) to detect and eliminate malicious traffic, pattern matching is increasingly gaining popularity while also faces threats from hackers' overloading attempts. The support of mixed case-sensitive and case-insensitive patterns, which is essential for NIDSes to detect possible attacks targeting different applications and operating systems, is currently a potential vulnerability since the widely used Convert-Search-Verify (CSV) approach encounters severe performance degradation in the worst-case scenarios. This paper firstly gives a thorough analysis on the reasons causing jams in the worst case, and then boosts up the performance by leveraging a novel mechanism named Convert-Search-incrementally-Verify (CSiV). CSiV differs from CSV in that it first merges possible case-sensitive matches to suspicious segments in the "Search" phase, and then leverages an Aho-Corasick like algorithm to verify them. The infeasibility of the simple Double Search (DS) approach is also explained by analyzing its low average-case throughput. Extensive experiments based on real pattern sets along with both collected and artificial traffic traces show that, the performance of the proposed approach outperforms the DS approach by a factor of 2 in the ordinary cases, and is better than the CSV approach up to 5 times under the worst-case scenario, indicating both its feasibility and robustness for a worst-case safe NIDS. © 2007 IEEE.
ON WU_MAMBER MULTIPLE PATTERN MATCHING ALGORITHM IN INTRUSION DETECTION SYSTEM

Wu Xihong

2008-01-01

Abstract:On the basis of analyzing and studying existing Wu_Manber algorithm in intrusion detection system, an improved Wu_Manber algorithm is designed. This algorithm combines the notion of QS algorithm and expands its shift distance to maximum in searching stage. At the same time, some experiments were made to compare the performances of the traditional and improved Wu_Manber algorithms. The experimental results shown that the improved Wu_Manber algorithm performs better in both English and Chinese text, and can detect intrusion faster and more accurately.
Research of an improved multiple differential feedback detection algorithm over fast rayleigh fading channel.

JIN Ning,JIN Xiao-ping,LI Xi-hua

DOI: https://doi.org/10.3785/j.issn.1008-9497.2009.03.011

2009-01-01

Abstract:It is known that multiple symbol differential detection(MSDD) has very high complexity,which is exponential with block length of N This paper presented an improved multiple differential feedback detection(IMDFD) algorithm,which not only has low computation complexity,but also can solve the problems of error propagation of MDFD algorithm and performance degradation at low signal-noise ratio.Simulation results showed that the algorithm is effective on filling the gap between differential detection and coherent detection with extending the observation interval.And with the length of signal sequence decrement,the performance of system is boosting up gradually with a bit increase of computation complexity.
Error-tolerant multi-pattern matching algorithms for intrusion detection

宋华,戴一奇

DOI: https://doi.org/10.3321/j.issn:1000-0054.2003.07.032

2003-01-01

Abstract:Many problems in intrusion detection can be solved by error-tolerant multi-pattern matching algorithms. Three algorithms were designed for intrusion detection. The first algorithm was a single pattern matching algorithm while the other two were multi-pattern matching algorithms. All three algorithms implemented as dynamic programming algorithms detected the occurrences of any pattern from a given pattern set in a text, allowing a limited number of errors in the attack event sequences. The algorithms exploited the ability to arrange the pattern set in a searching tree and detected multi intrusion patterns simultaneously. Analytical and experimental results showed that all three error-tolerant multi-pattern matching algorithms were fast when searching large sets of patterns.
Towards High-Performance Pattern Matching on Multi-Core Network Processing Platforms

Yaxuan Qi,Zongwei Zhou,Yiyao Wu,Yibo Xue,Jun Li

DOI: https://doi.org/10.1109/GLOCOM.2010.5684120

2010-01-01

Abstract:With the continual growth of network speed and the increasing sophistication of network applications, keeping network operations efficient and secure becomes more challenging. Pattern matching is one of the key technologies for content-ware network processing, such as traffic classification, application identification and intrusion prevention. In this paper, we propose a hybrid pattern matching algorithm optimized for multi-core network processing platforms. As a system-level solution, our scheme focuses on both performance stability and hardware/software co-design. To verify the effectiveness of our design, the proposed algorithm is implemented on a state-of-art 16-MIPS-core network processing platform and evaluated with real-life data sets. Experimental results show that, when compared with the traditional Aho-Corasick algorithm, our hybrid solution saves 60~95% memory space while guarantees stable performance on large pattern sets and against adverse test traffic.
Research of Single Pattern Matching Algorithm Based on Suffix Searching

WU Xi-hong

2012-01-01

Abstract:The network which is developed rapidly puts many people to convenience,at the same time the network safe problem is accompanied.The way to solve the problem is to enhance the intrusion detection technology.The pattern matching algorithms directly influence the high efficiency and accuracy performance of the system.It analyses in detail the characteristic of three single pattern matching algorithms based on suffix searching,then some experiments are done from three aspects which are matched time,trial times and characters' numbers through the number of different pattern strings.The experimental results show that QS algorithm and RF algorithm can shorten scanning time widely because they can skip longer string chars.So improve the speed of the pattern matching and better use in detection system.
Implementation of IDS Based on WinPcap and Boyer-Moore

Dan Cogălniceanu,Jijun Li

2004-01-01

Abstract:In this article,a network intrusion detection system based on WinPcap and Boyer-Moore in Windows2000 /XP is put forward and implemented.Its architecture,the capture to the packets with WinPcap and Boyer-Moore string matching algorithm are introduced and analyzed in detail.
Memory Efficient String Matching Algorithm for Network Intrusion Management System

Yu Jianming,Xue Yibo,Li Jun

DOI: https://doi.org/10.1016/s1007-0214(07)70137-2

2007-01-01

Tsinghua Science & Technology

Abstract:As the core algorithm and the most time consuming part of almost every modern network intrusion management system (NIMS), string matching is essential for the inspection of network flows at the line speed. This paper presents a memory and time efficient string matching algorithm specifically designed for NIMS on commodity processors. Modifications of the Aho-Corasick (AC) algorithm based on the distribution characteristics of NIMS patterns drastically reduce the memory usage without sacrificing speed in software implementations. In tests on the Snort pattern set and traces that represent typical NIMS workloads, the Snort performance was enhanced 1.48%-20% compared to other well-known alternatives with an automaton size reduction of 4.86-6.11 compared to the standard AC implementation. The results show that special characteristics of the NIMS can be used into a very effective method to optimize the algorithm design.
Recursive Shift Indexing : A Fast Multi-Pattern String Matching Algorithm

Bo Xu,Xin Zhou,Jun Li

2006-01-01

Abstract:String matching algorithms are essential for network devices that filter packets and flows based on their payload. Applications like intrusion detection/prevention, web filtering, anti-virus, and anti-spam all raise the demand for efficient algorithms dealing with string matching. This paper presents a novel multi-pattern string matching algorithm which reduces character comparisons based on recursive shift indexing. Theoretical analysis and experimental results show that the new algorithm is highly efficient: Its search time is cut down significantly compared with other popular existing algorithms, whereas its memory occupation stays at a low level. It is also demonstrated that the proposed algorithm has a simpler structure for easy implementation.1
High Camouflage Intrusion Detection Method for Structured Database Based on Multi Pattern Matching

Dawei Song,Xun Zhu,Fengjuan Ma

DOI: https://doi.org/10.1088/1742-6596/2066/1/012029

2021-11-01

Journal of Physics: Conference Series

Abstract:Abstract with the rise and rapid development of mobile communication, intelligent terminal and data system, we are entering the era of mobile Internet. In recent years, more and more data need to be processed and transmitted in daily life, and structured data is becoming more and more important. Among them, multi-mode matching technology can search data in a wider range. Matching for multiple patterns at a time avoids unnecessary matching, accelerates the matching process, and helps to find longer matching information and obtain higher accuracy. This paper mainly introduces the high camouflage intrusion detection method of structured database based on multi-mode matching. This paper uses the high disguised intrusion detection method of structured database based on multi-mode matching, collects sensitive information of wireless access points and stations through the communication of WLAN in multimodal matching, then intercepts and forges data packets to initiate replay attack. Replay attack is characterized by abnormal traffic in the network, which can be detected by statistical analysis. The experimental results show that the high camouflage intrusion detection method based on multi-mode matching makes the camouflage intrusion detection rate increase by 23%. The limitations of the design and research of camouflage intrusion detection are analyzed, discussed and summarized, so as to enrich the academic research results.
Efficient long signature matching for gigabit intrusion detection sensors

Zhang Kenong,Gao Ming,Lu Jiahua,Guan Xiaohong

DOI: https://doi.org/10.1109/icnsc.2006.1673277

2006-01-01

Abstract:Network Intrusion Detection Systems (NIDS) require the sensors to inspect the packet payloads at line rates. However, the software-only NIDS can not handle the large signature set with thousands of patterns of different lengths at line rates. Ternary content-addressable memories (TCAMs) have gained wide acceptance in the industry for storing and searching patterns in routers. But one important problem blocked the way to deploy TCAMs as deep package matching engines for NIDS: long patterns matching. A novel high speed long patterns matching architecture using cascade TCAMs for large signature set based NIDS is presented in this paper. Simple and efficient systems to handle tens of thousands of signatures with thousands of bytes length each can be built on such architecture. The matching system using for current SNORT signature set can work at the speeds greater than 2 Gbps. © 2006 IEEE.
Another CDFA Based Multi-Pattern Matching Algorithm and Architecture for Packet Inspection.

Tian Song,Dongsheng Wang

DOI: https://doi.org/10.1109/icccn.2011.6005927

2011-01-01

Abstract:Multi-pattern matching algorithm and architecture is critical for packet inspection based network security applications, especially for high speed network or large pattern sets. This paper presents a method to optimize the potential memory usage of DFA based algorithms for multi-pattern expression matching by the combining DFA's paths, named isomorphic path combination (IMPC). To achieve IMPC, a novel multi-pattern matching algorithm, called ACS, is proposed, which is based on CDFA. Compared to the algorithms on DFA, our method can reduce 78.6% states for Snort pattern set, which results in one of the most memory efficient methods. The most important is that our method is a kind of optimization and can be embedded to other algorithms as the second step for better results. Finally the architecture based on ACS is proposed and the experimental results show that 47.6% to 84.0% memory space can be saved for different size of pattern sets as compared to the best known architectures. The method is another one based on CDFA. It means that CDFA may be a more proper model for multi-pattern matching than other FAs.
Improving the performance of NIDS using symmetric multi-processor

Hai-guang LAI,Hao HUANG,Jun-yuan XIE

2005-01-01

Journal of Computer Applications

Abstract:Network-based intrusion detection system (NIDS) detects attacks by capturing and analyzing network packets. As network band increases, NIDS can hardly keep up with the speed of networks. A method of improving NIDS' process ability using symmetric multi-processor (SMP) was proposed in the paper. Several CPUs of the system were used to process network packets in parallel to improve the performance. After analyzing NIDS' process procedure, an effective parallel processing structure was devised, which guaranteed threads on different CPUs running in parallel. Moreover, the synchronization method of threads proposed avoided the mutually exclusive access to the shared resource, which further increased the parallelity of threads, and guaranteed the correctness of the functionality of the program. Experiments show that the NIDS implemented on a SMP system with dual CPUs is almost 80% faster than the one based on a system with unique CPU.
Optimizing multi-thread string matching for network processor based intrusion management system

Jianming Yu,Quan Huang,Yibo Xue

2006-01-01

Abstract:String matching is the core algorithm and the most time consuming operation of almost every modern Network Intrusion Management System (NIMS). In this paper we aim at integrating string matching with multi-thread parallelism to dramatically improve the performance of NIMS. The string matching procedure under multi-thread parallelism situation is modeled and researched. The results are utilized to instruct the design of an improved Aho-Corasick (AC) algorithm, named as AC_MT, for network processor (NP) based NIMS. A simplified NIMS prototype and both the AC and AC_MT algorithms are implemented on Intel's NP platform IXDP2850. The evaluation results tested with SmartBits 600 reveals that the performance of the NIMS prototype is improved by 44.7%~148.8% depending on the different lengths of the input packets and different number of threads, under both algorithms using the same number of threads situation.

An Improved Multi-Pattern Matching Algorithm and Its Application on NIDS

Multi-Gigabit Rate Pattern-Matching Engine Using Grouping CAM/TCAM

NIDS: Neural Network Based Intrusion Detection System

Research on high-performance pattern matching algorithm

Multi-agent based hybrid Intrusion detection system

Improved Pattern Matching Algorithm Based on Partition

Intrusion Detection And Simulation For High-Speed Networks

A robust approach for matching mixed case-sensitive and case-insensitive patterns

ON WU_MAMBER MULTIPLE PATTERN MATCHING ALGORITHM IN INTRUSION DETECTION SYSTEM

Research of an improved multiple differential feedback detection algorithm over fast rayleigh fading channel.

Error-tolerant multi-pattern matching algorithms for intrusion detection

Towards High-Performance Pattern Matching on Multi-Core Network Processing Platforms

Research of Single Pattern Matching Algorithm Based on Suffix Searching

Implementation of IDS Based on WinPcap and Boyer-Moore

Memory Efficient String Matching Algorithm for Network Intrusion Management System

Recursive Shift Indexing : A Fast Multi-Pattern String Matching Algorithm

High Camouflage Intrusion Detection Method for Structured Database Based on Multi Pattern Matching

Efficient long signature matching for gigabit intrusion detection sensors

Another CDFA Based Multi-Pattern Matching Algorithm and Architecture for Packet Inspection.

Improving the performance of NIDS using symmetric multi-processor

Optimizing multi-thread string matching for network processor based intrusion management system