Abstract:Regular expressions (regexes) provide rich expressiveness to specify the signatures of intrusions and are widely used in contemporary network security systems for signature-based intrusion detection. To perform very fast regex matching, deterministic finite automata (DFA) has been the first choice because its time complexity is constant O ( 1 ) . Unfortunately, DFA often suffers the well known state explosion problem and, consequently, tends to require prohibitive memory overhead in practical applications. To address the problem, a wide variety of DFA compression techniques have been proposed; however, few can keep up with the ever increasing network traffic bandwidth and regex set complexity. This paper proposes that the DFA problem is rooted in regexes (rather than in DFA), i.e., semantic overlapping of regexes, and accordingly presents a complete algorithmic solution PaCC (Partition, Compression, and Combination), that can transform the given large-scale set of complex regexes into a compact and fast matching engine using DFA as its core. PaCC fundamentally defuses state explosion for DFA by partitioning complex regexes into overlapping-free segments. By exploiting the massive repetitiveness among the resulting segments, PaCC can further deflate corresponding DFA in terms of the number of states. Moreover, on the basis of the characteristics of these segments, PaCC takes a tailor-made compression approach and reduces over 96% of the state transitions for the corresponding DFA. In the final matching engine, the combination of DFA and a small relation mapping table, built from segments and their syntagmatic relations, respectively, guarantees high performance and semantic equivalence. Experimental evaluation shows that PaCC produces succinct matching engines with memory usage proportional to the size of the real-world Snort and Bro regex sets, with speeds of up to 1.7Gbps per core on a HP Z220 SFF workstation with a 3.40GHz Intel Core i7-3770.

ReScue: Crafting Regular Expression DoS Attacks

Regular Expression Attack Technique On ReDoS Vulnerability

Runtime Recovery of Web Applications under Zero-Day ReDoS Attacks

SoK: A Literature and Engineering Review of Regular Expression Denial of Service

P4rex: Accelerating Regular Expression Matching with Programmable Switches

Regexes are Hard: Decision-making, Difficulties, and Risks in Programming Regular Expressions

Revery

TransRegex: Multi-modal Regular Expression Synthesis by Generate-and-Repair

Revery: From Proof-Of-Concept To Exploitable (One Step Towards Automatic Exploit Generation)

Practical Regular Expression Matching Free of Scalability and Performance Barriers

gMeta: Template-based Regular Expression Generation over Noisy Examples

Hes: Highly Efficient And Scalable Technique For Matching Regex Patterns

Why Aren't Regular Expressions a Lingua Franca? An Empirical Study on the Re-use and Portability of Regular Expressions

InfeRE: Step-by-Step Regex Generation via Chain of Inference

Uncover the Premeditated Attacks: Detecting Exploitable Reentrancy Vulnerabilities by Identifying Attacker Contracts

RE#: High Performance Derivative-Based Regex Matching with Intersection, Complement and Lookarounds

Efficient Matching with Memoization for Regexes with Look-around and Atomic Grouping (Extended Version)

Automatic generation of regular expressions for the Regex Golf challenge using a local search algorithm

Where URLs Become Weapons: Automated Discovery of SSRF Vulnerabilities in Web Applications

SmartReco: Detecting Read-Only Reentrancy via Fine-Grained Cross-DApp Analysis

REEF: A Framework for Collecting Real-World Vulnerabilities and Fixes