Characterizing the Modification Space of Signature IDS Rules

Ryan Guide,Eric Pauley,Yohan Beugin,Ryan Sheatsley,Patrick McDaniel

DOI: https://doi.org/10.1109/MILCOM58377.2023.10356225

2024-02-15

Abstract:Signature-based Intrusion Detection Systems (SIDSs) are traditionally used to detect malicious activity in networks. A notable example of such a system is Snort, which compares network traffic against a series of rules that match known exploits. Current SIDS rules are designed to minimize the amount of legitimate traffic flagged incorrectly, reducing the burden on network administrators. However, different use cases than the traditional one--such as researchers studying trends or analyzing modified versions of known exploits--may require SIDSs to be less constrained in their operation. In this paper, we demonstrate that applying modifications to real-world SIDS rules allow for relaxing some constraints and characterizing the performance space of modified rules. We develop an iterative approach for exploring the space of modifications to SIDS rules. By taking the modifications that expand the ROC curve of performance and altering them further, we show how to modify rules in a directed manner. Using traffic collected and identified as benign or malicious from a cloud telescope, we find that the removal of a single component from SIDS rules has the largest impact on the performance space. Effectively modifying SIDS rules to reduce constraints can enable a broader range of detection for various objectives, from increased security to research purposes.

Cryptography and Security

What problem does this paper attempt to address?

### What problem does this paper attempt to solve?

This paper aims to solve the problem that the rules of traditional signature - based intrusion detection systems (SIDS) are too strict. Specifically:

1. **Reduce false positives**: Current SIDS rules are carefully designed to minimize the mislabeling of legitimate traffic as malicious traffic, which reduces the burden on network administrators. However, this strictness limits the flexibility of SIDS in other application scenarios.

2. **Expand the detection range**: Different usage scenarios may require SIDS to have more relaxed operational constraints. For example, researchers may want to study traffic trends or analyze modified versions of known vulnerabilities, which requires SIDS to be able to capture more types of traffic, even if it means increasing some false positives.

3. **Explore the rule modification space**: The paper proposes an iterative method to explore the modification space of SIDS rules. By gradually removing certain conditions in the rules to relax the constraints and evaluating the impact of these modifications on performance. The author hopes to find a balance in this way, which can both expand the detection range and maintain a certain level of accuracy.

4. **Improve detection flexibility**: By making targeted modifications to the rules, SIDS can flexibly switch between different detection goals, from enhancing security to meeting research needs.

### Specific problem description

- **Existing problems**: The existing SIDS rules are too strict in order to minimize false positives, which limits their use in other application scenarios.
- **Solution**: By removing certain conditions in the rules, explore the space of rule modification, and find the rule configuration that can expand the detection range and maintain a certain level of accuracy.
- **Application prospects**: This method not only helps to improve the flexibility of security detection, but also provides researchers with a broader data set for analysis.

### Method overview

The author adopts an iterative method, gradually removing certain conditions in the rules, and evaluating the effect of each modification through the Receiver Operating Characteristic (ROC) curve. The specific steps include:

1. **Define the rule structure**: Formally define the structure of SIDS rules and their removable options.
2. **Calculate the modification space**: Calculate all possible removal combinations and ensure the feasibility of the calculation by limiting the search space.
3. **Iterative exploration**: Gradually build the Pareto frontier by successively removing single options and evaluating their impact on the ROC curve.
4. **Evaluate performance**: Evaluate the performance of the modified rules using a real - traffic data set from the Cloud Telescope and analyze the impact of different removal options.

### Main findings

- **The removal of a single option has the greatest impact**: Removing a single option has the most significant effect on expanding the performance space, increasing the area under the ROC curve by 11.3%.
- **Removal of content options in multi - content rules**: Removing some content options in multi - content rules can significantly improve the detection range while maintaining a low false positive rate.
- **The impact of specific options**: The removal of options such as `http_header` and `content` has a significant impact on the detection performance, but not all removals can bring positive effects.

Through these findings, the author shows how to expand the detection range of SIDS through targeted rule modifications to meet the needs of different application scenarios.