Abstract:Protocol reverse engineering, the process of (re)constructing the protocol context of communication sessions by an implementation, which involves translating a sequence of packets into protocol messages, grouping them into sessions, and modeling state transitions in the protocol state machine, is well-known to be invaluable for many network security applications, including intrusion prevention and detection, traffic normalization, and penetration testing, etc. However, current practice in deriving protocol specifications is either mostly manual or focusing on automatic reverse engineering the message format only and leaving the protocol state machine inverse undone. Although regular expressions offer superior expressive ability and flexibility, application protocols are described by regular expression manually based on sufficiently understanding protocol itself. At present there is not an effect method to realize classification, recognition and control automatically for the known applications and the unknown applications in future. In this paper a novel approach is presented to model network application specification. In this work, the whole automatic protocol reverse engineering is realized through accomplishing the protocol state machine, and then the FSMs are translated to corresponding regular expressions to enrich and update the pattern database. This approach uses grammatical inference and is motivated by the observation that an implementation of the protocol is inherently a state transition process, the state machine model the essence exactly. The important significance is to describe various state protocols with a common method through modeling the protocol state transition, including known and unknown ones. This approach had been implemented in the system and evaluated using real-world implementations of three different protocols: HTTP, SMTP, FTP, and compared the extracted protocol to the corresponding other newly system, such as l7-filter.

Automatic Network Protocol Automaton Extraction

Automatic Protocol Reverse Engineering for Industrial Control Systems with Dynamic Taint Analysis

ABInfer: A Novel Field Boundaries Inference Approach for Protocol Reverse Engineering

Position-based Automatic Reverse Engineering of Network Protocols

Reverse extraction of protocol model from network applications

EDSM-Based Binary Protocol State Machine Reversing

Protocol Specification Inference Based on Keywords Identification

Inferring Protocol State Machine From Real-World Trace

Inferring Protocol State Machine for Binary Communication Protocol

Extracting Protocol Format as State Machine via Controlled Static Loop Analysis

Automatic Protocol Feature Word Construction Based on Machine Learning

Automatic State Machine Inference for Binary Protocol Reverse Engineering

Towards Automatically Reverse Engineering Vehicle Diagnostic Protocols

Recovering Models of Network Protocol Using Grammatical Inference

Mining Protocol State Machines by Interactive Grammar Inference

State Reverse Method for Unknown Binary Protocol Based on State-Related Fields

Network Communication Protocol Reverse Engineering Based on Auto-Encoder

A Network Protocol Reverse Engineering Method Based On Dynamic Taint Propagation Similarity

Protocol Reverse Engineering Using LDA and Association Analysis.

A Progressive Learning Method on Unknown Protocol Behaviors.

Inferring Protocol State Machine from Network Traces: a Probabilistic Approach