Abstract:The continuous growth of Internet and its applications caused more difficulties for analyzing Internet communications which are becoming more and more complex, this has caused new challenges for monitoring and managing the huge and vast network traffic. It is not efficient to monitor and analyze individual IP addresses, so it is more useful to monitor groups of IP addresses that have similar behavior, which represents a certain application activity. Nowadays, such a grouping is either based on network prefixes that does not meet the requirement mentioned above as difference of traffic behavior of individual IP address not being considered, or clustering IP hosts based on their traffic patterns, which requires information about TCP/UDP port numbers (which are occasionally obfuscated) or packet payloads (which are sometimes encrypted or unavailable from aggregated flow records). This paper proposes a new methodology of clustering IP addresses within a managed network domain such as campus network or ISP clients with similar social relationship based on inter-IP connectivity structure. The key idea of this methodology is to split the entire IP address space into Internal (inside the managed domain) and External (outside) ones. The clustering strategy is to group inside IP addresses that communicate with common outside IP addresses, the similarity measure of two inside IP addresses is the unique number of the common outside IP addresses. We propose a novel approach with an approximation algorithm to discover communities on a large scale in the managed domain based on the bipartite networks and one mode projection and the basis of graph partitioning of the similarity graph. Bipartite networks were built using NetFlow datasets collected from a boundary router in an actual environment, and then a one-mode projection has been applied to build a social relationship similarity graph of the inside IP addresses. We propose a community detection algorithm to extract communities. Experimental results demonstrate that our approach can discover communities from real large scale managed domain networks with a high quality. We experimentally validate our approach in terms of IP networking by applying deep flow inspection (DFI) and deep packet inspection (DPI) on related traffic to prove that hosts with the same cluster tend to have some dominant network behavior. We demonstrated the practical benefits of exploring social behavior similarity of IP hosts in understanding application usage, users' behavior, detecting malicious users, and users of prohibited applications.

IP traffic behavior characterization via semantic mining

IP Backbone Traffic Behavior Characteristic Spectrum Composing and Role Mining

A New Way for User'S Web Communication Visualization and Measurement: Modeling, Experiment and Application

Behavior Pattern Mining from Traffic and Its Application to Network Anomaly Detection

Characterizing and Mining Traffic Patterns of IoT Devices in Edge Networks

Behavior analysis of internet traffic via bipartite graphs and one-mode projections

Cause of symmetry for traffic volume: Mining and exploring new characteristics of traffic in non-business networks

Large-scale IP Network Behavior Anomaly Detection and Identification Using Substructure-Based Approach and Multivariate Time Series Mining

Traffic features extraction and clustering analysis for abnormal behavior detection.

Survey and analysis on the Internet traffic model

Analyze Large-Scale Communication Network Behavior Through Mining TCP Control Segment Information

DeepHTTP: Semantics-Structure Model with Attention for Anomalous HTTP Traffic Detection and Pattern Mining

Traffic Features Measurement Based on Multi-Scale Aggregation Model

Characterizing Home Network Traffic: an Inside View

Social Relationship Discovery of IP Addresses in the Managed IP Networks by Observing Traffic at Network Boundary

Users' Behavior Character Analysis and Classification Approaches in Enterprise Networks

Measuring and classifying IP usage scenarios: a continuous neural trees approach

Behavior Spectrum: an Effective Method for User's Web Access Behavior Monitoring and Measurement

Dynamic Group Behavior Analysis and Its Application in Network Abnormal Behavior Detection.

Profiling IP Hosts Based on Traffic Behavior

Characterizing Application Behaviors for Classifying P2p Traffic