Abstract:The industry wide shift to multi-core architectures arouses great interests in parallelizing sequential applications. However, it I S very difficult to parallelize fine-grained applications for multicore architectures due to insufficient hardware support of fast communication and synchronization. Fortunately, network applications can be decomposed into pipelined structures that are amenable to streaming based parallel processing. To realize the potential of pipelining on multi-core architectures, it requires reevaluating the basic tradeoffs in parallel processing, including the ones between load balance and data locality and between general lock mechanisms and special lock-free data structures. This paper presents the practice of building a high-performance multi-core based network processing platform in which connection-affinity and lock-free design principles are applied effectively for better data locality and faster core-to-core synchronization and communication.We parallelize a complete Layer 2 to Layer 7 (L2-L7) network processing system on an Intel Core 2 Quad processor, including a TCP/IP stack based on Libnids (L2-L4) and a port-independent protocol identification engine by deep packet inspection (L7+). Furthermore, we develop a compiling method to transform sequential network applications to parallel ones to enable those applications to run on multi-core architectures. Our experience suggests that (1) fine-grained pipelining can be a good software solution for parallelizing network applications on multi-core architectures if connection-affinity and lock-free are used as the first design principles; (2) a delicate partitioning scheme is required to map pipelined structures onto specific multi-core architecture; (3) an automatic parallelization approach can work if domain knowledge is considered in the parallelizing process. Our multi-core based network processing platform can deliver not only 6Gbps processing speed for large packet sizes but also more challenging 2Gbps speed for smaller packets.

THE NETWORK PARALLEL COMPUTING OF SEISMIC DATA USING TCP/IP PROTOCOL

Development and Application of Networked Structural Laboratories System in Current Hierarchical Network Environment

Hybrid Performance Modeling And Analyzing Of Parallel Systems

Application Of Distributed And Parallel Computing In Traffic Network Simulation

A Method for Parallelizing Traffic Network Simulation Based on Space Domain Decomposition

Extraction of Drainage Network from Grid Terrain Datasets Using Parallel Computing

Efficient socket-based data transmission method and implementation in deep learning

Parallel computing for continuous traffic flow model

WIDE AREA MEASUREMENT AND REAL-TIME TRANSMISSION SYSTEM FOR PHASOR DATA UNDER NETWORK COMMUNICATION MODE

Parallel algorithm design and optimization of geodynamic numerical simulation application on the Tianhe new-generation high-performance computer

Establishment and Application of a Network Parallel Computation Environment Based on Windows Network Operating System

Decentralised Seismic Tomography Computing in Cyber-Physical Sensor Systems

The Development of a Parallel Simulation System in Backbone Network

An Equalisation Control Method for Network Big Data Transmission Based on Parallel Computing

Research on Parallel Computing of Spatial Vector Data Conversion Based on Common Interface

Practice Of Parallelizing Network Applications On Multi-Core Architectures

Study of reliable transmission protocol and algorithm in large-scale seismic exploration instrument

Parallel and Distributed Simulation Based on Networks

Distributed parallel modeling for traffic simulation

A High Performance Computing Method for Noise Cross-Correlation Functions of Seismic Data