Abstract:The industry wide shift to multi-core architectures arouses great interests in parallelizing sequential applications. However, it I S very difficult to parallelize fine-grained applications for multicore architectures due to insufficient hardware support of fast communication and synchronization. Fortunately, network applications can be decomposed into pipelined structures that are amenable to streaming based parallel processing. To realize the potential of pipelining on multi-core architectures, it requires reevaluating the basic tradeoffs in parallel processing, including the ones between load balance and data locality and between general lock mechanisms and special lock-free data structures. This paper presents the practice of building a high-performance multi-core based network processing platform in which connection-affinity and lock-free design principles are applied effectively for better data locality and faster core-to-core synchronization and communication.We parallelize a complete Layer 2 to Layer 7 (L2-L7) network processing system on an Intel Core 2 Quad processor, including a TCP/IP stack based on Libnids (L2-L4) and a port-independent protocol identification engine by deep packet inspection (L7+). Furthermore, we develop a compiling method to transform sequential network applications to parallel ones to enable those applications to run on multi-core architectures. Our experience suggests that (1) fine-grained pipelining can be a good software solution for parallelizing network applications on multi-core architectures if connection-affinity and lock-free are used as the first design principles; (2) a delicate partitioning scheme is required to map pipelined structures onto specific multi-core architecture; (3) an automatic parallelization approach can work if domain knowledge is considered in the parallelizing process. Our multi-core based network processing platform can deliver not only 6Gbps processing speed for large packet sizes but also more challenging 2Gbps speed for smaller packets.

Jparss: A Java Parallel Network Package For Grid Computing

A Public Grid Computing Framework Based on a Hierarchical Combination of Middleware

An Implementation of High Performance Data Transfer Client for Grid Computing

Portal Access to Scientific Computation and Customized Collaborative Visualization on Campus Grid

Grid Service-Based Parallel Finite Element Analysis

Gsrp: An Application-Level Protocol For Grid Environments

Intelligent File Transfer Protocol for Grid Environment

Corep2p: A Tailored Group Communication Scheme for P2p-Grid Gis

A web-based system for adaptive data transfer in grid

JXTPIA: a JXTA-based P2P network interface and architecture for grid computing

A Novel Parallel Transmission Strategy for Data Grid.

Applying Computational Grid Technology to Power System

A Grid Computing Based Approach For The Power System Dynamic Security Assessment

Gridfs: A Web-Based Data Grid With P2p Concepts And Writable Replicas

A Collaborative Application Framework Based on the P2P Grid Model

GridPPI: A Lightweight Grid-Enabled Parallel Programming Framework

Grid Computing Prototype Based on Internet

Block-Based Concurrent and Storage-Aware Data Streaming for Grid Applications with Lots of Small Files

Effective Resource Allocation in A Jxta-Based Grid Computing Platform Jxtpia

Practice Of Parallelizing Network Applications On Multi-Core Architectures

An Integrated Resource Management and Scheduling System for Grid Data Streaming Applications