Abstract:Emerging network-attached resource disaggregation architecture requires ultra-low latency rack-scale communication. However, current hardware offloading (e.g., RDMA) and user-space (e.g., mTCP) communication schemes still rely on heavily layered protocol stacks which requires the translation between PCIe bus and network protocol, or complex connection/memory resource management within RNICs, inevitably bringing latency overhead. We argue that PCIe Non-Transparent Bridge (NTB) is a superior high-speed in-rack network technology to interconnect PCIe-attached machines or devices with the same PCIe fabric since no translation is needed between PCIe and network protocol. We present NTSocks, the first user-space in-rack interconnect over PCIe fabric which virtualizes native NTB into high-level network functionalities for rack-scale systems with software-hardware co-design. NTSocks provides (1) compatibility with a fast socket-like abstraction, (2) multi-thread scalability using a core-driven dat-aplane model, and (3) fair and efficient resource sharing with a multi-tenant isolation mechanism. Even though PCIe NTB is originally designed for device communication across PCIe domains, NTSocks shows a flexible user-level indirection with performance close to bare-metal NTB while providing common network stack features. In the evaluations with latency-sensitive Key-Value Store, NTSocks achieves better latency by up to 24.5× and 1.58× than kernel and RDMA socket, respectively.

A Low Latency Implementation Scheme of Serial RapidIO Endpoint

A serial physical layer design in RapidIO

An effective buffer space management in Serial RapidIO endpoint

RFT: Toward Highly Reliable Flow Data Transmission in Network Measurement

Design and FPGA Verification of a Novel Reliable Real-Time Data Transfer System

Research on Physical Layer Traffic Management Schemes in Serial RapidIO Interconnect

Design of RapidIO logical core based on safety arbitration mechanisms

Buffer design based on flow control in RapidIO

Design of High-Speed Data Transmission System for IC Test

Selective Parallel Crc Computation Schemes for Rapidio

Low Latency High Throughout Circular Asynchronous FIFO

Latency Criticality Aware On-Chip Communication

Design and FPGA implementation of high-speed, fixed-latency serial transceivers

A Delay-Optimized Universal FPGA Routing Architecture

An Efficient Scheduling Mechanism With Flow-Based Packet Reordering In A High-Speed Network Processor

Improved CRC Calculation Strategies for 64-Bit Serial RapidIO

A Low-Cost and High-Throughput NoC-Aware Chip-to-Chip Interconnection

Design and Implementation of High-Speed Data Transmission Scheme Between FPGA Boards Based on Virtex ‐7 Series

Improvement of CRC Receiving Controller for Serial RapidIO

A Low-Latency and Low-Power Hybrid Scheme for On-Chip Networks

An Ultra-Low Latency and Compatible PCIe Interconnect for Rack-Scale Communication.