Abstract:Modern cloud computing infrastructures are steadily pushing the performance of their network stacks. At the hardware-level, already some cloud providers have upgraded parts of their network to 10GbE. At the same time there is a continuous effort within the cloud community to improve the network performance inside the virtualization layers. The low-latency/high-throughput properties of those network interfaces are not only opening the cloud for HPC applications, they will also be well received by traditional large scale web applications or data processing frameworks. However, as commodity networks get faster the burden on the end hosts increases. Inefficient memory copying in socket-based networking takes up a significant fraction of the end-to-end latency and also creates serious CPU load on the host machine. Years ago, the supercomputing community has developed RDMA network stacks like Infiniband that offer both low end-to-end latency as well as a low CPU footprint. While adopting RDMA to the commodity cloud environment is difficult (costly, requires special hardware) we argue in this paper that most of the benefits of RDMA can in fact be provided in software. To demonstrate our findings we have implemented and evaluated a prototype of a software-based RDMA stack. Our results, when compared to a socket/TCP approach (with TCP receive copy offload) show significant reduction in end-to-end latencies for messages greater than modest 64kB and reduction of CPU load (w/o TCP receive copy offload) for better efficiency while saturating the 10Gbit/s link.

A Survey of RDMA Distributed Storage

A Survey of Storage Systems in the RDMA Era

RDMAvisor: Toward Deploying Scalable and Simple RDMA as a Service in Datacenters

Design and Optimization of a Distributed File System Based on RDMA

RDMA Load Balancing via Data Partition

POSTER: CAVER: Enhancing RDMA Load Balancing by Hunting Less-Congested Paths

A Behind-the-Scenes Story on Applying Cross-Layer Coordination to Disks and RAIDs

A Comprehensive Evaluation of RDMA-enabled Concurrency Control Protocols.

Toward Effective and Fair RDMA Resource Sharing.

X-RDMA: Effective RDMA Middleware in Large-scale Production Environments.

MC-RDMA: Improving Replication Performance of RDMA-based Distributed Systems with Reliable Multicast Support

Thinking More about RDMA Memory Semantics

Research on the Consensus of Big Data Systems Based on RDMA and NVM

When Cloud Storage Meets RDMA.

Srdma: A General and Low-Overhead Scheduler for RDMA.

Efficient RDMA Communication Protocols

The case for distributed shared-memory databases with RDMA-enabled memory disaggregation

RDMA is Turing complete, we just did not know it yet!

Scalable and Reliable RDMA.

A case for RDMA in clouds: turning supercomputer networking into commodity

Maximizing the Benefit of RDMA at End Hosts