FNotify: A Low-Latency and Scalable Publish/Subscribe System using RDMA.

Yusen Xu,Sijie Shen,Mingcong Han,Rong Chen
DOI: https://doi.org/10.1109/HPCC-DSS-SmartCity-DependSys57074.2022.00074
2022-01-01
Abstract:Publish/subscribe systems are widely used as messaging middleware in various scenarios, such as online business, financial trading, and social media. However, the network overhead intro-duced by the pub/sub system can severely impact application performance. Especially in the latency-sensitive applications of the data center, optimizing the latency of publishing or delivering messages of the pub/sub system is particularly important. RDMA network has gained popularity in relevant applications due to its low latency, high bandwidth and ability to bypass the CPU. Never-theless, it is still challenging to design and optimize the pub/sub system using RDMA. Through an in-depth analysis of RDMA features, we design and implement FNotify, a high-performance RDMA-accelerated pub/sub system. FNotify optimizes latency and CPU load using in-memory message management and one-sided RDMA primitives for message collection. FNotify also proposes a topic-based partitioning scheme to achieve good scal-ability. We conducted a comprehensive evaluation and analysis on an 8-node RDMA cluster to verify the validity of our design by comparing the performance of FNotify with Apache Kafka (a state-of-the-art pub/sub system) in a typical pub/sub application scenario and various micro-benchmarks. Experimental results show that FNotify improves overall performance by <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$1.5\times$</tex> to <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$3.9\times$</tex> and reduces end-to-end latency by more than <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$20\times$</tex> compared to Apache Kafka.
What problem does this paper attempt to address?