Abstract:Emerging interconnects, such as CXL and NVLink, have been integrated into the intra-host topology to scale more accelerators and facilitate efficient communication between them, such as GPUs. To keep pace with the accelerator's growing computing throughput, the interconnect has seen substantial enhancement in link bandwidth, e.g., 256GBps for CXL 3.0 links, which surpasses Ethernet and InfiniBand network links by an order of magnitude or more. Consequently, when data-intensive jobs, such as LLM training, scale across multiple hosts beyond the reach limit of the interconnect, the performance is significantly hindered by the limiting bandwidth of the network infrastructure. We address the problem by proposing DFabric, a two-tier interconnect architecture. We address the problem by proposing DFabric, a two-tier interconnect architecture. First, DFabric disaggregates rack's computing units with an interconnect fabric, i.e., CXL fabric, which scales at rack-level, so that they can enjoy intra-rack efficient interconnecting. Second, DFabric disaggregates NICs from hosts, and consolidates them to form a NIC pool with CXL fabric. By providing sufficient aggregated capacity comparable to interconnect bandwidth, the NIC pool bridges efficient communication across racks or beyond the reach limit of interconnect fabric. However, the local memory accessing becomes the bottleneck when enabling each host to utilize the NIC pool efficiently. To the end, DFabric builds a memory pool with sufficient bandwidth by disaggregating host local memory and adding more memory devices. We have implemented a prototype of DFabric that can run applications transparently. We validated its performance gain by running various microbenchmarks and compute-intensive applications such as DNN and graph.

FLAX: A Flexible Architecture for Large Scale Cloud Fabric

Cloud-Fog Automation: Heterogenous Applications over New Generation Infrastructure of Virtualized Computing and Converged Networks

RIFFLE: A Distributed Network Operating System with Re-configurability for Multiple Domains

DFabric: Scaling Out Data Parallel Applications with CXL-Ethernet Hybrid Interconnects

Scalable Data Center Network with distributed placement of component sets in optical networks

Sdn Based Vxlan Optimization In Cloud Computing Networks

A reconfigurable fabric for accelerating large-scale datacenter services

A Practical Cross-Datacenter Fault-Tolerance Algorithm in the Cloud Storage System.

FlexGate: High-performance Heterogeneous Gateway in Data Centers

FlexWAN: Software Hardware Co-design for Cost-Effective and Resilient Optical Backbones.

Dependable Virtualized Fabric on Programmable Data Plane

Building the Next-Generation Data Centers Infrastructure Cloud

A 12-Rack, 180-Server Datacenter Network (Dcn) Using Multiwavelength Optical Switching And Full Stack Optimization

Predictable Vfabric on Informative Data Plane

Flex: High-Availability Datacenters With Zero Reserved Power

FlexNF: Flexible Network Function Orchestration for Scalable On-Path Service Chain Serving

Flor: An Open High Performance RDMA Framework Over Heterogeneous RNICs.

SRFabric: A Semi-Reconfigurable Rack Scale Topology

FlexCross: High-Speed and Flexible Packet Processing via a Crosspoint-Queued Crossbar

Halflife: An Adaptive Flowlet-based Load Balancer with Fading Timeout in Data Center Networks.

FiConn: Using Backup Port for Server Interconnection in Data Centers