Abstract:Designing a cost-effective network topology for data centers that can deliver sufficient bandwidth and consistent latency performance to a large number of servers has been an important and challenging problem. Many server-centric data center network topologies have been proposed recently due to their significant advantage in cost efficiency and data center agility, such as BCube, FiConn, and Bidimensional Compound Network BCN. However, existing server-centric topologies are either not expandable or demanding prohibitive expansion cost. As the scale of data centers increases rapidly, the lack of expandability in existing server-centric data center networks imposes a severe obstacle for data center upgrade. In this paper, we present a novel server-centric data center network topology called BCube connected crossbars BCCCs, which can provide good network performance using inexpensive commodity off-the-shelf switches and commodity servers with only two network interface card NIC ports. A significant advantage of BCCC is its good expandability. When there is a need for expansion, we can easily add new servers and switches into the existing BCCC with little alteration of the existing structure. Meanwhile, BCCC can accommodate a large number of servers while keeping a very small network diameter. A desirable property of BCCC is that its diameter increases only linearly to the network order i.e., the number of dimensions, which is superior to most of the existing server-centric networks, such as FiConn and BCN, whose diameters increase exponentially with network order. In addition, there are a rich set of parallel paths with similar length between any pair of servers in BCCC, which enables BCCC to not only deliver sufficient bandwidth capacity and predictable latency to end hosts, but also provide graceful performance degradation in case of component failure. We conduct comprehensive comparisons between BCCC with other popular server-centric network topologies, such as FiConn and BCN. We also propose an effective addressing scheme and routing algorithms for BCCC. We show that BCCC has significant advantages over the existing server-centric topologies in many important metrics, such as expandability, server port utilization, and network diameter.

A MapReduce-supported Network Structure for Data Centers

RCFile: A Fast and Space-Efficient Data Placement Structure in MapReduce-based Warehouse Systems

GHDC: a dual-centric data center network architecture by using multi-port servers with greater incremental scalability

Flexible, highly scalable and cost-effective network structures for data centers

BCN: Expansible Network Structures for Data Centers Using Hierarchical Compound Graphs

ABCCC: an Advanced Cube Based Network for Data Centers

Distributed data management using MapReduce

A MapReduce Cluster Deployment Optimization Framework with Geo-distributed Data.

BCDC: A High-Performance, Server-Centric Data Center Network

Expandable and Cost-Effective Network Structures for Data Centers Using Dual-Port Servers.

A Novel Network Structure with Power Efficiency and High Availability for Data Centers.

The performance of MapReduce: an in-depth study

MDCube: a high performance network structure for modular data center interconnection.

The Performance of MapReduce

QoS-Aware Data Placement for MapReduce Applications in Geo-Distributed Data Centers

RCube: A Power Efficient and Highly Available Network for Data Centers.

BCCC: an Expandable Network for Data Centers.

Energy-aware Routing in Data Center Network

Query optimization for massively parallel data processing.

Optimizing the Topologies of Virtual Networks for Cloud-Based Big Data Processing

BCCC: an expandable network for data centers