Abstract:Many important cloud services require replicating massive data from one datacenter (DC) to multiple DCs. While the performance of pair-wise inter-DC data transfers has been much improved, prior solutions are insufficient to optimize bulk-data multicast, as they fail to explore the capability of servers to store-and-forward data, as well as the rich inter-DC overlay paths that exist in geo-distributed DCs. To take advantage of these opportunities, we present BDS , an application-level multicast overlay network for large-scale inter-DC data replication. At the core of BDS is a fully centralized architecture, allowing a central controller to maintain an up-to-date global view of data delivery status of intermediate servers, in order to fully utilize the available overlay paths. To quickly react to network dynamics and workload churns, BDS speeds up the control algorithm by decoupling it into selection of overlay paths and scheduling of data transfers, each can be optimized efficiently. This enables BDS to update overlay routing decisions in near realtime (e.g., every other second) at the scale of multicasting hundreds of TB data over tens of thousands of overlay paths. A pilot deployment in one of the largest online service providers shows that BDS can achieve 3-5 x speedup over the provider's existing system and several well-known overlay routing baselines.

The Optimization of Replica Distribution in the Unstructured Overlays

Load Balance Optimization with Replication Degree Customization.

Rer: A Replica Efficiency Based Replication Strategy

Optimization of Overlay Topology for Unstructured Peer-to-Peer

An Overlay Topology Based Proactive Replication in Unstructured P2P Systems

An Adaptive Replication Algorithm in Overlay Networking

An Optimized Replica Distribution Method in Cloud Storage System

DHT-assisted Probabilistic Exhaustive Search in Unstructured P2P Networks

An Optimal Segment Replication Strategy in P2P-VoD Systems

Stationary and Adaptive Replication Approach to Data Availability in Structured Peer-to-peer Overlay Networks.

Replication Strategy in Unstructured Peer-to-Peer Systems

Analysis of Optimal Query, Fairness and Replication for DHT-based Mobility Management

Survey of Overlay Optimization Techniques in Unstructured P2P Systems

A dynamic optimal replication strategy in data grid environment

A Two-Hop Solution to Solving

A Distributed Approach to Solving Overlay Mismatching Problem

Distributed Algorithms for a Replication Problem of Popular Network Data

An Optimal Replication Strategy for Data Grid Systems

Achieving Global Optimal Replication in Distributed Networks.

BDS: a Centralized Near-Optimal Overlay Network for Inter-Datacenter Data Replication

Optimal Distributed Broadcasting With Per-Neighbor Queues In Acyclic Overlay Networks With Arbitrary Underlay Capacity Constraints