Towards a Scalable, Adaptive and Network-Aware Content Distribution Network
Yan Chen,Randy H. Katz
2003-01-01
Abstract:The Internet has evolved to become a critical commercial infrastructure for service delivery. However, the Internet being an enormous, highly-dynamic, heterogeneous, and untrusted environment raises several challenges for building Internet-scale services (such as content delivery) with good scalability, efficiency, agility and security. In this thesis, we explore these issues by developing a scalable, adaptive and network-aware infrastructure for efficient content delivery, namely Scalable Content Access Network (SCAN). SCAN has four components: object location, replica placement and update multicast tree construction, replica management, and overlay network monitoring services. First, we propose a novel simulation-based network Denial of Service (DoS) resilience benchmark, and apply it to evaluate and compare the centralized, replicated, and emerging distributed object location services. Second, we propose the first algorithm that dynamically places close to optimal number of replicas while meeting client QoS and server resource constraint, with overlay network topology only. Third, we apply cooperative clustering-based replication to SCAN, which achieves comparable users' perceived performance to the conventional CDNs, while having only 4–5% of replication and update traffic, and 1–2% of the computation and replica management cost. Fourth, to provide a general foundation for applications to take advantage of network awareness, we develop a scalable overlay network measurement and monitoring system with two components: Internet Iso-bar for latency estimation, and TOM (Tomography-based Overlay network Monitoring) for loss rate estimation. TOM selectively monitors and measures the loss rates of a minimal basis set of O(n log n) linearly independent paths, and then applies them to estimate the loss rates of all other paths. Finally, to demonstrate the effectiveness of the monitoring services, we develop a adaptive overlay streaming media system which leverages our monitoring services for real-time path congestion/failure information, and an overlay network for adaptive packet relaying and buffering within the delivery infrastructure. (Abstract shortened by UMI.)