Abstract:The Internet has evolved to become a critical commercial infrastructure for service delivery. However, the Internet being an enormous, highly-dynamic, heterogeneous, and untrusted environment raises several challenges for building Internet-scale services (such as content delivery) with good scalability, efficiency, agility and security. In this thesis, we explore these issues by developing a scalable, adaptive and network-aware infrastructure for efficient content delivery, namely Scalable Content Access Network (SCAN). SCAN has four components: object location, replica placement and update multicast tree construction, replica management, and overlay network monitoring services. First, we propose a novel simulation-based network Denial of Service (DoS) resilience benchmark, and apply it to evaluate and compare the centralized, replicated, and emerging distributed object location services. Second, we propose the first algorithm that dynamically places close to optimal number of replicas while meeting client QoS and server resource constraint, with overlay network topology only. Third, we apply cooperative clustering-based replication to SCAN, which achieves comparable users' perceived performance to the conventional CDNs, while having only 4–5% of replication and update traffic, and 1–2% of the computation and replica management cost. Fourth, to provide a general foundation for applications to take advantage of network awareness, we develop a scalable overlay network measurement and monitoring system with two components: Internet Iso-bar for latency estimation, and TOM (Tomography-based Overlay network Monitoring) for loss rate estimation. TOM selectively monitors and measures the loss rates of a minimal basis set of O(n log n) linearly independent paths, and then applies them to estimate the loss rates of all other paths. Finally, to demonstrate the effectiveness of the monitoring services, we develop a adaptive overlay streaming media system which leverages our monitoring services for real-time path congestion/failure information, and an overlay network for adaptive packet relaying and buffering within the delivery infrastructure. (Abstract shortened by UMI.)

A Content Aware Scheduling System for Network Services in Linux Clusters 1

Awareness scheduling and algorithm implementation for collaborative virtual environment

A Novel Job Scheduling Model to Enhance Efficiency and Overall User Fairness of Cloud Computing Environment.

A User-Level NUMA-Aware Scheduler for Optimizing Virtual Machine Performance.

A New Network Storage Architecture of Cache-Cluster Based on Scheduling Decision Strategy

A Simulation-Based Study of Scheduling Mechanisms for a Dynamic Cluster Environment.

Scalable Scheduling on a Network of Workstations.

A Throughput-Driven Scheduling Algorithm of Differentiated Service for Web Cluster

Multi-Cluster Co-Allocation Scheduling Algorithms in Virtual Computing Environment

Research and Application of Real-Time Multi-Task Cluster Management System

Energy Efficient Scheduling in Content Distribution Collaborative Mobile Clusters

Towards a Scalable, Adaptive and Network-Aware Content Distribution Network

Bahs: A Bandwidth-Aware Heterogeneous Scheduling Approach For Sdn-Based Cluster Systems

NetClust: A Framework for Scalable and Pareto-Optimal Media Server Placement

Scheduling Best-Effort and Real-Time Pipelined Applications on Time-Shared Clusters.

Impact of Workload and System Parameters on Next Generation Cluster Scheduling Mechanisms

Content-aware URL Dispatch Algorithm for Web Server Cluster

ClusterSchedSim: A Unifying Simulation Framework for Cluster Scheduling Strategies

Request-Aware Scheduling for Busy Internet Services

A Scheduler System for Large-Scale Distributed Data Computing in Cloud

Dynamic Feedback Scheduling Algorithm of the Video Server Cluster System