Abstract:With the substantial incensement of broadband networks, Internet applications have shifted from simple web browsing to content-centric applications. From the perspective of the content distributor, how to reduce the cost while satisfying quality of service and how to respond timely to users are of great concern. Currently, using multi-cloud technology is a feasible solution to provide more agile and scalable services. Meanwhile, big data techniques, such as Spark and Hadoop, can help content distributors make load-direct decisions more timely and accurately. In this paper, we present a multi-cloud architecture-supported resource allocation and scheduling optimized strategy through CDN (content delivery network) operation big data analysis. We firstly analyze quantities of CDN operation log data on Spark to evaluate quality of service (QoS) between end users and multi-cloud-based CDN operator. Then we perform a long-term resource deployment algorithm to book the minimum resources to meet users' requests with higher QoS and lower cost. We apply the prediction model ARIMA on Spark to predict the short term demand through analyzing a longer time series data. When the predicted resources cannot satisfy burst demand, we design a new multi-cloud extension algorithm to schedule additional cloud resource to handle overload requests and use precopying algorithm to select media contents to be stored in the new prepared cloud. We implement and evaluate our scheme with real operation logs data provided by China's biggest CDN distributor to show the efficiency of our algorithms.

An efficient data dissemination approach for cloud monitoring

Cloud Data Management: A Survey

The Service of Secure Data Publication and Subscription on Cloud Platforms

JTangCMS: an Efficient Monitoring System for Cloud Platforms.

Data Management Solutions Based on the Data Distribution Service Communication Model

Research on Service-Oriented Data Mining Engine Based on Cloud Computing

Security-Aware Efficient Mass Distributed Storage Approach for Cloud Systems in Big Data

Optimizing Multi-Cloud CDN Deployment and Scheduling Strategies Using Big Data Analysis

Service Intelligence Oriented Distributed Data Stream Integration

Research on Cloud-Edge Collaborative Structure of Distribution Internet of Things for Data Monitoring and Analysis

Recent Developments in Parallel and Distributed Computing for Remotely Sensed Big Data Processing

SimMon: A Toolkit for Simulating Monitoring Mechanism in Cloud Computing Environments.

A New Block-Based Data Distribution Mechanism in Cloud Computing

Distributed Cache Memory Data Migration Strategy Based on Cloud Computing

SSC: Cloud-Based Data Stream Management in Distributed Environments

Providing Scalable Database Services On The Cloud

SLA-DO: A SLA-Based Data Distribution Strategy on Multiple Cloud Storage Systems

Ripple: A publish/subscribe service for multidata item updates propagation in the cloud

MonValley: an Unified Monitoring and Management Framework for Cloud Services

A Scheduler System for Large-Scale Distributed Data Computing in Cloud

Multilevel Cooperative On-Line Computing Architecture under Dispatching Cloud