Abstract:Performance of data streaming applications is co-determined by both networking and computing resources, and therefore they should be co-scheduled and co-allocated in an integrated and coordinated way. Dynamic control of resource scheduling and allocation is required, because unilateral redundancy in either networking or computing resources may result in the overprovision of it and the other may become a bottleneck. To avoid resource shortage as well as overprovision, in this paper, a virtualized platform is utilized to implement data streaming and processing. In this platform, fuzzy logic controllers are designed to allocate CPU resources; iterative bandwidth allocation is applied and is processing-and storage-aware to guarantee on-demand data provisioning. Experimental results show that our approach leads to higher application performance as well as higher resource utilization, compared with other resource scheduling and allocation methods.Note to Practitioners-Data streaming has become an important paradigm in many business and scientific applications, such as fraud detection in banking industry and gravitational-wave observation in astronomy. In this paper, we apply virtualization to provide better support for these applications, leveraging its advantage in resource on-demand allocation. A novel dynamic control method is proposed so that CPU and bandwidth can be co-scheduled and co-allocated since for data streaming applications these resources are tightly coupled from the performance perspective. In this control method, fuzzy logic control is applied for CPU allocation and an iterative algorithm is adopted for processing-, congestion-, and storage-aware bandwidth allocation. We use fuzzy control because it does not rely on the mathematical modeling of an object and can implement human experts' heuristic knowledge via IF-THEN rules. Therefore, it is desirable in our data streaming scenario, due to the variable coupling and heavily nonlinear nature of the system.

Resource-Aware Stream Processing in High Performance Cloud Environment

A Novel Job Scheduling Model to Enhance Efficiency and Overall User Fairness of Cloud Computing Environment.

A Game-Theoretic Method of Fair Resource Allocation for Cloud Computing Services

Dynamic Controlling of Data Streaming Applications for Cloud Computing

Study and Implementation of Elastic Stream Computing in the Cloud

Intelligent resource allocation scheme for cloud-edge-end framework aided multi-source data stream

Throughput-Guarantee Resource Provisioning for Streaming Analytical Workflows in the Cloud

Dynamic Control of Data Streaming and Processing in a Virtualized Environment

Resource Scheduling for Real-Time Analytical Workflow Services in the Cloud

Elastic Allocator: An Adaptive Task Scheduler for Streaming Query in the Cloud

A Framework for Partitioning and Execution of Data Stream Applications in Mobile Cloud Computing

Design and Implementation of Adaptive Resource Co-Allocation Approaches for Cloud Service Environments

Resource- and Message Size-Aware Scheduling of Stream Processing at the Edge with application to Realtime Microscopy

A Dynamic Two-Phase Schedule Strategy for Streaming Media Edge Cloud

Cost-Effective Resource Provisioning for Real-Time Workflow in Cloud

An Adaptive Management Approach Collaborating Heterogeneous Resources in Cloud Service Environments

Fairness-Efficiency Allocation of CPU-GPU Heterogeneous Resources

Resource Deployment in Data Communication Intensive Cloud Environment

Multi-resource Fair Allocation with Bandwidth Requirement Compression in the Cloud–edge System

Cost-Aware Streaming Workflow Allocation on Geo-Distributed Data Centers.

Adaptive Scheduling Framework of Streaming Applications based on Resource Demand Prediction with Hybrid Algorithms