Abstract:Datacenters running on-line, data-intensive applications (OLDIs) consume significant amounts of energy. However, reducing their energy is challenging due to their tight response time requirements. A key aspect of OLDIs is that each user query goes to all or many of the nodes in the cluster, so that the overall time budget is dictated by the tail of the replies' latency distribution; replies see latency variations both in the network and compute. Previous work proposes to achieve load-proportional energy by slowing down the computation at lower datacenter loads based directly on response times (i.e., at lower loads, the proposal exploits the average slack in the time budget provisioned for the peak load). In contrast, we propose TimeTrader to reduce energy by exploiting the latency slack in the sub- critical replies which arrive before the deadline (e.g., 80% of replies are 3-4x faster than the tail). This slack is present at all loads and subsumes the previous work's load-related slack. While the previous work shifts the leaves' response time distribution to consume the slack at lower loads, TimeTrader reshapes the distribution at all loads by slowing down individual sub-critical nodes without increasing missed deadlines. TimeTrader exploits slack in both the network and compute budgets. Further, TimeTrader leverages Earliest Deadline First scheduling to largely decouple critical requests from the queuing delays of sub- critical requests which can then be slowed down without hurting critical requests. A combination of real-system measurements and at-scale simulations shows that without adding to missed deadlines, TimeTrader saves 15-19% and 41-49% energy at 90% and 30% loading, respectively, in a datacenter with 512 nodes, whereas previous work saves 0% and 31-37%.

TailGuard: Tail Latency SLO Guaranteed Task Scheduling for Data-Intensive User-Facing Applications

SafeTail: Efficient Tail Latency Optimization in Edge Service Scheduling via Computational Redundancy Management

TailCutter: Wisely Cutting Tail Latency in Cloud CDNs under Cost Constraints

Tail-Learning: Adaptive Learning Method for Mitigating Tail Latency in Autonomous Edge Systems

Optimizing Tail Latency by Critical Window-based Dynamic Cache Space Allocation

Towards Minimal Tardiness Of Data-Intensive Applications In Heterogeneous Networks

Cost-efficient Security-Aware Scheduling for Dependent Tasks with Endpoint Contention in Edge Computing

QoS-Aware Task Placement with Fault-Tolerance in the Edge-Cloud

PCS: Predictive Component-level Scheduling for Reducing Tail Latency in Cloud Online Services

Strongly Tail-Optimal Scheduling in the Light-Tailed M/G/1

Energy-Efficient Fault-Tolerant Scheduling Algorithm for Real-Time Tasks in Cloud-Based 5G Networks.

Task scheduling in cloud environment: optimization, security prioritization and processor selection schemes

Building a Fault Tolerant Framework with Deadline Guarantee in Big Data Stream Computing Environments.

Security-Aware and Time-Guaranteed Service Placement in Edge Clouds

Efficiency-First Fault-Tolerant Replica Scheduling Strategy for Reliability Constrained Cloud Application.

An Index Based Task Assignment Policy For Achieving Optimal Power-Delay Tradeoff In Edge Cloud Systems

Background Traffic Optimization for Meeting Deadlines in Data Center Storage

QoS-Aware Task Scheduling in Cloud-Edge Environment

TimeTrader: Exploiting Latency Tail to Save Datacenter Energy for On-line Data-Intensive Applications

Offloading Optimization with Delay Distribution in the 3-tier Federated Cloud, Edge, and Fog Systems

Fault-tolerant scheduling and data placement for scientific workflow processing in geo-distributed clouds