In Serverless, OS Scheduler Choice Costs Money: A Hybrid Scheduling Approach for Cheaper FaaS

Yuxuan Zhao,Weikang Weng,Rob van Nieuwpoort,Alexandru Uta

DOI: https://doi.org/10.1145/3652892.3700757

2024-11-13

Abstract:In Function-as-a-Service (FaaS) serverless, large applications are split into short-lived stateless functions. Deploying functions is mutually profitable: users need not be concerned with resource management, while providers can keep their servers at high utilization rates running thousands of functions concurrently on a single machine. It is exactly this high concurrency that comes at a cost. The standard Linux Completely Fair Scheduler (CFS) switches often between tasks, which leads to prolonged execution times. We present evidence that relying on the default Linux CFS scheduler increases serverless workloads cost by up to 10X. In this article, we raise awareness and make a case for rethinking the OS-level scheduling in Linux for serverless workloads composed of many short-lived processes. To make serverless more affordable we introduce a hybrid two-level scheduling approach that relies on FaaS characteristics. Short-running functions are executed in FIFO fashion without preemption, while longer-running functions are passed to CFS after a certain time period. We show that tailor-made OS scheduling is able to significantly reduce user-facing costs without adding any provider-facing overhead.

Distributed, Parallel, and Cluster Computing

What problem does this paper attempt to address?

### Problems the Paper Attempts to Solve

This paper aims to address the operating system scheduling issues in serverless computing, particularly for short-lived functions in Function-as-a-Service (FaaS). Currently, the standard Linux Completely Fair Scheduler (CFS) leads to frequent context switches when handling highly concurrent short-lived functions, thereby increasing execution time and cost. Specifically, the paper points out that relying on the default Linux CFS scheduler can increase the cost of serverless workloads by up to 10 times.

To make serverless computing more cost-effective, the paper proposes a hybrid two-layer scheduling method. This method leverages the characteristics of FaaS by executing short-lived functions in a First-In-First-Out (FIFO) manner without preemption; long-lived functions are handed over to the CFS scheduler after a certain period. By doing so, the paper demonstrates that customized operating system scheduling can significantly reduce user costs without increasing the provider's overhead.

### Main Contributions

1. **Evidence Presentation**: The paper provides evidence that CFS scheduling for serverless functions is suboptimal and increases user costs.
2. **Hybrid Scheduler Design and Implementation**: A user-space hybrid scheduler is designed and implemented, dividing CPU cores into two groups, each adopting different scheduling strategies.
3. **Time Limit Adaptation Mechanism**: A mechanism is designed and implemented to dynamically adjust the FIFO time limit based on recent task execution.
4. **Experimental Validation**: Experiments demonstrate that this scheduler can significantly reduce the execution time and cost of serverless functions on actual platforms (e.g., Firecracker).

### Background and Motivation

- **Characteristics of Serverless Functions**: Serverless functions typically exhibit burstiness and short lifecycles. Most functions execute within 1 second, and memory allocation is usually less than 400MB.
- **Cost Model**: Cloud providers charge per millisecond, so frequent context switches can significantly increase user costs.
- **Limitations of Existing Schedulers**: The standard CFS scheduler, while aiming for fairness, leads to extended execution times due to frequent context switches, thereby increasing user costs.

### Hybrid Scheduler Design

- **Two Groups of CPU Cores**: CPU cores are divided into two groups, one adopting the FIFO strategy and the other adopting the CFS strategy.
- **Short Task Group**: Dedicated to quickly processing short-lived tasks, which are not preempted if completed within a preset time.
- **Long Task Group**: Handles long-lived tasks, with preempted tasks being allocated to this group of cores for continued execution.
- **Dynamic Adjustment**: The FIFO time limit is dynamically adjusted based on recent task execution, and the number of cores in each group is dynamically adjusted based on CPU utilization.

Through these designs, the proposed method in the paper can maintain high CPU utilization while significantly reducing user costs and latency.