WarmSwap: Sharing Dependencies for Accelerating Cold Starts in Serverless Functions

Rui Li,Devesh Tiwari,Gene Cooperman

2024-10-21

Abstract:This work presents WarmSwap, a novel provider-side cold-start optimization for serverless computing. This optimization reduces cold-start time when booting and loading dependencies at runtime inside a function container. Previous approaches to the optimization of cold starts tend to fall into two categories: optimizing the infrastructure of serverless computing to benefit all serverless functions; or function-specific tuning for individual serverless functions. In contrast, WarmSwap offers a broad middle ground, which optimizes entire categories of serverless functions. WarmSwap eliminates the need to initialize middleware or software dependencies when launching a new serverless container, by migrating a pre-initialized live dependency image to the new function instance. WarmSwap respects the provider's cache constraints, as a single pre-warmed dependency image in the cache is shared among all serverless functions requiring that software dependency image. WarmSwap has been tested on seven representative functions from FunctionBench. In those tests, WarmSwap accelerates dependency loading for serverless functions with large dependency requirements by a factor ranging from 2.2 to 3.2. Simulation experiments using Azure traces indicate that WarmSwap can save 88\% of optimization space when sharing a dependency image among ten different functions.

Distributed, Parallel, and Cluster Computing

What problem does this paper attempt to address?

The paper attempts to address the issue of long cold start times in serverless computing, particularly when loading software dependencies. Specifically, when a cloud provider first loads a user function into a serverless instance upon invocation, this process is known as a cold start. Cold starts are typically slow because middleware or software dependencies need to be initialized. This issue negatively impacts user experience and cost, especially in scenarios where functions are frequently invoked or the system needs to scale rapidly.

To tackle this challenge, the paper proposes WarmSwap, a new provider-side cold start optimization method. WarmSwap reduces cold start time by migrating pre-initialized dependency images to new function instances, rather than loading these dependencies from scratch. This approach not only speeds up dependency loading but also reduces memory usage, thereby improving the overall efficiency of serverless computing.

The main contributions of WarmSwap include:
1. Proposing a new infrastructure optimization method aimed at accelerating dependency loading and reducing cold start time, particularly for functions that are infrequently invoked.
2. Providing a cost-effective solution by storing middleware in separate dependency images rather than the entire container, allowing a single pre-warmed dependency image to be shared across different serverless functions and worker nodes.
3. Optimizing for the "long tail" of serverless function invocation distribution in real production environments, where functions are invoked infrequently and are not suitable for specific function optimizations.

The design and implementation details of WarmSwap include the generation of dependency image pools, migration design, and the overall system architecture. Experimental validation shows that WarmSwap is 2.2 to 3.2 times faster than the traditional AWS Lambda startup process when loading large dependencies such as "sklearn+pandas," "numpy+keras," and "numpy+torch." Additionally, it can save 88% of memory space when pre-warming 10 different serverless functions that share the same dependencies.