Abstract:Serverless computing is an emerging cloud computing paradigm that enables developers to build applications at the function level, known as serverless applications. Amazon Web Services (AWS), the leading provider in this domain, provides the Serverless Application Model (AWS SAM), the most widely adopted configuration schema for configuring and managing serverless applications through a specified file. However, misconfigurations pose a significant challenge in serverless development. Traditional data-driven techniques may struggle with serverless applications because the complexity of serverless configurations hinders pattern recognition, and it is challenging to gather complete datasets that cover all possible configurations. Leveraging vast amounts of publicly available data during pre-training, LLMs can have the potential to assist in identifying and explaining misconfigurations in serverless applications. In this paper, we introduce SlsDetector, the first framework leveraging LLMs to detect misconfigurations in serverless applications. SlsDetector utilizes effective prompt engineering with zero-shot learning to identify configuration issues. It designs multi-dimensional constraints specifically tailored to the configuration characteristics of serverless applications and leverages the Chain of Thought technique to enhance LLMs inferences. We evaluate SlsDetector on a curated dataset of 110 configuration files. Our results show that SlsDetector, based on ChatGPT-4o, achieves a precision of 72.88%, recall of 88.18%, and F1-score of 79.75%, outperforming state-of-the-art data-driven approaches by 53.82, 17.40, and 49.72 percentage points, respectively. Furthermore, we investigate the generalization capability of SlsDetector by applying recent LLMs, including Llama 3.1 (405B) Instruct Turbo and Gemini 1.5 Pro, with results showing consistently high effectiveness across these models.

What problem does this paper attempt to address?

### What problem does this paper attempt to solve? This paper aims to solve the problem of misconfigurations detection in Serverless Computing. Specifically, the paper proposes an LLM - based framework, **SlsDetector**, for AWS serverless application configuration files, which is used to automatically detect and interpret errors in the configuration. #### Background and Problem Description Serverless computing is an emerging cloud - computing paradigm that allows developers to build and run applications without having to manage the underlying infrastructure. Amazon Web Services (AWS) is a leading provider in this field and offers Serverless Application Model (AWS SAM), the most widely - adopted configuration scheme for configuring and managing serverless applications. However, misconfigurations are a significant challenge in serverless application development. These errors can lead to serious security vulnerabilities and operational problems. For example: - A COVID - 19 testing company had its AWS S3 bucket misconfigured, resulting in the scan IDs of more than 50,000 patients and thousands of COVID - 19 test results being made public. - Another company had a data leak of 4.9 million customers due to API misconfiguration. These problems indicate that misconfigurations are not just individual incidents but systemic issues, posing a significant risk to serverless applications. Traditional data - driven methods (such as learning configuration patterns based on historical data to identify anomalies) are not effective in the serverless environment because: - The data sets are incomplete or inaccurate. - The configurations are complex, involving domain - specific languages, complex dependencies and nested objects, covering more than 800 cloud - resource types. #### Solution To solve the above problems, the authors propose an LLM - based solution - **SlsDetector**. This framework can efficiently identify configuration problems by using advanced prompt engineering and zero - shot learning techniques. The main features of SlsDetector include: 1. **Multi - dimensional Constraint Design**: According to the characteristics of serverless application configurations, multi - dimensional constraints are designed, covering resource types, configuration items, values, dependencies, etc. 2. **Chain of Thought (CoT) Technique**: The reasoning ability of the LLM is enhanced through step - by - step reasoning, improving the detection accuracy. 3. **Customized Response**: Generate a structured output and provide detailed error explanations to ensure that the response is not only structured but also actionable. #### Experimental Results The authors evaluated SlsDetector using a data set of 110 configuration files (including correct configurations, real - world misconfigurations and deliberately injected errors). The experimental results show that SlsDetector based on ChatGPT - 4o achieves 72.88%, 88.18% and 79.75% in precision, recall and F1 - score respectively, significantly outperforming existing data - driven methods. In addition, SlsDetector also shows consistently high effectiveness on other representative LLMs such as Llama 3.1 Instruct Turbo and Gemini 1.5 Pro. In conclusion, this paper shows how to effectively detect misconfigurations in serverless applications by introducing SlsDetector, thereby improving the security and reliability of serverless applications.

LLM-Based Misconfiguration Detection for AWS Serverless Computing

Detection of Compromised Functions in a Serverless Cloud Environment

Configuration Validation with Large Language Models

Identifying Performance-Sensitive Configurations in Software Systems through Code Analysis with LLM Agents

ServerlessLLM: Low-Latency Serverless Inference for Large Language Models

LLMScan: Causal Scan for LLM Misbehavior Detection

Edge-Assisted Adaptive Configuration for Serverless-Based Video Analytics

AutoDetect: Towards a Unified Framework for Automated Weakness Detection in Large Language Models

LLMs are Imperfect, Then What? An Empirical Study on LLM Failures in Software Engineering

Exploring Automatic Cryptographic API Misuse Detection in the Era of LLMs

Beyond Binary: Towards Fine-Grained LLM-Generated Text Detection via Role Recognition and Involvement Measurement

Assessing Hidden Risks of LLMs: An Empirical Study on Robustness, Consistency, and Credibility

Utilizing Precise and Complete Code Context to Guide LLM in Automatic False Positive Mitigation

Detectors for Safe and Reliable LLMs: Implementations, Uses, and Limitations

MetaLLM: A High-performant and Cost-efficient Dynamic Framework for Wrapping LLMs

LLMGuard: Guarding Against Unsafe LLM Behavior

Enabling Efficient Serverless Inference Serving for LLM (Large Language Model) in the Cloud

Legilimens: Practical and Unified Content Moderation for Large Language Model Services

Intent Detection in the Age of LLMs

DisLLM: Distributed LLMs for Privacy Assurance in Resource-Constrained Environments