Abstract:We introduce MPLSandbox, an out-of-the-box multi-programming language sandbox designed to provide unified and comprehensive feedback from compiler and analysis tools for Large Language Models (LLMs). It can automatically identify the programming language of the code, compiling and executing it within an isolated sub-sandbox to ensure safety and stability. In addition, MPLSandbox also integrates both traditional and LLM-based code analysis tools, providing a comprehensive analysis of generated code. MPLSandbox can be effortlessly integrated into the training and deployment of LLMs to improve the quality and correctness of their generated code. It also helps researchers streamline their workflows for various LLM-based code-related tasks, reducing the development cost. To validate the effectiveness of MPLSandbox, we integrate it into training and deployment approaches, and also employ it to optimize workflows for a wide range of real-world code-related tasks. Our goal is to enhance researcher productivity on LLM-based code-related tasks by simplifying and automating workflows through delegation to MPLSandbox.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the limitations of existing sandbox tools when dealing with code generated by large - language models (LLMs), which are specifically as follows: 1. **Insufficient support for multiple programming languages**: Most of the existing sandbox tools only support a single programming language and cannot be easily integrated into the training and deployment processes of LLMs. This restricts the application of LLMs in tasks involving multiple programming languages. 2. **Lack of unified compiler feedback**: Compiler feedback for different programming languages is inconsistent, making it difficult to provide unified and comprehensive code analysis results to LLMs, which affects the quality and correctness of the code generated by LLMs. 3. **Diversity and complexity of code analysis tools**: Different programming languages have different code analysis tools, and the application of these tools increases the cost for researchers to develop and use these tools. In addition, the same type of analysis tool may vary in different programming languages, further increasing the complexity. 4. **The need for simplifying and automating the workflow**: Researchers need a workflow tool that can simplify and automate the processing of code generated by LLMs to improve productivity and reduce development costs. To solve these problems, the paper proposes **MPLSandbox**, which is a multi - programming - language sandbox environment aiming to provide unified compiler feedback and comprehensive code analysis for LLMs. MPLSandbox has the following features: - **Security and stability**: By constructing sub - sandboxes for each programming language, it ensures that the code is compiled and executed in an isolated environment, preventing malicious code or vulnerabilities from harming the external environment. - **Support for multiple programming languages**: Automatically identify the programming language of the code and send it to the corresponding sandbox environment for thorough analysis, significantly reducing the cost for researchers to develop and deploy LLMs. - **Usability and extensibility**: Integrates multiple analysis tools, and users can easily design tool templates and integrate their own tools. In addition, users can also construct prompt templates, combining compiler feedback and analysis results to enhance the performance of LLMs. - **Distributed architecture**: Designed for distributed deployment. In large - scale training scenarios, training nodes can access optional MPLSandbox nodes, providing higher efficiency compared to single - machine deployment. Through these improvements, MPLSandbox can simplify the application of LLMs in code - related tasks, provide reliable verification and feedback, and thus improve the quality and reliability of the code generated by LLMs.

Multi-Programming Language Sandbox for LLMs