Abstract:Domain-specific languages (DSLs) are integral to various software workflows. Such languages offer domain-specific optimizations and abstractions that improve code readability and maintainability. However, leveraging these languages requires developers to rewrite existing code using the specific DSL's API. While large language models (LLMs) have shown some success in automatic code transpilation, none of them provide any functional correctness guarantees on the transpiled code. Another approach for automating this task is verified lifting, which relies on program synthesis to find programs in the target language that are functionally equivalent to the source language program. While several verified lifting tools have been developed for various application domains, they are specialized for specific source-target languages or require significant expertise in domain knowledge to make the search efficient. In this paper, leveraging recent advances in LLMs, we propose an LLM-based approach (LLMLift) to building verified lifting tools. We use the LLM's capabilities to reason about programs to translate a given program into its corresponding equivalent in the target language. Additionally, we use LLMs to generate proofs for functional equivalence. We develop lifting-based compilers for {\em four different} DSLs targeting different application domains. Our approach not only outperforms previous symbolic-based tools in both the number of benchmarks transpiled and transpilation time, but also requires significantly less effort to build.

What problem does this paper attempt to address?

The problem addressed in this paper is how to use large-scale language models (LLMs) for code transformation while ensuring the correctness of the transformed code. Existing methods, such as domain-specific languages (DSLs), enhance code readability and maintainability but require manual code rewriting to fit the specific DSL's API. Although LLMs have achieved some success in automatic code transformation, they cannot provide guarantees of the correctness of the transformed code. On the other hand, verified lifting relies on program synthesis to find target language programs that are functionally equivalent to the source code and then formally verifies their equivalence. However, this approach is usually specific to certain source-target languages or requires a lot of expertise. The paper proposes an LLM-based technique called LLMLIFT for verified lifting, which leverages the capability of LLMs to translate a given program into an equivalent program in the target language and produces a proof of functional equivalence. In this way, they developed compilers for four DSLs in different application domains, surpassing previous symbol-based methods in terms of the number and time of transformation benchmarks and significantly reducing the required effort. The main contributions of the paper include: 1. The first proposal of using LLMs for formal verification in code transformation. 2. Using Python as an intermediate representation (IR) language, avoiding the need for training data specific to a DSL or fine-tuning of LLMs. 3. Simplification of the verified lifting process, reducing the manual workload compared to traditional techniques. 4. Combining LLM-generated code with correctness proofs to ensure the correctness of the generated code. 5. Demonstrating the effectiveness of the compilers in four DSL application domains, with improvements in accuracy and efficiency compared to existing tools. In conclusion, the paper aims to address the problem of automatically and correctly transforming code using LLMs and improves the efficiency and reliability of code transformation through innovative approaches.

Verified Code Transpilation with LLMs

LLM4PLC: Harnessing Large Language Models for Verifiable Programming of PLCs in Industrial Control Systems

Towards Automated Verification of LLM-Synthesized C Programs

Enhancing Translation Validation of Compiler Transformations with Large Language Models

Don't Transform the Code, Code the Transforms: Towards Precise Code Rewriting using LLMs

LASSI: An LLM-based Automated Self-Correcting Pipeline for Translating Parallel Scientific Codes

LLM4VV: Exploring LLM-as-a-Judge for Validation and Verification Testsuites

An LLM-Tool Compiler for Fused Parallel Function Calling

Evaluating Large Language Models for Automatic Register Transfer Logic Generation via High-Level Synthesis

LEVER: Learning to Verify Language-to-Code Generation with Execution

Towards Large Language Models as Copilots for Theorem Proving in Lean

Combining LLM Code Generation with Formal Specifications and Reactive Program Synthesis

VeCoGen: Automating Generation of Formally Verified C Code with Large Language Models

Scalable, Validated Code Translation of Entire Projects using Large Language Models

Advanced Large Language Model (LLM)-Driven Verilog Development: Enhancing Power, Performance, and Area Optimization in Code Synthesis

LLM4VV: Developing LLM-driven testsuite for compiler validation

Make Every Move Count: LLM-based High-Quality RTL Code Generation Using MCTS

Towards Large Language Model Aided Program Refinement

LLM-Vectorizer: LLM-based Verified Loop Vectorizer

CoTran: An LLM-based Code Translator using Reinforcement Learning with Feedback from Compiler and Symbolic Execution