Repository-Level Compositional Code Translation and Validation

Ali Reza Ibrahimzada,Kaiyao Ke,Mrigank Pawagi,Muhammad Salman Abid,Rangeet Pan,Saurabh Sinha,Reyhaneh Jabbarvand

2024-11-01

Abstract:Code translation transforms programs from one programming language (PL) to another. Several rule-based transpilers have been designed to automate code translation between different pairs of PLs. However, the rules can become obsolete as the PLs evolve and cannot generalize to other PLs. Recent studies have explored the automation of code translation using Large Language Models (LLMs). One key observation is that such techniques may work well for crafted benchmarks but fail to generalize to the scale and complexity of real-world projects with dependencies, custom types, PL-specific features, etc. We propose AlphaTrans, a neuro-symbolic approach to automate repository-level code translation. AlphaTrans translates both source and test code, and employs multiple levels of validation to ensure the translation preserves the functionality of the source program. To break down the problem for LLMs, AlphaTrans leverages program analysis to decompose the program into fragments and translates them in the reverse call order. We leveraged AlphaTrans to translate ten real-world open-source projects consisting of <836, 8575, 2719> classes, methods, and tests. AlphaTrans translated the entire repository of these projects consisting of 6899 source code fragments. 99.1% of the translated code fragments are syntactically correct, and AlphaTrans validates the translations' runtime behavior and functional correctness for 25.8%. On average, the integrated translation and validation take 36 hours to translate a project, showing its scalability in practice. For the syntactically or semantically incorrect translations, AlphaTrans generates a report including existing translation, stack trace, test errors, or assertion failures. We provided these artifacts to two developers to fix the translation bugs in four projects. They were able to fix the issues in 20.1 hours on average and achieve all passing tests.

Software Engineering,Machine Learning

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the complexity and challenges of automated code translation and verification in practical projects. Specifically, the authors proposed solutions to the following problems:

1. **Complexity Caused by Differences in Programming Languages**:
- There may be fundamental differences between the source language and the target language, such as differences in programming paradigms, type systems, and memory management mechanisms.
- Some features specific to certain programming languages (for example, constructor overloading in Java) may not exist in other languages or may be implemented differently.

2. **Verifying Functional Equivalence**:
- The translated code should maintain the functionality of the source code, that is, functional equivalence.
- Existing technologies usually translate first and then verify, which may lead to verification delays and the inability to use verification results to correct translation errors.
- It is difficult to use test cases in the source language to verify the translated code, especially when dealing with complex call chains.

3. **Context Window Limitations**:
- For large - scale projects, the amount of code in the entire project or class may exceed the context window limitations of current large language models (LLMs).
- Even assuming an infinitely large context window, LLMs may not be able to correctly capture dependencies in the code due to short attention spans.

To solve these problems, the paper proposed a neuro - symbolic approach named AlphaTrans for automated code translation and verification at the repository level. The main features of AlphaTrans include:

- **Program Analysis and Reconstruction**: Decompose the source code into smaller fragments through static analysis and reconstruct features specific to a particular programming language to adapt to the target language.
- **Segmented Translation and Verification**: Translate code fragments one by one in reverse call order and use existing tests for verification to ensure the functional equivalence of each fragment.
- **Multi - level Verification**: Combine language interoperability and unit test decomposition to provide multiple levels of verification to ensure the accuracy and reliability of the translation.

Through these methods, AlphaTrans can efficiently complete code translation and verification for large - scale projects while maintaining functional equivalence. Experimental results show that AlphaTrans can successfully translate and verify multiple real - world Java projects into Python, demonstrating its effectiveness and scalability in practical applications.