Abstract:The advent of large language models (LLMs) has significantly advanced the field of code translation, enabling automated translation between programming languages. However, these models often struggle with complex translation tasks due to inadequate contextual understanding. This paper introduces a novel approach that enhances code translation through Few-Shot Learning, augmented with retrieval-based techniques. By leveraging a repository of existing code translations, we dynamically retrieve the most relevant examples to guide the model in translating new code segments. Our method, based on Retrieval-Augmented Generation (RAG), substantially improves translation quality by providing contextual examples from which the model can learn in real-time. We selected RAG over traditional fine-tuning methods due to its ability to utilize existing codebases or a locally stored corpus of code, which allows for dynamic adaptation to diverse translation tasks without extensive retraining. Extensive experiments on diverse datasets with open LLM models such as Starcoder, Llama3-70B Instruct, CodeLlama-34B Instruct, Granite-34B Code Instruct, and Mixtral-8x22B, as well as commercial LLM models like GPT-3.5 Turbo and GPT-4o, demonstrate our approach's superiority over traditional zero-shot methods, especially in translating between Fortran and CPP. We also explored varying numbers of shots i.e. examples provided during inference, specifically 1, 2, and 3 shots and different embedding models for RAG, including Nomic-Embed, Starencoder, and CodeBERT, to assess the robustness and effectiveness of our approach.

Improving Sentence Embeddings with Automatic Generation of Training Data Using Few-shot Examples

GenEOL: Harnessing the Generative Power of LLMs for Training-Free Sentence Embeddings

Simple Techniques for Enhancing Sentence Embeddings in Generative Language Models

Few-Shot Natural Language Inference Generation with PDD: Prompt and Dynamic Demonstration

Making Text Embedders Few-Shot Learners

Boosting LLMS with Ontology-Aware Prompt for Ner Data Augmentation

AUGNLG: Few-shot Natural Language Generation using Self-trained Data Augmentation

Does Few-Shot Learning Help LLM Performance in Code Synthesis?

Enhancing Text Annotation through Rationale-Driven Collaborative Few-Shot Prompting

LLM-DA: Data Augmentation via Large Language Models for Few-Shot Named Entity Recognition

Few-Shot Dialogue Generation Without Annotated Data: A Transfer Learning Approach

Enhancing Unsupervised Sentence Embeddings via Knowledge-Driven Data Augmentation and Gaussian-Decayed Contrastive Learning

On the Effectiveness of Sentence Encoding for Intent Detection Meta-Learning

Improving Text Embeddings with Large Language Models

Prompting-based Synthetic Data Generation for Few-Shot Question Answering

Meta-Task Prompting Elicits Embeddings from Large Language Models

Enhancing Code Translation in Language Models with Few-Shot Learning via Retrieval-Augmented Generation

Exploiting the Potential of Seq2Seq Models as Robust Few-Shot Learners

Towards Practical Few-shot Federated NLP

Assertion Enhanced Few-Shot Learning: Instructive Technique for Large Language Models to Generate Educational Explanations