Abstract:The code written by developers usually suffers from efficiency problems and contain various performance bugs. These inefficiencies necessitate the research of automated refactoring methods for code optimization. Early research in code optimization employs rule-based methods and focuses on specific inefficiency issues, which are labor-intensive and suffer from the low coverage issue. Recent work regards the task as a sequence generation problem, and resorts to deep learning (DL) techniques such as large language models (LLMs). These methods typically prompt LLMs to directly generate optimized code. Although these methods show state-of-the-art performance, such one-step generation paradigm is hard to achieve an optimal solution. First, complex optimization methods such as combinatorial ones are hard to be captured by LLMs. Second, the one-step generation paradigm poses challenge in precisely infusing the knowledge required for effective code optimization within LLMs, resulting in under-optimized <a class="link-external link-http" href="http://code.To" rel="external noopener nofollow">this http URL</a> address these problems, we propose to model this task from the search perspective, and propose a search-based LLMs framework named SBLLM that enables iterative refinement and discovery of improved optimization methods. SBLLM synergistically integrate LLMs with evolutionary search and consists of three key components: 1) an execution-based representative sample selection part that evaluates the fitness of each existing optimized code and prioritizes promising ones to pilot the generation of improved code; 2) an adaptive optimization pattern retrieval part that infuses targeted optimization patterns into the model for guiding LLMs towards rectifying and progressively enhancing their optimization methods; and 3) a genetic operator-inspired chain-of-thought prompting part that aids LLMs in combining different optimization methods and generating improved optimization methods.

Code Optimization Chain-of-Thought: Structured Understanding and Self-Checking

Chain of Code: Reasoning with a Language Model-Augmented Code Emulator

CodeChain: Towards Modular Code Generation Through Chain of Self-revisions with Representative Sub-modules

Learning to Check: Unleashing Potentials for Self-Correction in Large Language Models

Search-Based LLMs for Code Optimization

Enhancing Large Language Models in Coding Through Multi-Perspective Self-Consistency

Chain of Preference Optimization: Improving Chain-of-Thought Reasoning in LLMs

Improving Natural Language Capability of Code Large Language Model

LLM-Assisted Code Cleaning For Training Accurate Code Generators

Chain-of-Thought in Neural Code Generation: From and For Lightweight Language Models

AI Chain on Large Language Model for Unsupervised Control Flow Graph Generation for Statically-Typed Partial Code

Should AI Optimize Your Code? A Comparative Study of Current Large Language Models Versus Classical Optimizing Compilers

What's Wrong with Your Code Generated by Large Language Models? An Extensive Study

VISUALCODER: Guiding Large Language Models in Code Execution with Fine-grained Multimodal Chain-of-Thought Reasoning

Enhancing Code Generation Performance of Smaller Models by Distilling the Reasoning Ability of LLMs

Training LLMs to Better Self-Debug and Explain Code

Improving LLM Reasoning through Scaling Inference Computation with Collaborative Verification

Large Language Models for Code Analysis: Do LLMs Really Do Their Job?

MoTCoder: Elevating Large Language Models with Modular of Thought for Challenging Programming Tasks

Self-planning Code Generation with Large Language Models