Abstract:Software Engineering (SE) is the systematic design, development, and maintenance of software applications, underpinning the digital infrastructure of our modern mainworld. Very recently, the SE community has seen a rapidly increasing number of techniques employing Large Language Models (LLMs) to automate a broad range of SE tasks. Nevertheless, existing information of the applications, effects, and possible limitations of LLMs within SE is still not well-studied. In this paper, we provide a systematic survey to summarize the current state-of-the-art research in the LLM-based SE community. We summarize 30 representative LLMs of Source Code across three model architectures, 15 pre-training objectives across four categories, and 16 downstream tasks across five categories. We then present a detailed summarization of the recent SE studies for which LLMs are commonly utilized, including 155 studies for 43 specific code-related tasks across four crucial phases within the SE workflow. Besides, we summarize existing attempts to empirically evaluate LLMs in SE, such as benchmarks, empirical studies, and exploration of SE education. We also discuss several critical aspects of optimization and applications of LLMs in SE, such as security attacks, model tuning, and model compression. Finally, we highlight several challenges and potential opportunities on applying LLMs for future SE studies, such as exploring domain LLMs and constructing clean evaluation datasets. Overall, our work can help researchers gain a comprehensive understanding about the achievements of the existing LLM-based SE studies and promote the practical application of these techniques. Our artifacts are publicly available and will continuously updated at the living repository: \url{https://github.com/iSEngLab/AwesomeLLM4SE}.

The Current Challenges of Software Engineering in the Era of Large Language Models

Large Language Models for Software Engineering: Survey and Open Problems

Software Service Engineering in the Era of Large Language Models

A Survey on Large Language Models for Software Engineering

Large Language Models for Software Engineering: A Systematic Literature Review

Towards an Understanding of Large Language Models in Software Engineering Tasks

A systematic study in intelligent software engineering based on Large Language Model

Large Language Model-Based Agents for Software Engineering: A Survey

From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Future

A Software Engineering Perspective on Testing Large Language Models: Research, Practice, Tools and Benchmarks

Prompting Large Language Models to Tackle the Full Software Development Lifecycle: A Case Study

A Survey: Collaborative Hardware and Software Design in the Era of Large Language Models

Breaking the Silence: the Threats of Using LLMs in Software Engineering

Software Testing with Large Language Models: Survey, Landscape, and Vision

Efficient and Green Large Language Models for Software Engineering: Literature Review, Vision, and the Road Ahead

Systems Engineering Issues for Industry Applications of Large Language Model

A Survey on Large Language Models for Code Generation

LLM-Based Multi-Agent Systems for Software Engineering: Literature Review, Vision and the Road Ahead

The Systems Engineering Approach in Times of Large Language Models

Impact of Large Language Models on Generating Software Specifications