Abstract:Software Engineering (SE) is the systematic design, development, and maintenance of software applications, underpinning the digital infrastructure of our modern mainworld. Very recently, the SE community has seen a rapidly increasing number of techniques employing Large Language Models (LLMs) to automate a broad range of SE tasks. Nevertheless, existing information of the applications, effects, and possible limitations of LLMs within SE is still not well-studied. In this paper, we provide a systematic survey to summarize the current state-of-the-art research in the LLM-based SE community. We summarize 30 representative LLMs of Source Code across three model architectures, 15 pre-training objectives across four categories, and 16 downstream tasks across five categories. We then present a detailed summarization of the recent SE studies for which LLMs are commonly utilized, including 155 studies for 43 specific code-related tasks across four crucial phases within the SE workflow. Besides, we summarize existing attempts to empirically evaluate LLMs in SE, such as benchmarks, empirical studies, and exploration of SE education. We also discuss several critical aspects of optimization and applications of LLMs in SE, such as security attacks, model tuning, and model compression. Finally, we highlight several challenges and potential opportunities on applying LLMs for future SE studies, such as exploring domain LLMs and constructing clean evaluation datasets. Overall, our work can help researchers gain a comprehensive understanding about the achievements of the existing LLM-based SE studies and promote the practical application of these techniques. Our artifacts are publicly available and will continuously updated at the living repository: \url{https://github.com/iSEngLab/AwesomeLLM4SE}.

Applications and Implications of Large Language Models in Qualitative Analysis: A New Frontier for Empirical Software Engineering

Large Language Model for Qualitative Research -- A Systematic Mapping Study

Large Language Models in Qualitative Research: Can We Do the Data Justice?

Large Language Models for Software Engineering: Survey and Open Problems

Large Language Models for Software Engineering: A Systematic Literature Review

Can Large Language Models Serve as Data Analysts? A Multi-Agent Assisted Approach for Qualitative Data Analysis

A Framework For Discussing LLMs as Tools for Qualitative Analysis

A Survey on Large Language Models for Software Engineering

Towards an Understanding of Large Language Models in Software Engineering Tasks

An Examination of the Use of Large Language Models to Aid Analysis of Textual Data

Are We Testing or Being Tested? Exploring the Practical Applications of Large Language Models in Software Testing

A Case Study on Test Case Construction with Large Language Models: Unveiling Practical Insights and Challenges

Software Testing with Large Language Models: Survey, Landscape, and Vision

Apprentices to Research Assistants: Advancing Research with Large Language Models

Expanding Horizons in HCI Research Through LLM-Driven Qualitative Analysis

An Empirical Study on Usage and Perceptions of LLMs in a Software Engineering Project

Enhancing qualitative research in psychology with large language models: a methodological exploration and examples of simulations

On the use of Large Language Models in Model-Driven Engineering

A systematic study in intelligent software engineering based on Large Language Model