Abstract:Detecting and refactoring code smells is challenging, laborious, and sustaining. Although large language models have demonstrated potential in identifying various types of code smells, they also have limitations such as input-output token restrictions, difficulty in accessing repository-level knowledge, and performing dynamic source code analysis. Existing learning-based methods or commercial expert toolsets have advantages in handling complex smells. They can analyze project structures and contextual information in-depth, access global code repositories, and utilize advanced code analysis techniques. However, these toolsets are often designed for specific types and patterns of code smells and can only address fixed smells, lacking flexibility and scalability. To resolve that problem, we propose iSMELL, an ensemble approach that employs various code smell detection toolsets via Mixture of Experts (MoE) architecture for comprehensive code smell detection, and enhances the LLMs with the detection results from expert toolsets for refactoring those identified code smells. First, we train a MoE model that, based on input code vectors, outputs the most suitable expert tool for identifying each type of smell. Then, we select the recommended toolsets for code smell detection and obtain their results. Finally, we equip the prompts with the detection results from the expert toolsets, thereby enhancing the refactoring capability of LLMs for code with existing smells, enabling them to provide different solutions based on the type of smell. We evaluate our approach on detecting and refactoring three classical and complex code smells, i.e., Refused Bequest, God Class, and Feature Envy. The results show that, by adopting seven expert code smell toolsets, iSMELL achieved an average F1 score of 75.17% on code smell detection, outperforming LLMs baselines by an increase of 35.05% in F1 score. We further evaluate the code refactored by the enhanced LLM. The quantitative and human evaluation results show that iSMELL could improve code quality metrics and conduct satisfactory refactoring toward the identified code smells. We believe that our proposed solution could provide new insights into better leveraging LLMs and existing approaches to resolving complex software tasks.

How Propense Are Large Language Models at Producing Code Smells? A Benchmarking Study

Evaluating Large Language Models in Detecting Test Smells

Test smells in LLM-Generated Unit Tests

A Survey on Evaluating Large Language Models in Code Generation Tasks

Large Language Models for Code Analysis: Do LLMs Really Do Their Job?

CodeJudge: Evaluating Code Generation with Large Language Models

Evaluating Large Language Models in Class-Level Code Generation

Benchmarking and Explaining Large Language Model-based Code Generation: A Causality-Centric Approach

Where Do Large Language Models Fail When Generating Code?

A Survey on Large Language Models for Code Generation

L2CEval: Evaluating Language-to-Code Generation Capabilities of Large Language Models

Examination of Code generated by Large Language Models

Helping LLMs Improve Code Generation Using Feedback from Testing and Static Analysis

Benchmarks and Metrics for Evaluations of Code Generation: A Critical Review

An Empirical Study of Code Generation Errors made by Large Language Models

Do Large Language Models Pay Similar Attention Like Human Programmers When Generating Code?

Evaluation of large language models for assessing code maintainability

A Deep Dive into Large Language Model Code Generation Mistakes: What and Why?

CodeJudge-Eval: Can Large Language Models be Good Judges in Code Understanding?

Ismell: Assembling LLMs with Expert Toolsets for Code Smell Detection and Refactoring