Abstract:For any software system, concise and meaningful method names are critical for program comprehension and maintenance. However, for various reasons, the method names might be inconsistent with their corresponding implementations. Such inconsistent method names are confusing and misleading, often resulting in incorrect method invocations. To this end, a few intelligent deep learning-based approaches based on neural networks have been proposed to identify such inconsistent method names in the industry. Existing evaluations suggest that the performance of such DL-based approaches is promising. However, the evaluations are conducted with a perfectly balanced dataset where the number of inconsistent method names is exactly equivalent to that of consistent ones. In addition, the construction method of this balanced dataset is flawed, leading to false positives in this dataset. Consequently, the reported performance may not represent their efficiency in the field where most method names are consistent with their corresponding method bodies and only a small part of method names are inconsistent with corresponding method bodies. To this end, in this paper, we conduct an empirical study to assess the state-of-the-art DL-based approaches in the automated identification of inconsistent method names. We first build a new benchmark (dataset) by using both automatic identification from commit history and manual inspection by developers, aiming to reduce the number of false positives. Based on the benchmark, we evaluate five representative DL-based approaches to identifying inconsistent method names (one is retrieval-based and two are generation-based). Our evaluation results suggest that the performance of the evaluated approaches is substantially reduced when we switch from the existing balanced dataset to our new benchmark. Furthermore, to reveal where and why the evaluated approaches work/fail, we conduct quantitative and qualitative analyses of the evaluation results. Our analysis results suggest that the evaluated approaches work well on methods with simple bodies and short names, and retrieval-based approaches are especially good at methods whose names start with popular first sub-tokens. Retrieval-based approaches fail frequently because the adopted method representation technique is not efficient enough. Another possible reason for the failures is their unverified rationale, i.e., two methods with similar bodies should have similar names. Generation-based approaches frequently fail because of inaccurate similarity calculation formulas and immature method name generation techniques. Through the data analysis, we also propose two possible ways for better identifying inconsistent method names by leveraging contrastive learning and LLMs. Overall, our empirical study suggests that the state-of-the-art DL-based approaches in inconsistent method name identification deserve significant improvement before applying them to practical software systems.

Just-In-Time Method Name Updating with Heuristics and Neural Model

Automating Just-In-Time Comment Updating

Properly and Automatically Naming Java Methods: A Machine Learning Based Approach

Just-In-Time Defect Identification and Localization: A Two-Phase Framework.

An intelligent java method name recommendation framework via two-phase neural networks

How are We Detecting Inconsistent Method Names? An Empirical Study from Code Review Perspective

Keywords Guided Method Name Generation

Machine Learning Based Recommendation of Method Names: How Far Are We

A Naming Pattern Based Approach for Method Name Recommendation

Just-In-Time Obsolete Comment Detection and Update.

Deep Learning Based Identification of Inconsistent Method Names: How Far Are We?

Improving Just-In-Time Comment Updating Via AST Edit Sequence

Exploiting Method Names to Improve Code Summarization: A Deliberation Multi-Task Learning Approach.

NameChecker: Detecting Inconsistency Between Method Names and Method Bodies

Learning to Recommend Method Names with Global Context

Predictive Comment Updating with Heuristics and AST-Path-Based Neural Learning: A Two-Phase Approach.

Lightweight global and local contexts guided method name recommendation with prior knowledge

Method name suggestion with hierarchical attention networks.

A Hybrid Code Representation Learning Approach for Predicting Method Names

How Important Are Good Method Names in Neural Code Generation? A Model Robustness Perspective

Pre-Implementation Method Name Prediction for Object-Oriented Programming