Abstract:Frequent modifications of unit test cases are inevitable due to software's continuous underlying changes in source code, design, and requirements. Since manually maintaining software test suites is tedious, timely, and costly, automating the process of generation and maintenance of test units will significantly impact the effectiveness and efficiency of software testing processes. To this end, we propose an automated approach which exploits both structural and semantic properties of source code methods and test cases to recommend the most relevant and useful unit tests to the developers. The proposed approach initially trains a neural network to transform method-level source code, as well as unit tests, into distributed representations (embedded vectors) while preserving the importance of the structure in the code. Retrieving the semantic and structural properties of a given method, the approach computes cosine similarity between the method's embedding and the previously-embedded training instances. Further, according to the similarity scores between the embedding vectors, the model identifies the closest methods of embedding and the associated unit tests as the most similar recommendations. The results on the Methods2Test dataset showed that, while there is no guarantee to have similar relevant test cases for the group of similar methods, the proposed approach extracts the most similar existing test cases for a given method in the dataset, and evaluations show that recommended test cases decrease the developers' effort to generating expected test cases.

What problem does this paper attempt to address?

The paper aims to address the issue of automating the generation and recommendation of unit test cases in software testing. It is inevitable to frequently modify unit test cases to adapt to the continuous changes in software source code, design, and requirements, but manually maintaining software test suites is both time-consuming and costly. Therefore, automating the generation and maintenance of test units will significantly improve the effectiveness and efficiency of the software testing process. To achieve this goal, researchers have proposed an automated method that utilizes the structural and semantic properties of source code methods and test cases to recommend the most relevant and useful unit tests to developers. Specifically, the method first trains a neural network to convert method-level source code and unit tests into distributed representations (embedding vectors), while preserving the importance of code structure. By capturing the semantic and structural properties of a given method, the method calculates the cosine similarity between the method embedding and its previously embedded training instances. Based on the similarity scores between embedding vectors, the model identifies the closest method embedding and its associated unit test as the most similar recommendation. Experimental results show that while it cannot be guaranteed that a set of similar methods will have relevant test cases, the proposed method is able to extract the most similar existing test cases from the dataset for a given method, and evaluations show that the recommended test cases can reduce the effort required by developers to generate expected test cases. The paper discusses two main research questions: 1. Given structural and semantic code embeddings, can a neural network be trained to recommend a relevant test case for a given method? This evaluates two alternative approaches, one based on the similarity between the given method and previously seen methods, and the other based on the similarity between the given method and pre-embedded test cases. 2. To what extent do the recommended test cases help reduce the effort of developers in maintaining test suites? This is measured by calculating the Levenshtein distance between the recommended test cases and the expected test cases, demonstrating the syntactic similarity of the required test cases. By developing two different variants of applications, researchers further explored whether methods with similar functionalities require semantically and structurally similar unit tests, or whether methods with different functionalities tend to need unit tests related to the data structures in use, such as tests for array or matrix structures. This exploration is crucial for determining the practicality of recommended unit tests.

Test Case Recommendations with Distributed Representation of Code Syntactic Features

Test Case Recommendation Based on Balanced Distance of Test Targets

Test Case Reuse Based on Software Testing Knowledge Graph and Collaborative Filtering Recommendation Algorithm

Identifying Similar Test Cases That Are Specified in Natural Language

HomoTR

MUTCR: Test Case Recommendation via Multi-Level Signature Matching

Test recommendation system based on slicing coverage filtering

Code Recommendation With Natural Language Tags And Other Heterogeneous Data

Combining Word Embedding with Information Retrieval to Recommend Similar Bug Reports

On Learning Meaningful Assert Statements for Unit Test Cases

Improving Code Example Recommendations on Informal Documentation Using BERT and Query-Aware LSH: A Comparative Study

In Defense of Simple Techniques for Neural Network Test Case Selection

Automating modern code review processes with code similarity measurement

SLAMPA: Recommending Code Snippets with Statistical Language Model

Unit Test Case Generation with Transformers and Focal Context

A Collaborative Filtering Recommender System for Test Case Prioritization in Web Applications

A content-based literature recommendation system for datasets to improve data reusability - A case study on Gene Expression Omnibus (GEO) datasets

Test2Vec: An Execution Trace Embedding for Test Case Prioritization

Test case selection and prioritization approach for automated regression testing using ontology and COSMIC measurement

Detection of Semantically Similar Code

A Relation-Based Method Combining Functional and Structural Testing for Test Case Generation