Machine Learning and Evolutionary Computing for GUI-based Regression Testing

Daniel Kraus

DOI: https://doi.org/10.48550/arXiv.1802.03768

2018-02-12

Abstract:ReTest is a novel testing tool for Java applications with a graphical user interface (GUI), combining monkey testing and difference testing. Since this combination sidesteps the oracle problem, it enables the generation of GUI-based regression tests. ReTest makes use of evolutionary computing (EC), particularly a genetic algorithm (GA), to optimize these tests towards code coverage. While this is indeed a desirable goal in terms of software testing and potentially finds many bugs, it lacks one major ingredient: human behavior. Consequently, human testers often find the results less reasonable and difficult to interpret. This thesis proposes a new approach to improve the initial population of the GA with the aid of machine learning (ML), forming an ML-technique enhanced-EC (MLEC) algorithm. In order to do so, existing tests are exploited to extract information on how human testers use the given GUI. The obtained data is then utilized to train an artificial neural network (ANN), which ranks the available GUI actions respectively their underlying GUI components at runtime---reducing the gap between manually created and automatically generated regression tests. Although the approach is implemented on top of ReTest, it can be easily used to guide any form of monkey testing. The results show that with only little training data, the ANN is able to reach an accuracy of 82% and the resulting tests represent an improvement without reducing the overall code coverage and performance significantly.

Software Engineering

What problem does this paper attempt to address?

The problem this paper attempts to address is how to enhance ReTest's genetic algorithm (GA) through machine learning (ML) techniques to generate GUI regression tests that better align with human behavior patterns. Specifically, while existing genetic algorithm-based automated testing methods can optimize code coverage, the generated test cases often lack the rationality and interpretability of human testers. Therefore, this paper proposes a new approach that uses artificial neural networks (ANN) to train on existing test data, thereby ranking GUI actions and their underlying components at runtime to reduce the gap between automatically generated regression tests and manually created ones.

### Main Issues:
1. **Limitations of Existing Methods**: Although existing genetic algorithm-based automated testing methods can effectively improve code coverage, the generated test cases lack the rationality and interpretability of human testers.
2. **Improvement Goals**: By introducing machine learning techniques, particularly artificial neural networks (ANN), to enhance the genetic algorithm (GA), the generated test cases can better align with human behavior patterns, thereby improving the rationality and interpretability of the tests.

### Solution:
1. **Feature Extraction**: Extract features from existing test data, including absolute and relative features, to train the artificial neural network.
2. **Model Training**: Use the extracted features to train an artificial neural network that can rank GUI actions and their underlying components at runtime.
3. **Integration and Evaluation**: Integrate the trained artificial neural network into ReTest and evaluate its performance through experiments to ensure that the generated test cases better align with human behavior patterns without significantly reducing code coverage and performance.

### Objectives:
- **Improve the Rationality of Test Cases**: The generated test cases should better align with the logic and behavior patterns of human testers.
- **Maintain or Improve Code Coverage**: While improving the rationality of test cases, do not significantly reduce code coverage and performance.
- **Build a Prototype System**: Design and implement a prototype system to validate the effectiveness of the proposed ML-enhanced evolutionary computation (MLEC) algorithm.

Through these methods, the paper aims to address the issue of the lack of human behavior characteristics in test cases generated by existing automated testing methods, thereby improving the rationality and interpretability of the tests.