Test-data generation guided by static defect detection

HaoDan,ZhangLu,LiuMing-Hao,LiHe,SunJia-Su

IF: 1.871

2009-01-01

Journal of Computer Science and Technology

Abstract:Software testing is an important technique to assure the quality of software systems, especially high-confidence systems. To automate the process of software testing, many automatic test-data gener...

What problem does this paper attempt to address?

Test-Data Generation Guided by Static Defect Detection

Dan Hao,Lu Zhang,Ming-Hao Liu,He Li,Jia-Su Sun

DOI: https://doi.org/10.1007/s11390-009-9224-5

IF: 1.871

2009-01-01

Journal of Computer Science and Technology

Abstract:Software testing is an important technique to assure the quality of software systems, especially high-confidence systems. To automate the process of software testing, many automatic test-data generation techniques have been proposed. To generate effective test data, we propose a test-data generation technique guided by static defect detection in this paper. Using static defect detection analysis, our approach first identifies a set of suspicious statements which are likely to contain faults, then generates test data to cover these suspicious statements by converting the problem of test-data generation to the constraint satisfaction problem. We performed a case study to validate the effectiveness of our approach, and made a simple comparison with another test-data generation on-line tool, JUnit Factory. The results show that, compared with JUnit Factory, our approach generates fewer test data that are competitive on fault detection.
An Adaptive Framework for Test Data Generation from Business Specification

Fan Wang,Xiaohu Yang,Xiaochun Zhu

DOI: https://doi.org/10.1109/ITCS.2009.126

2009-01-01

Abstract:In this paper a framework for test data generation from business specification is presented. This framework separates data definition from their implementation and enables user to define logic and manage test data in a structural way. Data generation logic can be maintained and reused in different test scenarios. Test data with different format will be generated to integrate with other test automation tools. The internal mechanism is introduced and enhancement point is discussed then.
Automatic Early Defects Detection in Use Case Documents

Shuang Liu,Jun Sun,Yang Liu,Yue Zhang,Bimlesh Wadhwa,Jin Song Dong,Xinyu Wang

DOI: https://doi.org/10.1145/2642937.2642969

2014-01-01

Abstract:Use cases, as the primary techniques in the user requirement analysis, have been widely adopted in the requirement engineering practice. As developed early, use cases also serve as the basis for function requirement development, system design and testing. Errors in the use cases could potentially lead to problems in the system design or implementation. It is thus highly desirable to detect errors in use cases. Automatically analyzing use case documents is challenging primarily because they are written in natural languages. In this work, we aim to achieve automatic defect detection in use case documents by leveraging on advanced parsing techniques. In our approach, we first parse the use case document using dependency parsing techniques. The parsing results of each use case are further processed to form an activity diagram. Lastly, we perform defect detection on the activity diagrams. To evaluate our approach, we have conducted experiments on 200+ real-world as well as academic use cases. The results show the effectiveness of our method.
A Test Automation Solution On Gui Functional Test

zhu xiaochun,zhou bo,li juefeng,gao qiu

DOI: https://doi.org/10.1109/INDIN.2008.4618325

2008-01-01

Abstract:Although it is widely believed that software quality will be improved by the use of automated testing, automation Is still not well-off in industry today. There are quite a few issues of traditional software test automation mode, for example, high cost, knowledge barriers, and management troubles.In the paper, a test automation solution on GUI functional test is proposed. The solution integrates test case generation and selection, test case execution, and test reporting to facilitate testing. It introduces the concept of test driver which is designed to take over the communication between test cases and the execution engine. At last, some enhancements are proposed for future work.
Research on computer static software defect detection system based on big data technology

Zhaoxia Li,Jianxing Zhu,K. Arumugam,Jyoti Bhola,Rahul Neware

DOI: https://doi.org/10.1515/jisys-2021-0260

2022-01-01

Journal of Intelligent Systems

Abstract:Abstract To study the static software defect detection system, based on the traditional static software defect detection system design, a new static software defect detection system design based on big data technology is proposed. The proposed method can optimize the distribution of test resources and improve the quality of software products by predicting the potential defect program modules and design the software and hardware of the static software defect detection system of big data technology. It is found that the traditional static software defect detection system design based on code source data takes a long time, averaging 65 h /day. However, the traditional static software defect detection system based on deep learning has a short detection time, averaging 35 h/day. In this article, the detection time of the static software defect detection system based on big data is shorter than that of the other two traditional system designs, with an average of 15 h/day. Because the system design adjusts the operating state of the system, it improves the accuracy of data operation. On the premise of data collection, the system inspection research is completed, which ensures the operational safety of software data, alleviates the contradiction between system and data to a high degree, improves the efficiency of system operation, reduces unnecessary operations, further shortens the time required for inspection, improves the system performance, and has higher research and operation value.
Dyta: Dynamic Symbolic Execution Guided With Static Verification Results

Xi Ge,Kunal Taneja,Tao Xie,Nikolai Tillmann

DOI: https://doi.org/10.1145/1985793.1985971

2011-01-01

Abstract:Software-defect detection is an increasingly important research topic in software engineering. To detect defects in a program, static verification and dynamic test generation are two important proposed techniques. However, both of these techniques face their respective issues. Static verification produces false positives, and on the other hand, dynamic test generation is often time consuming. To address the limitations of static verification and dynamic test generation, we present an automated defect-detection tool, called DyTa, that combines both static verification and dynamic test generation. DyTa consists of a static phase and a dynamic phase. The static phase detects potential defects with a static checker; the dynamic phase generates test inputs through dynamic symbolic execution to confirm these potential defects. DyTa reduces the number of false positives compared to static verification and performs more efficiently compared to dynamic test generation.
Improving Defect Detection Ability of Derived Test Cases Based on Mutated UML Activity Diagrams

Haiying Sun,Mingsong Chen,Min Zhang,Jing Liu,Ying Zhang

DOI: https://doi.org/10.1109/COMPSAC.2016.136

2016-01-01

Abstract:Structure coverage driven test generation is the key approach for automatic testing at source code level. However, the defect detection ability of the generated test cases should be carefully evaluated since the correlation between coverage and test effectiveness is in doubt. In this paper, we propose a test generation approach based on mutation testing with the intent to derive test cases towards finding defects rather than just covering certain syntactic structures. Moreover, instead of generating test cases from the code under test directly, we base our approach on UML activity diagrams to make it possible to decide verdicts of test inputs. Mutation operators for activity diagrams are defined and the test generation algorithms are based on solving mutated path constraints. Experimental results have shown that by applying the proposed mutated testing approach, test cases with higher defect detection ability can be generated.
Enhancing Defect Prediction with Static Defect Analysis.

Hao Tang,Tian Lan,Dan Hao,Lu Zhang

DOI: https://doi.org/10.1145/2875913.2875922

2015-01-01

Abstract:In the software development process, how to develop better software at lower cost has been a major issue of concern. One way that helps is to find more defects as early as possible, on which defect prediction can provide effective guidance. The most popular defect prediction technique is to build defect prediction models based on machine learning. To improve the performance of defect prediction model, selecting appropriate features is critical. On the other hand, static analysis is usually used in defect detection. As static defect analyzers detects defects by matching some well-defined "defect patterns", its result is useful for locating defects. However, defect prediction and static defect analysis are supposed to be two parallel areas due to the differences in research motivation, solution and granularity. In this paper, we present a possible approach to improve the performance of defect prediction with the help of static analysis techniques. Specifically, we present to extract features based on defect patterns from static defect analyzers to improve the performance of defect prediction models. Based on this approach, we implemented a defect prediction tool and set up experiments to measure the effect of the features.
Hybrid Test Data Generation

Zicong Liu,Zhenyu Chen,Chunrong Fang,Qingkai Shi

DOI: https://doi.org/10.1145/2591062.2591150

2014-01-01

Abstract:Many automatic test data generation techniques have been proposed in the past decades. Each technique can only deal with very restrictive data types so far. This limits the usefulness of test data generation in practice. We present a preliminary approach on hybrid test data generation, by combining Random Strategy (RS), Dynamic Symbolic Execution (DSE), and Search-based Strategy (SBS). It is expected to take advantage of the state-of-the-arts to enhance the robustness and scalability, in terms of different types of test data.
Automatic Test Data Generation Based on Usage Model and Genetic Algorithm

Xian-zhong ZHOU,Yong-cheng SUN,Jin-long JIANG

DOI: https://doi.org/10.3321/j.issn:1000-1093.2006.06.022

2006-01-01

Abstract:Automatic generation technology of test data is an important field of software test, which is an effective method to promote the efficiency and the effect of software test. In order to ensure that the more frequently used features would be tested more thoroughly, usage model and genetic algorithm were integrated together. One programming model was proposed, in which the optimal objective function was defined as the required test data number for whole software test. The methods to decide the weight of path and to select the target path set were presented. Three key elements, i.e. path weight, path coverage ratio and path approximation level were incorporated for designing the fitness function. An example shows the rationality and validity of the model and the methods.
What You See Is What You Get: Attention-based Self-guided Automatic Unit Test Generation

Xin Yin,Chao Ni,Xiaodan Xu,Xiaohu Yang

2024-12-01

Abstract:Software defects heavily affect software's functionalities and may cause huge losses. Recently, many AI-based approaches have been proposed to detect defects, which can be divided into two categories: software defect prediction and automatic unit test generation. While these approaches have made great progress in software defect detection, they still have several limitations in practical application, including the low confidence of prediction models and the inefficiency of unit testing models. To address these limitations, we propose a WYSIWYG (i.e., What You See Is What You Get) approach: Attention-based Self-guided Automatic Unit Test GenERation (AUGER), which contains two stages: defect detection and error triggering. In the former stage, AUGER first detects the proneness of defects. Then, in the latter stage, it guides to generate unit tests for triggering such an error with the help of critical information obtained by the former stage. To evaluate the effectiveness of AUGER, we conduct a large-scale experiment by comparing with the state-of-the-art (SOTA) approaches on the widely used datasets (i.e., Bears, <a class="link-external link-http" href="http://Bugs.jar" rel="external noopener nofollow">this http URL</a>, and Defects4J). AUGER makes great improvements by 4.7% to 35.3% and 17.7% to 40.4% in terms of F1-score and Precision in defect detection, and can trigger 23 to 84 more errors than SOTAs in unit test generation. Besides, we also conduct a further study to verify the generalization in practical usage by collecting a new dataset from real-world projects.

Software Engineering
Development and Application of Quality Assurance and Rating Software Program for Power Equipment Defect Texts

LI Yanru,WANG Huifang,CHEN Hao,ZHANG Jiali,JIANG Fan,XIE Yawen

DOI: https://doi.org/10.19585/j.zjdl.202207011

2022-01-01

Abstract:In view of the lack of intelligent tools for the quality assurance and rating of power equipment defect text， the paper proposes a quality assurance and rating software program for power equipment defect text based on standards and historical defects. Firstly， it analyzes the software requirements and determines that the software needs to analyze text quality and automatically rate the defects. Then， according to the characteristics of the two bases， two application processes are designed. Then， the detailed program design process of key functional modules is introduced. The standard-based text quality analysis and automatic defect rating adopt standard tree matching methods. The text quality analysis and automatic defect rating based on history defects adopt graph retrieval methods based on knowledge graph （KG）. Finally， the operation effect of the software is demonstrated by examples. The results show that the software realizes the intelligent quality control and automatic rating of the input defect text by interactive means， which renders powerful support for the closed-loop management of power equipment defects and the improvement of the intelligence level of operation and inspection.
Study of Test Case Generation for Web Applications

PENG Shu-shen,GU Qing,CHEN Dao-xu

DOI: https://doi.org/10.3969/j.issn.1002-137X.2010.06.037

2010-01-01

Computer Science

Abstract:Along with the fast evolution of Internet,Web application development has become one of the main categories of software application development,and Web application programs have become more and more complicated.Improving the quality of Web applications from different aspects is inevitable task of Web application developers.An important way of ensuring the quality of Web applications is software testing,the core work of which is test case generation.In this paper,we classified the methods for Web test case generation into four categories:Capture/Replay,HTML analysis,Source code analysis,and User-Session analysis.We discussed these four methods in detail,and concluded general steps of generating test cases in these methods.Finally,we listed the advantages and disadvantages of the four method categories,and compared their performance from various perspectives,including the effort needed to implement the method,maximum attainable code coverage,and so on.
Automatic Testing Data Generation in the Testing Based on EFSM

张涌,钱乐秋,王渊峰

DOI: https://doi.org/10.3321/j.issn:0254-4164.2003.10.012

2003-01-01

Jisuanji Xuebao/Chinese Journal of Computers

Abstract:We propose an automatic testing data generation method in the testing based on EFSM. In this method, we use two key phases, interval narrowing and subsection gradient optimal descent algorithms to generate testing data automatically. In the interval-narrowing phase, we design a group of interval narrowing operators to reduce the variables' value scope. If the precondition in the state transitions is simple, we can get the testing data in this phase. In the secondphase, we use subsection gradient optimal descent algorithms to generate testing data from reduced variables' value scope. The experiment shows that we can generate most of testing data automatically using this method, and has speedy convergence. In the situation of having no solution of input variables, interval narrowing can provide small intervals of the value of input variables for tester, which would be convenient for tester to select testing data from those small intervals.
Defect Prediction Guided Search-Based Software Testing

Anjana Perera,Aldeida Aleti,Marcel Böhme,Burak Turhan

DOI: https://doi.org/10.48550/arXiv.2109.12645

2021-09-26

Software Engineering

Abstract:Today, most automated test generators, such as search-based software testing (SBST) techniques focus on achieving high code coverage. However, high code coverage is not sufficient to maximise the number of bugs found, especially when given a limited testing budget. In this paper, we propose an automated test generation technique that is also guided by the estimated degree of defectiveness of the source code. Parts of the code that are likely to be more defective receive more testing budget than the less defective parts. To measure the degree of defectiveness, we leverage Schwa, a notable defect prediction technique. We implement our approach into EvoSuite, a state of the art SBST tool for Java. Our experiments on the Defects4J benchmark demonstrate the improved efficiency of defect prediction guided test generation and confirm our hypothesis that spending more time budget on likely defective parts increases the number of bugs found in the same time budget.
A Method of Automated Test Data Generation for Web Service

JIANG Ying,XIN Guo-Mao,SHAN Jin-Hui,ZHANG Lu,XIE Bing,YANG Fu-Qing

DOI: https://doi.org/10.3321/j.issn:0254-4164.2005.04.015

2005-01-01

Chinese Journal of Computers

Abstract:Software testing is one of the most important techniques used to assure the quality of Web Services at present. Test data generation is an important topic in Web Services testing. The quality of test data will influence the efficiency and cost during Web Services testing. Based on the design by contract testing technology for Web Services, this paper presents a method of automated test data generation for Web Services. Firstly, according to the description information and contracts in WSDL document of Web Services, the initial test data are generated automatically by random method. Then the test data are selected by means of contract mutation testing. The algorithms about initial test data generation and test data selection are presented. In order to improve the quality and efficiency of Web Services testing, this method can generate a test suite meeting a certain contract mutation score. Finally, a prototype has been developed on the Microsoft. NET platform. For the same Web Service with different contracts, the experiments are carried out on this prototype. The contract mutation score of generated test data is compared with the statement coverage score and condition coverage score. The results have shown that the method is effective in automated test data generation for Web Services.
Design and Implement of a Tool for Pairwise Testing Data Generation

ZHOU Xiao-Yu,NIE Chang-Hai,XU Bao-Wen,CHEN Huo-Wang

2005-01-01

Computer Science

Abstract:Pairwise testing is practical and effective for various types of software systems,and it is a key problem that the test data generation method for the pairwise testing. This paper introduces the design and implement of a tool for the pairwise test data generation,which is based on the foundation of the former and our recent research results. The experiments in using this tool show that our tool has good properties and merits compared with the related tool.
Mutation-Based Simulation Test Data Generation for Testing Complex Real-Time Software

Xiaoying Bai,Shufang Lee,Yinong Chen

DOI: https://doi.org/10.1109/anss.2007.30

2007-01-01

Abstract:Simulation is proven to be an effective way for testing complex real-time hardware/software systems. However, simulation test data generation is a challenging issue which determines the efficiency in such testing. This paper proposed a data-driven mutation based approach for test data generation to address the unique requirements of realtime, input domain coverage, adaptability, and reliability. The architecture is designed to simulate the process for data sampling, transforming, packaging, and transmitting segments in an aerospace ground control system. Various mutant operators are defined for signal generation and data package generation to produce fault-sensitive inputs based on the system's fault pattern analysis. Mechanisms are defined to generate a large number of test cases by combination and composition of various data outputs from different dimensions to achieve a high coverage. A configuration mechanism is introduced to enable re-composition and re-combination of simulation components. The selective simulation testing method is discussed to improve test effectiveness. A case study shows that the proposed approach can achieve a high fault-coverage with a small number of effective test cases
Accelerating Software Quality: Unleashing the Power of Generative AI for Automated Test-Case Generation and Bug Identification

Yatin Bajaj,Manoj Kumar Samal

DOI: https://doi.org/10.22214/ijraset.2023.54628

2023-07-31

International Journal for Research in Applied Science and Engineering Technology

Abstract:Abstract: This paper explores the benefits, challenges, and real-world applications of automated test-case generation and bug identification using generative artificial intelligence (AI). In today's software development and testing landscape, ensuring code quality and minimizing bugs are crucial. However, manual testing methods can be time-consuming and error-prone. Generative AI algorithms, such as generative models, can automatically generate test cases based on inputs, specifications, or system behavior. These algorithms employ machine learning techniques to analyze codebases, uncover test scenarios, and produce comprehensive test cases with broad coverage. The advantages of automated test-case generation include amplified test coverage, supercharged efficiency and time savings, and seamless scalability. Generative AI models also excel in bug identification by scrutinizing codebases, execution traces, and test results. They can detect coding mistakes and identify anomalous patterns indicating potential bugs, memory leaks, or security vulnerabilities. However, challenges such as data quality and bias, domain specificity, and the need for human expertise must be addressed. Real-world applications of automated test-case generation and bug identification using generative AI include software development and security testing. By leveraging generative AI, organizations can enhance test coverage, improve efficiency, and ensure the quality of software products. To successfully implement this approach, challenges related to data quality, domain specificity, and human expertise must be navigated. Generative AI has the potential to revolutionize software testing and contribute to the development of robust and reliable systems in the complex digital landscape.
Can Defects Be Fixed with Weak Test Suites? an Analysis of 50 Defects from Defects4J

Jiajun Jiang,Yingfei Xiong

DOI: https://doi.org/10.48550/arxiv.1705.04149

2017-01-01

Abstract:Automated program repair techniques, which target to generating correct patches for real world defects automatically, have gained a lot of attention in the last decade. Many different techniques and tools have been proposed and developed. However, even the most sophisticated program repair techniques can only repair a small portion of defects while producing a lot of incorrect patches. A possible reason for this low performance is that the test suites of real world programs are usually too weak to guarantee the behavior of the program. To understand to what extent defects can be fixed with weak test suites, we analyzed 50 real world defects from Defects4J, in which we found that up to 84% of them could be correctly fixed. This result suggests that there is plenty of space for current automated program repair techniques to improve. Furthermore, we summarized seven fault localization strategies and seven patch generation strategies that were useful in localizing and fixing these defects, and compared those strategies with current repair techniques. The results indicate potential directions to improve automatic program repair in the future research.

Test-data generation guided by static defect detection

Test-Data Generation Guided by Static Defect Detection

An Adaptive Framework for Test Data Generation from Business Specification

Automatic Early Defects Detection in Use Case Documents

A Test Automation Solution On Gui Functional Test

Research on computer static software defect detection system based on big data technology

Dyta: Dynamic Symbolic Execution Guided With Static Verification Results

Improving Defect Detection Ability of Derived Test Cases Based on Mutated UML Activity Diagrams

Enhancing Defect Prediction with Static Defect Analysis.

Hybrid Test Data Generation

Automatic Test Data Generation Based on Usage Model and Genetic Algorithm

What You See Is What You Get: Attention-based Self-guided Automatic Unit Test Generation

Development and Application of Quality Assurance and Rating Software Program for Power Equipment Defect Texts

Study of Test Case Generation for Web Applications

Automatic Testing Data Generation in the Testing Based on EFSM

Defect Prediction Guided Search-Based Software Testing

A Method of Automated Test Data Generation for Web Service

Design and Implement of a Tool for Pairwise Testing Data Generation

Mutation-Based Simulation Test Data Generation for Testing Complex Real-Time Software

Accelerating Software Quality: Unleashing the Power of Generative AI for Automated Test-Case Generation and Bug Identification

Can Defects Be Fixed with Weak Test Suites? an Analysis of 50 Defects from Defects4J