Abstract:A test case consists of two parts: a test input to exercise the program under test and a test oracle to check the correctness of the test execution. A test oracle is often in the form of executable assertions such as in the JUnit testing framework. Manually generated test cases are valuable in exposing program faults in the current program version or regression faults in future program versions. However, manually generated test cases are often insufficient for assuring high software quality. We can then use an existing test-generation tool to generate new test inputs to augment the existing test suite. However, without specifications these automatically generated test inputs often do not have test oracles for exposing faults. In this paper, we have developed an automatic approach and its supporting tool, called Orstra, for augmenting an automatically generated unit-test suite with regression oracle checking. The augmented test suite has an improved capability of guarding against regression faults. In our new approach, Orstra first executes the test suite and collects the class under test's object states exercised by the test suite. On collected object states, Orstra creates assertions for asserting behavior of the object states. On executed observer methods (public methods with non-void returns), Orstra also creates assertions for asserting their return values. Then later when the class is changed, the augmented test suite is executed to check whether assertion violations are reported. We have evaluated Orstra on augmenting automatically generated tests for eleven subjects taken from a variety of sources. The experimental results show that an automatically generated test suite's fault-detection capability can be effectively improved after being augmented by Orstra.

Improving Effectiveness of Automated Software Testing in the Absence of Specifications

Automated Software Testing in the Absence of Specifications

An Adaptive Framework for Test Data Generation from Business Specification

Mutually Enhancing Test Generation and Specification Inference

Exploiting Synergy Between Testing and Inferred Partial Specifications

Improving Generation of Object-Oriented Test Suites by Avoiding Redundant Tests

Automatic identification of common and special object-oriented unit tests.

Improving Failure Detection by Automatically Generating Test Cases Near the Boundaries.

An Empirical Study on Automated Test Generation Tools for Java: Effectiveness and Challenges

Perspectives on Automated Testing of Aspect-Oriented Programs

What Good Are Strong Specifications?

Improving the Efficiency of Software Reliability Demonstration Testing by Introducing Testing Effectiveness

Advances on Improving Automation in Developer Testing.

Tool-assisted Unit-Test Generation and Selection Based on Operational Abstractions

A Model and Implementation of Function Test Automation for Web Applications

Improving students' code correctness and test completeness by informal specifications

Augmenting automatically generated unit-test suites with regression oracle checking

Automatically Identifying Special and Common Unit Tests Based on Inferred Statistical Algebraic Abstractions

Tool-assisted Unit Test Selection Based on Operational Violations

An Empirical Evaluation of Test Suite Reduction for Boolean Specification-Based Testing (short Paper)

Cooperative Software Testing and Analysis：Advances and Challenges