Abstract:A test case consists of two parts: a test input to exercise the program under test and a test oracle to check the correctness of the test execution. A test oracle is often in the form of executable assertions such as in the JUnit testing framework. Manually generated test cases are valuable in exposing program faults in the current program version or regression faults in future program versions. However, manually generated test cases are often insufficient for assuring high software quality. We can then use an existing test-generation tool to generate new test inputs to augment the existing test suite. However, without specifications these automatically generated test inputs often do not have test oracles for exposing faults. In this paper, we have developed an automatic approach and its supporting tool, called Orstra, for augmenting an automatically generated unit-test suite with regression oracle checking. The augmented test suite has an improved capability of guarding against regression faults. In our new approach, Orstra first executes the test suite and collects the class under test's object states exercised by the test suite. On collected object states, Orstra creates assertions for asserting behavior of the object states. On executed observer methods (public methods with non-void returns), Orstra also creates assertions for asserting their return values. Then later when the class is changed, the augmented test suite is executed to check whether assertion violations are reported. We have evaluated Orstra on augmenting automatically generated tests for eleven subjects taken from a variety of sources. The experimental results show that an automatically generated test suite's fault-detection capability can be effectively improved after being augmented by Orstra.

Rostra: a framework for detecting redundant object-oriented unit tests

Improving Generation of Object-Oriented Test Suites by Avoiding Redundant Tests

Detecting Redundant Unit Tests for AspectJ Programs

Symstra: a framework for generating object-oriented unit tests using symbolic execution

Augmenting automatically generated unit-test suites with regression oracle checking

Auto-generation and redundancy reduction of test cases for reactive systems

Tool-assisted Unit-Test Generation and Selection Based on Operational Abstractions

OCAT: object capture-based automated testing.

An Experimental Study of Four Typical Test Suite Reduction Techniques

Automatically Removing Unnecessary Stubbings from Test Suites

An Empirical Comparison of Automated Generation and Classification Techniques for Object-Oriented Unit Testing

Automatic identification of common and special object-oriented unit tests.

Automatic extraction of abstract-object-state machines from unit-test executions.

On-demand test suite reduction

Improving Effectiveness of Automated Software Testing in the Absence of Specifications

A study of relative redundancy in test-suite reduction while retaining or improving fault-localization effectiveness.

Pc-Trt: A Test Case Reuse and Generation Tool to Achieve High Path Coverage for Unit Test

Eliminating Harmful Redundancy for Testing-Based Fault Localization Using Test Suite Reduction: an Experimental Study.

Tool-assisted Unit Test Selection Based on Operational Violations

Automated Behavioral Regression Testing

An Empirical Study on Automated Test Generation Tools for Java: Effectiveness and Challenges