Abstract:In grammar-based testing, the test suites that drive the system under test are typically constructed from a given context-free grammar through a set of derivations that jointly satisfy some coverage criterion. In this paper, we describe and evaluate a new algorithm that instead constructs test suites from a set of valid paths that cover all edges in a labeled directed graph corresponding to an LR-automaton that accepts the language of the grammar. Vertices in this graph correspond to states in the LR-automaton; two vertices are connected by an edge iff the top of the LR-automaton's stack can change from one state to the other, either by shifting a terminal or non-terminal symbol (push edges), or by reducing with a grammar rule (pop edges). The algorithm constructs a unique reduction path for each pop edge in the graph. These reduction paths are recursively embedded into each other, and any unresolved non-terminal push edges are substituted by shortest derivations for the non-terminal symbol. The algorithm can work with different types of LR-automata, including LR(0)- and LR(1)-automata, and can successfully generate a test suite from an LR-graph even if the underlying LR-automaton construction leads to shift/reduce or reduce/reduce conflicts. The algorithm only constructs valid paths over the LR-graphs that correspond to sentences in the language and thus generates only positive tests. We therefore also describe mutations to the positive paths that are guaranteed to generate negative tests without needing any further verification by an oracle. Our algorithm is substantially more efficient than an earlier algorithm that explores LR-graphs with two consecutive breadth-first graph traversals and our experimental evaluation shows that it scales to large production-quality grammars. It is robust against random choices made to resolve ambiguity in the construction of the tests, while the code coverage of the different test suite variants is relatively uniform. Finally, our evaluation shows that the negative test suites constructed by path mutation identify more faults in a set of student grammars than those constructed by rule mutation.

Automatic Grammar-Based Test Generation.

A Dynamic Stochastic Model for Automatic Grammar-Based Test Generation

CodeGen-Test: An Automatic Code Generation Model Integrating Program Test Information

Random Grammar-based Testing for Covering All Non-Terminals

Automated Test Oracle Generation via Denotational Semantics

Automatic Test Data Generation Based on Usage Model and Genetic Algorithm

Automatic Test Data Generation Tool Based on Genetic Simulated Annealing Algorithm

Automatic generating test paper system based on genetic algorithm

Automated Test Case Generation from Input Specification in Natural Language

Grammar-based test suite construction using coverage-directed algorithms over LR-graphs

A regression test case generation method guided with branch probability

Guiding Random Test Generation with Program Analysis.

DiffGen: Automated Unit Test Generation for Regression Testing

A New Approach of Automated Test Case Generation on Extended Finite State Machine

DiffGen: Automated Regression Unit-Test Generation

Intergenerational Test Generation for Natural Language Processing Applications

AUTOMATIC TEST CASE GENERATION BASED ON GENETIC ALGORITHM

An Empirical Study on Automated Test Generation Tools for Java: Effectiveness and Challenges

Regression Test Generation Approach Based on Tree-Structured Analysis

Reggae: Automated Test Generation for Programs Using Complex Regular Expressions.

GRT: an Automated Test Generator Using Orchestrated Program Analysis