Abstract:Boundary value analysis (BVA) is a common technique in software testing that uses input values that lie at the boundaries where significant changes in behavior are expected. This approach is widely recognized and used as a natural and effective strategy for testing software. Test coverage is one of the criteria to measure how much the software execution paths are covered by the set of test cases. This paper focuses on evaluating test coverage with respect to BVA by defining a metric called boundary coverage distance (BCD). The BCD metric measures the extent to which a test set covers the boundaries. In addition, based on BCD, we consider the optimal test input generation to minimize BCD under the random testing scheme. We propose three algorithms, each representing a different test input generation strategy, and evaluate their fault detection capabilities through experimental validation. The results indicate that the BCD-based approach has the potential to generate boundary values and improve the effectiveness of software testing.

What problem does this paper attempt to address?

The paper attempts to address the issues of effectiveness and test case generation in software testing through Boundary Value Analysis (BVA). Specifically, the paper focuses on the following aspects: 1. **Measurement of Boundary Coverage**: Traditional boundary value analysis methods face difficulties in evaluating the boundary coverage of test cases, especially when boundary values are defined as continuous values. The paper proposes a new metric—Boundary Coverage Distance (BCD)—to assess the extent to which test cases cover boundary values. 2. **Optimal Test Case Generation**: Based on the BCD metric, the paper proposes three algorithms aimed at generating optimal test cases to minimize BCD. These algorithms optimize under random testing schemes by selecting key test cases to improve testing efficiency and effectiveness. 3. **Experimental Validation**: The paper experimentally validates the effectiveness of these three algorithms in fault detection and compares them with traditional Random Testing (RT), Adaptive Random Testing (ART), and Symbolic Execution Testing. ### Main Contributions - **Definition of Boundary Coverage Distance (BCD)**: A new metric is proposed to evaluate the extent to which test cases cover boundary values. - **Development of BCD-based Test Case Generation Algorithms**: Three algorithms are proposed to generate optimal test cases by minimizing BCD. ### Background and Motivation - **Importance of Software Testing**: Software testing is a critical step to ensure software reliability, primarily involving the generation and execution of test cases to verify software behavior. - **Boundary Value Analysis (BVA)**: BVA is a commonly used testing technique that designs test cases by selecting boundary values, as software defects often occur near the boundaries of the input domain. - **Limitations of Traditional BVA**: Traditional BVA methods typically rely on manual analysis of specifications to identify boundary values, which is time-consuming and error-prone. Additionally, as the number of input variables and dimensions increases, generating comprehensive test cases becomes very complex and resource-intensive. ### Solution - **Boundary Coverage Distance (BCD)**: By defining the BCD metric, the extent to which test cases cover boundary values can be quantified, thus evaluating the quality of test cases. - **Optimal Test Case Generation Algorithms**: Based on the BCD metric, the paper proposes three algorithms that generate and optimize test cases randomly to minimize BCD, thereby improving testing efficiency and effectiveness. ### Experimental Results - **Fault Detection Capability**: Experimental validation shows that the BCD-based test case generation algorithms perform well in fault detection, especially in detecting boundary-related defects. - **Comparison with Other Methods**: Compared to traditional random testing, adaptive random testing, and symbolic execution testing, the BCD-based methods exhibit higher fault detection rates in certain scenarios. Overall, the paper introduces the BCD metric and corresponding test case generation algorithms, providing a new approach to enhance the efficiency and effectiveness of software testing.

Optimal test case generation for boundary value analysis

Improving Failure Detection by Automatically Generating Test Cases Near the Boundaries.

Guided test generation for coverage criteria

AN ALGORITHM FOR AUTOMATICALLY GENERATING BLACK-BOX TEST CASES

B4: Towards Optimal Assessment of Plausible Code Solutions with Plausible Tests

A Bioinspired Test Generation Method Using Discretized and Modified Bat Optimization Algorithm

Optimal Test Suite Generation Methods for Neighbor Factors Combinatorial Testing

MA-BBOB: A Problem Generator for Black-Box Optimization Using Affine Combinations and Shifts

An Investigation of Classification-Based Algorithms for Modified Condition/Decision Coverage Criteria

Generating Optimal Test Set for Neighbor Factors Combinatorial Testing

A new speedy generation algorithm for pairwise testing

Optimal testing resource allocation, and sensitivity analysis in software development

Employment of Multiple Algorithms for Optimal Path-based Test Selection Strategy

Tool-assisted Unit Test Selection Based on Operational Violations

A New Pairwise Covering Test Data Generation Algorithm for the System with Many 2-Level Factors

Test Case Minimization with Quantum Annealers

Uncertainty-Driven Black-Box Test Data Generation

Tool-assisted Unit-Test Generation and Selection Based on Operational Abstractions

Search Based Combinatorial Testing

A boundary identification approach for the feasible space of structural optimization using a virtual sampling technique-based support vector machine