Abstract:To assess test quality in timed safety‐critical systems modeled as Timed Automata, we present MUPPAAL, a mutation testing framework that detects and removes useless mutants (equivalent and duplicates). By integrating scalable bisimulation algorithms and fast approximate checks through biased simulation, MUPPAAL reduces the impact of duplicates, which can account for up to 32% of generated mutants. We evaluate MUPPAAL on six systems, showing it achieves up to ten times faster performance while effectively minimizing unnecessary mutant exploration. To assess test quality, mutation testing (MT) creates mutants by injecting artificial faults into the system and evaluates the ability of tests to distinguish these mutants. Tests distinguishing more mutants have also been proven empirically to detect more real faults. MT has been applied to many domains. We focus on MT for timed safety‐critical systems modelled as Timed Automata (TA). While powerful, MT usually yields equivalent and duplicate mutants, the former having the same behaviour as the original system and the latter other mutants. Such useless mutants bring no value, waste execution time and can be difficult to detect. We integrate useless mutant detection and removal strategies in our mutation framework MUPPAAL. MUPPAAL leverages existing equivalence‐avoiding mutation operators and focuses on detecting mutant duplicates using a scalable bisimulation algorithm and a fast approximate one based on biased simulation. We also demonstrate how to design an operator that reduces the occurrence of mutant duplicates. We evaluate MUPPAAL on six systems, demonstrating that (1) mutant duplicates account for up to 32% of all generated mutants, (2) our bisimulation approach scales effectively with these systems and (3) biased simulations further enhance performance. Our heuristic is 10 times faster than bisimulation and limits the exploration to two times the number of exact duplicates compared to up to 10 times for the baseline.

Killing Stubborn Mutants with Symbolic Execution

Test generation via Dynamic Symbolic Execution for mutation testing

EXAMINER-PRO: Testing Arm Emulators Across Different Privileges

MUPPAAL: Efficient Elimination and Reduction of Useless Mutants in Real‐Time Model‐Based Systems

Xemu: An Efficient Qemu Based Binary Mutation Testing Framework For Embedded Software

Leveraging Propagated Infection to Crossfire Mutants

Heuristics for Strongly Killing Mutants in Software Mutation Testing

Mutation-driven Test Case Generation Using Short-lived Concurrent Mutants -- First Results

Divide, Conquer and Verify: Improving Symbolic Execution Performance

Binary Mutation Testing Through Dynamic Translation

Neuro-Symbolic Execution: Augmenting Symbolic Execution with Neural Constraints

Learning from Mutants: Using Code Mutation to Learn and Monitor Invariants of a Cyber-Physical System

Steering Symbolic Execution to Less Traveled Paths

Pbse: Phase-Based Symbolic Execution

MuRS: Mutant Ranking and Suppression using Identifier Templates

Engineering a Formally Verified Automated Bug Finder

Symbolic Execution with Test Cases Generated by Large Language Models

Fuzzing for CPS Mutation Testing

$μ$SE: Mutation-based Evaluation of Security-focused Static Analysis Tools for Android

How to Kill Symbolic Deobfuscation for Free; or Unleashing the Potential of Path-Oriented Protections

Selecting Fault Revealing Mutants