Abstract:Few mobile robot developers already test their software on simulated robots in virtual environments or sceneries. However, the majority still shy away from simulation-based test campaigns because it remains challenging to specify and execute suitable testing scenarios, that is, models of the environment and the robots' tasks. Through developer interviews, we identified that managing the enormous variability of testing scenarios is a major barrier to the application of simulation-based testing in robotics. Furthermore, traditional CAD or 3D-modelling tools such as SolidWorks, 3ds Max, or Blender are not suitable for specifying sceneries that vary significantly and serve different testing objectives. For some testing campaigns, it is required that the scenery replicates the dynamic (e.g., opening doors) and static features of real-world environments, whereas for others, simplified scenery is sufficient. Similarly, the task and mission specifications used for simulation-based testing range from simple point-to-point navigation tasks to more elaborate tasks that require advanced deliberation and decision-making. We propose the concept of composable and executable scenarios and associated tooling to support developers in specifying, reusing, and executing scenarios for the simulation-based testing of robotic systems. Our approach differs from traditional approaches in that it offers a means of creating scenarios that allow the addition of new semantics (e.g., dynamic elements such as doors or varying task specifications) to existing models without altering them. Thus, we can systematically construct richer scenarios that remain manageable. We evaluated our approach in a small simulation-based testing campaign, with scenarios defined around the navigation stack of a mobile robot. The scenarios gradually increased in complexity, composing new features into the scenery of previous scenarios. Our evaluation demonstrated how our approach can facilitate the reuse of models and revealed the presence of errors in the configuration of the publicly available navigation stack of our SUT, which had gone unnoticed despite its frequent use.

Symbolic Model-based Design and Generation of Logical Scenarios for Autonomous Vehicles Validation

SceML - A Graphical Modeling Framework for Scenario-based Testing of Autonomous Vehicles

An Open-Source Scenario Architect for Autonomous Vehicles

SOTIF-Compliant Scenario Generation Using Semi-Concrete Scenarios and Parameter Sampling

Automatic Generation of Scenarios for System-level Simulation-based Verification of Autonomous Driving Systems

Scenarios for Development, Test and Validation of Automated Vehicles

Verifying Collision Risk Estimation using Autonomous Driving Scenarios Derived from a Formal Model

From Functional to Logical Scenarios: Detailing a Keyword-Based Scenario Description for Execution in a Simulation Environment

Temporal Logic Formalisation of ISO 34502 Critical Scenarios: Modular Construction with the RSS Safety Distance

Is Scenario Generation Ready for SOTIF? A Systematic Literature Review

Navigating Dimensionality through State Machines in Automotive System Validation

Bridging Data-Driven and Knowledge-Driven Approaches for Safety-Critical Scenario Generation in Automated Vehicle Validation

Specification-Guided Critical Scenario Identification for Automated Driving

Multimodal Large Language Model Driven Scenario Testing for Autonomous Vehicles

Critical Scenario Generation for Developing Trustworthy Autonomy

Composable and executable scenarios for simulation-based testing of mobile robots

A Driver-Vehicle Model for ADS Scenario-based Testing

A Concept for Semi-Automatic Configuration of Sufficiently Valid Simulation Setups for Automated Driving Systems

Critical concrete scenario generation using scenario-based falsification

Functional Testing Scenario Library Generation Framework for Connected and Automated Vehicles

Realistic Corner Case Generation for Autonomous Vehicles with Multimodal Large Language Model