Abstract:Few mobile robot developers already test their software on simulated robots in virtual environments or sceneries. However, the majority still shy away from simulation-based test campaigns because it remains challenging to specify and execute suitable testing scenarios, that is, models of the environment and the robots' tasks. Through developer interviews, we identified that managing the enormous variability of testing scenarios is a major barrier to the application of simulation-based testing in robotics. Furthermore, traditional CAD or 3D-modelling tools such as SolidWorks, 3ds Max, or Blender are not suitable for specifying sceneries that vary significantly and serve different testing objectives. For some testing campaigns, it is required that the scenery replicates the dynamic (e.g., opening doors) and static features of real-world environments, whereas for others, simplified scenery is sufficient. Similarly, the task and mission specifications used for simulation-based testing range from simple point-to-point navigation tasks to more elaborate tasks that require advanced deliberation and decision-making. We propose the concept of composable and executable scenarios and associated tooling to support developers in specifying, reusing, and executing scenarios for the simulation-based testing of robotic systems. Our approach differs from traditional approaches in that it offers a means of creating scenarios that allow the addition of new semantics (e.g., dynamic elements such as doors or varying task specifications) to existing models without altering them. Thus, we can systematically construct richer scenarios that remain manageable. We evaluated our approach in a small simulation-based testing campaign, with scenarios defined around the navigation stack of a mobile robot. The scenarios gradually increased in complexity, composing new features into the scenery of previous scenarios. Our evaluation demonstrated how our approach can facilitate the reuse of models and revealed the presence of errors in the configuration of the publicly available navigation stack of our SUT, which had gone unnoticed despite its frequent use.

(Demo) Systematic Experimentation Using Scenarios in Agent Simulation: Going Beyond Parameter Space

Different Facets for Different Experts: A Framework for Streamlining The Integration of Qualitative Insights into ABM Development

A Multi-Agent Simulation of Retail Management Practices

Agent-based Simulation with Netlogo to Evaluate AmI Scenarios

ABIDES-Economist: Agent-Based Simulation of Economic Systems with Learning Agents

Automated and Distributed Statistical Analysis of Economic Agent-Based Models

Smart Agent-Based Modeling: On the Use of Large Language Models in Computer Simulations

Interventionally Consistent Surrogates for Agent-based Simulators

Tutorial on agent-based modeling and simulation

Agent cognition through micro-simulations: Adaptive and tunable intelligence with NetLogo LevelSpace

Surrogate Assisted Methods for the Parameterisation of Agent-Based Models

Exploring Gameplay With AI Agents

Multi-Agent Simulation and Management Practices

An empirically grounded agent based simulator for the air traffic management in the SESAR scenario

ABIDES: Towards High-Fidelity Market Simulation for AI Research

Exploring the Intersection of Large Language Models and Agent-Based Modeling via Prompt Engineering

Autoformalizing and Simulating Game-Theoretic Scenarios using LLM-augmented Agents

Quantitatively Assessing the Benefits of Model-driven Development in Agent-based Modeling and Simulation

AgentSimulator: An Agent-based Approach for Data-driven Business Process Simulation

Learning Complex Spatial Behaviours in ABM: An Experimental Observational Study

Composable and executable scenarios for simulation-based testing of mobile robots