Abstract:Testing and validating Cyber-Physical Systems (CPSs) in the aerospace domain, such as field testing of drone rescue missions, poses challenges due to volatile mission environments, such as weather conditions. While testing processes and methodologies are well established, structured guidance and execution support for field tests are still weak. This paper identifies requirements for field testing of drone missions, and introduces the Field Testing Scenario Management (FiTS) approach for adaptive field testing guidance. FiTS aims to provide sufficient guidance for field testers as a foundation for efficient data collection to facilitate quality assurance and iterative improvement of field tests and CPSs. FiTS shall leverage concepts from scenario-based requirements engineering and Behavior-Driven Development to define structured and reusable test scenarios, with dedicated tasks and responsibilities for role-specific guidance. We evaluate FiTS by (i) applying it to three use cases for a search-and-rescue drone application to demonstrate feasibility and (ii) interviews with experienced drone developers to assess its usefulness and collect further requirements. The study results indicate FiTS to be feasible and useful to facilitate drone field testing and data analysis
What problem does this paper attempt to address?
The problem that this paper attempts to solve is how to provide sufficient guidance to ensure efficient data collection and quality assurance in the field tests of UAV (unmanned aerial vehicle) missions. Specifically, the paper aims to address the following key issues:
1. **Challenges in Field Tests**: In cyber - physical systems (CPSs) in the aerospace field, such as UAV rescue missions, field tests become very challenging due to the volatility of environmental factors (such as weather conditions). Although the test processes and methodologies are relatively mature, the application of structured guidance and support in field tests is still relatively weak.
2. **The Gap between Simulation and Reality**: Although various CPS simulation environments exist, from basic simulations with low - fidelity to 3D simulations with high - fidelity, these simulation environments cannot fully and accurately represent the physical law performance in the real environment, resulting in the "simulation - reality" gap. Therefore, it is necessary to verify the simulation results through field tests and ensure that the system behaves as expected in the real world.
3. **Lack of Structured and Systematic Execution Guidance**: Field tests require thorough planning and verification because they pose significant risks to the participants. For example, in field tests, UAVs need to maintain a safe distance to avoid collisions with other UAVs, human operators, and obstacles. However, current tests are usually ad - hoc, lacking appropriate guidance for testers, and requiring a great deal of internal knowledge.
4. **Insufficient Documentation**: Records of test steps and their successful or failed executions are often insufficient or missing. Traditional tool support usually includes manually created documents and checklists, and these tools have limited ability to explain the feasibility of scenarios and adapt to volatile test conditions.
To solve these problems, the paper introduces the **Field Testing Scenario Management (FiTS)** method, which is a scenario - based adaptive field - test - guidance method. The main objectives of FiTS are:
- To provide sufficient guidance to support efficient field - test data collection.
- To promote quality assurance and iterative improvement of field tests and CPSs.
- To use the concepts of scenario - requirements engineering and behavior - driven development (BDD) to define structured and reusable test scenarios and clearly assign role - specific tasks and responsibilities.
- To support the end - to - end process of test design, management, and analysis, including a feedback mechanism to feed the observed problems back into the quality - assurance and engineering processes.
Through FiTS, researchers hope to solve the above problems and improve the efficiency and reliability of UAV field tests.