OSU-Wing PIC Phase I Evaluation: Baseline Workload and Situation Awareness Results

Julie A. Adams,Christopher A. Sanchez,Vivek Mallampati,Joshua Bhagat Smith,Emily Burgess,Andrew Dassonville

2024-11-28

Abstract:The common theory is that human pilot's performance degrades when responsible for an increased number of uncrewed aircraft systems (UAS). This theory was developed in the early 2010's for ground robots and not highly autonomous UAS. It has been shown that increasing autonomy can mitigate some performance impacts associated with increasing the number of UAS. Overall, the Oregon State University-Wing collaboration seeks to understand what factors negatively impact a pilot's ability to maintain responsibility and control over an assigned set of active UAS. The Phase I evaluation establishes baseline data focused on the number of UAS and the number of nests increase. This evaluation focuses on nominal operations as well as crewed aircraft encounters and adverse weather changes. The results demonstrate that the pilots were actively engaged and had very good situation awareness. Manipulation of the conditions did not result in any significant differences in overall workload. The overall results debunk the theory that increasing the number of UAS is detrimental to pilot's performance.

Human-Computer Interaction,Robotics

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: **Will increasing the number of Unmanned Aerial Systems (UAS) and the introduction of unexpected events have a negative impact on the performance of the Pilot In Command (PIC)**, especially in terms of workload, situation awareness, and attention focus?

Specifically, the research background is based on a common theory that as the number of unmanned systems in charge increases, the performance of human pilots will decline. However, this theory was mainly proposed for ground robots in the early 2010s and is not fully applicable to highly autonomous UAS. Recent research has shown that increasing the degree of automation can, to a certain extent, alleviate the performance impact caused by the increase in the number of UAS. For example, drone light shows and Wing's delivery drone system have demonstrated reliable automation capabilities and the ability of pilots to manage multiple UAS simultaneously.

To verify these hypotheses, Oregon State University (OSU) and Wing Company cooperated to conduct a two - phase user evaluation process. **The main purpose of Phase I evaluation is to establish baseline data**, which will be used for the subsequent Phase II evaluation to identify and manipulate factors that may affect pilot performance. The focus of Phase I evaluation is:

- **Nominal operating conditions**: Conduct routine operations without unexpected events.
- **Single - manned aircraft encounter**: Simulate the situation where the pilot needs to monitor and respond to a single - manned aircraft.
- **Two - manned aircraft consecutive encounters**: Simulate the situation where two - manned aircraft appear simultaneously.
- **Adverse weather changes**: Simulate the sudden deterioration of weather conditions.

Through these changes in conditions, researchers hope to understand the impact of different factors on pilots' workload, situation awareness, and attention. The experimental design includes different numbers of UAS (low and high), the number of nests (10 or 24), and whether there are unexpected events. Combinations of all these variables form three test conditions: 10:Low, 24:Low, and 24:High.

In summary, the core question of this paper is: **In a highly automated environment, when pilots manage multiple UAS, what factors will affect their performance?** Through Phase I evaluation, researchers hope to provide baseline data for Phase II evaluation, so as to better understand how to optimize the operational safety and efficiency of pilots.