Immersive Operation of a Semi-Autonomous Aerial Platform for Detecting and Mapping Radiation

P. Dayani,N. Orr,A. Thomopoulos,V. Saran,S. Krishnaswamy,E. Zhang,N. Hu,D. McPherson,J. Menke,A. Yang,K. Vetter

DOI: https://doi.org/10.1109/TNS.2021.3122452

2021-03-18

Abstract:Recent advancements in radiation detection and computer vision have enabled small unmanned aerial systems (sUASs) to produce 3D nuclear radiation maps in real-time. Currently these state-of-the-art systems still require two operators: one to pilot the sUAS and another operator to monitor the detected radiation. In this work we present a system that integrates real-time 3D radiation visualization with semi-autonomous sUAS control. Our Virtual Reality interface enables a single operator to define trajectories using waypoints to abstract complex flight control and utilize the semi-autonomous maneuvering capabilities of the sUAS. The interface also displays a fused radiation visualization and environment map, thereby enabling simultaneous remote operation and radiation monitoring by a single operator. This serves as the basis for development of a single system that deploys and autonomously controls fleets of sUASs.

Robotics,Human-Computer Interaction

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is the need for two operators in the operation process of the existing radiation detection and 3D nuclear radiation mapping systems. Specifically:

1. **Limitations of the existing systems**:
- The most advanced systems currently still require two operators: one is responsible for operating the small Unmanned Aerial System (sUAS), and the other is responsible for monitoring the detected radiation.
- This two - operator mode has the following problems:
- There is a lack of stereoscopic depth cues when viewing 3D data on a traditional 2D display, resulting in less intuitive understanding of space by the operator.
- The two operators need to manually synchronize key mission data, which may affect the real - time response speed, especially in time - critical situations.

2. **The proposed new solution**:
- The paper introduces a system that integrates real - time 3D radiation visualization with semi - autonomous sUAS control.
- Through the virtual reality (VR) interface, a single operator can define waypoints to simplify complex flight control and utilize the semi - autonomous maneuvering capabilities of the sUAS.
- The VR interface simultaneously displays the fused radiation visualization and the environmental map, enabling a single operator to perform remote operation and radiation monitoring simultaneously.

3. **Specific improvements**:
- **Enhanced spatial awareness**: The stereoscopic depth cues provided by VR enhance the operator's understanding of 3D data and improve the efficiency of remote sUAS control.
- **Simplified operation process**: A single operator can complete all tasks through the VR interface without relying on a separate pilot or interpreter.
- **Improved safety**: It reduces the need for complex manual synchronization and lowers the risk of misoperation.

In summary, this paper aims to simplify the operation process of radiation detection and 3D nuclear radiation mapping, improve efficiency and safety, and reduce the number of required operators by introducing an integrated VR control system.