Abstract:Machine Learning (ML) and Artificial intelligence (AI) have increased automation potential within defense applications such as border protection, compound security, and surveillance applications. Advances in low-size weight and power (SWAP) computing platforms and unmanned aerial systems (UAS) have enabled autonomous systems to meet the critical needs of future defense systems. Recent academic advances in deep learning aided computer vision yielding impressive results on object detection and recognition, necessary capabilities to enable autonomy in defense applications. These advances, often open-sourced, enable the opportunistic integration of state-of-the-art (SOTA) algorithms. However, these systems require a large amount of object-relevant data to transfer from general academic domains to more relevant situations. Additionally, UAS systems require costly verification and validation of autonomy logic. These challenges can lead to high costs for both training data generation and costly field autonomy integration and testing activities. To address these challenges, in conjunction with partners, Elbit America has developed a multipurpose synthetic simulation environment capable of generating synthetic training data and prototyping, verifying, and validating autonomous distributed behaviors. We integrated a thermal modeling capability into Unreal Engine to create realistic training data by enabling the real-time simulation of SWIR, MWIR, and LWIR sensors. This radiometrically correct sensor model capability enables the simulation-based training data generation for our object recognition and classification pipeline, called Rapid Algorithm Development and Deployment (RADD). Several drones were instantiated using emulated flight controllers to enable end-to-end autonomy training and development before hardware availability. Herein, we describe an overview of the simulation environment and its relevance to detection, classification, and distributed autonomous decision-making.

Simulation environment for development of quad-copter controls incorporating physical environment in urban setting

Autonomous Flights in Dynamic Environments with Onboard Vision

A Comprehensive Simulation Testbench For Aerial Robot In Dynamic Scenario Using Gazebo-Ros

An Open-source Hardware/Software Architecture and Supporting Simulation Environment to Perform Human FPV Flight Demonstrations for Unmanned Aerial Vehicle Autonomy

A simulation environment for drone cinematography

Development, Implementation, and Experimental Outdoor Evaluation of Quadcopter Controllers for Computationally Limited Embedded Systems

Developing 3D Virtual Safety Risk Terrain for UAS Operations in Complex Urban Environments

Navigating Horizons - A Technical Exploration of Autonomous Drone Guidance

Photorealistic simulation platform for autonomous landing of fixed-wing aircraft

Quad-rotor Flight Simulation in Realistic Atmospheric Conditions

Design and Flight Demonstration of a Quadrotor for Urban Mapping and Target Tracking Research

Indoor Testing and Simulation Platform for Close-distance Visual Inspection of Complex Structures using Micro Quadrotor UAV

Learning to Fly -- a Gym Environment with PyBullet Physics for Reinforcement Learning of Multi-agent Quadcopter Control

Multi-agent collaboration environment simulation

UrbanFly: Uncertainty-Aware Planning for Navigation Amongst High-Rises with Monocular Visual-Inertial SLAM Maps

A Hybrid Controller for Inflight Stability and Maneuverability of an Unmanned Aerial Vehicle in Indoor Terrains

XTDrone: A Customizable Multi-Rotor UAVs Simulation Platform

End-to-End UAV Simulation for Visual SLAM and Navigation

Unified Simulation and Test Platform for Control Systems of Unmanned Vehicles

A Digital Smart City for Emerging Mobility Systems

A Framework for Visually Realistic Multi-robot Simulation in Natural Environment