Abstract:Machine Learning (ML) and Artificial intelligence (AI) have increased automation potential within defense applications such as border protection, compound security, and surveillance applications. Advances in low-size weight and power (SWAP) computing platforms and unmanned aerial systems (UAS) have enabled autonomous systems to meet the critical needs of future defense systems. Recent academic advances in deep learning aided computer vision yielding impressive results on object detection and recognition, necessary capabilities to enable autonomy in defense applications. These advances, often open-sourced, enable the opportunistic integration of state-of-the-art (SOTA) algorithms. However, these systems require a large amount of object-relevant data to transfer from general academic domains to more relevant situations. Additionally, UAS systems require costly verification and validation of autonomy logic. These challenges can lead to high costs for both training data generation and costly field autonomy integration and testing activities. To address these challenges, in conjunction with partners, Elbit America has developed a multipurpose synthetic simulation environment capable of generating synthetic training data and prototyping, verifying, and validating autonomous distributed behaviors. We integrated a thermal modeling capability into Unreal Engine to create realistic training data by enabling the real-time simulation of SWIR, MWIR, and LWIR sensors. This radiometrically correct sensor model capability enables the simulation-based training data generation for our object recognition and classification pipeline, called Rapid Algorithm Development and Deployment (RADD). Several drones were instantiated using emulated flight controllers to enable end-to-end autonomy training and development before hardware availability. Herein, we describe an overview of the simulation environment and its relevance to detection, classification, and distributed autonomous decision-making.

Human-in-the-loop for Bayesian autonomous materials phase mapping

Human-In-the-Loop for Bayesian Autonomous Materials Phase Mapping

A Haptic Shared Control Architecture for Tracking of a Moving Object.

On-the-fly Closed-loop Autonomous Materials Discovery via Bayesian Active Learning

Physics in the Machine: Integrating Physical Knowledge in Autonomous Phase-Mapping

Closed-Loop Bayesian Semantic Data Fusion for Collaborative Human-Autonomy Target Search

On-the-fly closed-loop materials discovery via Bayesian active learning

HARPS: An Online POMDP Framework for Human-Assisted Robotic Planning and Sensing

Scalable Multi-Agent Lab Framework for Lab Optimization

Real-time experiment-theory closed-loop interaction for autonomous materials science

MatPilot: an LLM-enabled AI Materials Scientist under the Framework of Human-Machine Collaboration

A Bayesian experimental autonomous researcher for mechanical design

Bayesian Active Learning for Scanning Probe Microscopy: from Gaussian Processes to Hypothesis Learning

Multi-agent collaboration environment simulation

Problem-fluent models for complex decision-making in autonomous materials research

Advances in Kriging-Based Autonomous X-Ray Scattering Experiments

Artificial-intelligence-driven scanning probe microscopy

A dynamic Bayesian optimized active recommender system for curiosity-driven partially Human-in-the-loop automated experiments

Towards Adaptive Benthic Habitat Mapping

Bayesian Online Learning for Human-assisted Target Localization

Bayesian Conavigation: Dynamic Designing of the Material Digital Twins via Active Learning