Peaking into the Black-box: Prediction Intervals Give Insight into Data-driven Quadrotor Model Reliability

Jasper van Beers,Coen de Visser

DOI: https://doi.org/10.2514/6.2023-0125

2024-08-12

Abstract:Ensuring the reliability and validity of data-driven quadrotor model predictions is essential for their accepted and practical use. This is especially true for grey- and black-box models wherein the mapping of inputs to predictions is not transparent and subsequent reliability notoriously difficult to ascertain. Nonetheless, such techniques are frequently and successfully used to identify quadrotor models. Prediction intervals (PIs) may be employed to provide insight into the consistency and accuracy of model predictions. This paper estimates such PIs for polynomial and Artificial Neural Network (ANN) quadrotor aerodynamic models. Two existing ANN PI estimation techniques - the bootstrap method and the quality driven method - are validated numerically for quadrotor aerodynamic models using an existing high-fidelity quadrotor simulation. Quadrotor aerodynamic models are then identified on real quadrotor flight data to demonstrate their utility and explore their sensitivity to model interpolation and extrapolation. It is found that the ANN-based PIs widen considerably when extrapolating and remain constant, or shrink, when interpolating. While this behaviour also occurs for the polynomial PIs, it is of lower magnitude. The estimated PIs establish probabilistic bounds within which the quadrotor model outputs will likely lie, subject to modelling and measurement uncertainties that are reflected through the PI widths.

Systems and Control,Artificial Intelligence

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is to ensure the reliability and effectiveness of data - driven quadrotor unmanned aerial vehicle (UAV) model prediction, especially for grey - box and black - box models. The mapping from input to prediction in these models is opaque, so their reliability is difficult to evaluate. The paper provides insights into the consistency and accuracy of model predictions through Prediction Intervals (PIs). Specifically, the goals of the paper include: 1. **Evaluating the effectiveness of prediction intervals**: Verify the application effects of two existing ANN prediction interval estimation techniques (the Bootstrap method and the quality - driven method) in the quadrotor UAV aerodynamic model. 2. **Model identification and verification**: Use high - fidelity quadrotor UAV simulation data to identify polynomial and ANN aerodynamic models and verify the performance of prediction intervals. 3. **Application of actual flight data**: Apply the models and prediction intervals to real - world quadrotor UAV flight data to demonstrate their practicality and sensitivity to interpolation and extrapolation. 4. **Exploring model behavior**: Investigate the behavior of prediction intervals during model interpolation and extrapolation, especially when dealing with inputs outside the range of training data. Through these goals, the paper aims to provide a reliable method for data - driven models of quadrotor UAVs to evaluate and improve the credibility of their predictions.

Peaking into the Black-box: Prediction Intervals Give Insight into Data-driven Quadrotor Model Reliability

Grey-box modeling of a small-scale helicopter using physical knowledge and Bayesian Techniques

Quantification Of Performance Uncertainty For A Transonic Compressor Rotor Using An Adaptive Nipc Method

Integral Backstepping-based Nonlinear Flight Control Strategy for Quadrotor Aerial Robot with Unknown Mass

Physics-Infused Machine Learning Based Prediction of VTOL Aerodynamics with Sparse Datasets

Comprehensive Review of Neural Network-Based Prediction Intervals and New Advances

Data-Driven System Identification of Quadrotors Subject to Motor Delays

Prediction Interval Estimation of Aeroengine Remaining Useful Life Based on Bidirectional Long Short-Term Memory Network

Direct Interval Forecasting of Wind Speed Using Radial Basis Function Neural Networks in a Multi-Objective Optimization Framework.

NeuroBEM: Hybrid Aerodynamic Quadrotor Model

Dual Accuracy-Quality-Driven Neural Network for Prediction Interval Generation

Machine learning enhanced real-time aerodynamic forces prediction based on sparse pressure sensor inputs

A New Wind Power Interval Prediction Approach Based on Reservoir Computing and a Quality-Driven Loss Function.

Predicting On-Axis Rotorcraft Dynamic Responses Using Machine Learning Techniques

Neural Network Based Model Predictive Control for a Quadrotor UAV

Machine-Learning-Enhanced Real-Time Aerodynamic Forces Prediction Based on Sparse Pressure Sensor Inputs

Applying Machine Learning Techniques: Uncertainty Quantification in Nonlinear Dynamics Characters Predictions via Gated Recurrent Unit-Based Reduced-Order Models

RAMP-Net: A Robust Adaptive MPC for Quadrotors via Physics-informed Neural Network

An Interval Prediction Approach of Wind Power Based on Skip-GRU and Block-Bootstrap Techniques

Short-Term Probabilistic Wind Speed Predictions Integrating Multivariate Linear Regression and Generative Adversarial Network Methods

Multi-objective prediction intervals for wind power forecast based on deep neural networks