Abstract:Numerous studies have focused on learning and understanding the dynamics of physical systems from video data, such as spatial intelligence. Artificial intelligence requires quantitative assessments of the uncertainty of the model to ensure reliability. However, there is still a relative lack of systematic assessment of the uncertainties, particularly the uncertainties of the physical data. Our motivation is to introduce conformal prediction into the uncertainty assessment of dynamical systems, providing a method supported by theoretical guarantees. This paper uses the conformal prediction method to assess uncertainties with benchmark operator learning methods. We have also compared the Monte Carlo Dropout and Ensemble methods in the partial differential equations dataset, effectively evaluating uncertainty through straight roll-outs, making it ideal for time-series tasks.

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is the systematic deficiency in uncertainty assessment in dynamic systems, especially the uncertainty in physical data. Specifically, the author introduced the Conformal Prediction (CP) method to assess the uncertainty of dynamic systems and provided a method with theoretical guarantees. ### Main Problems 1. **Uncertainty Assessment in Dynamic Systems**: - Although a large number of studies have focused on learning and understanding the dynamic behaviors of physical systems from video data, there is a relative lack of quantitative assessment of model uncertainty, especially for the uncertainty of physical data. - Artificial intelligence requires reliable uncertainty assessment to ensure its reliability, and existing methods are insufficient in dealing with cumulative uncertainty, especially in time - series tasks. 2. **Limitations of Existing Methods**: - Classical uncertainty research mainly focuses on single - step prediction, ignoring the cumulative uncertainty that evolves over time in forward or reverse problems. - Existing uncertainty assessment methods such as Bayesian methods, Monte Carlo Dropout, and ensemble methods have certain effects, but are insufficient in using calibration data to improve the accuracy of prediction intervals. ### Solutions To address the above problems, this paper proposes the following solutions: 1. **Introducing the Conformal Prediction Method**: - Conformal Prediction is a relatively new uncertainty assessment method. By converting the algorithm's predictions into prediction sets, it ensures that the coverage rate of the prediction interval is not lower than the set confidence level. - This method utilizes the information of the calibration data set to ensure the coverage rate under a finite sample, is suitable for time - series tasks, and provides theoretical upper and lower bound guarantees. 2. **Rotational Symmetry as an Evaluation Basis**: - The rotational invariance of the physical system is used as a reference for evaluating uncertainty. The uncertainty is eliminated through the calibration data set to make it consistent with the physical interpretation. 3. **Comprehensive Uncertainty Assessment**: - This research conducted a complete quantitative assessment of multiple uncertainty quantification methods, including Monte Carlo Dropout, Conformal Prediction, and ensemble methods, especially their applications on the Partial Differential Equations (PDEs) data set. ### Experimental Verification - The performance of different methods on the turbulence data set (Navier - Stokes equations) was verified through experiments, and their performance in forward problems and symmetry tests was compared. - The results show that the Conformal Prediction method performs excellently in uncertainty estimation and calibration, especially in using calibration data to improve the accuracy of prediction intervals, which is superior to other methods. ### Formula Presentation The core of Conformal Prediction lies in constructing a prediction set \(\hat{C}_n(x)\) that satisfies the finite - sample coverage property, that is: \[ P(Y_{n + 1}\in\hat{C}_n(X_{n + 1})|(X_i,Y_i)_{i = 1}^n)\geq1-\alpha \] where: - \((X_i,Y_i)\sim P\) represents samples independently and identically distributed drawn from an unknown distribution \(P\). - \(\alpha\) is a predefined error tolerance level, for example, \(\alpha = 0.05\) corresponds to a 95% confidence level. Through these methods and experiments, this paper aims to provide a more reliable and more systematic uncertainty assessment method for dynamic systems, thereby improving the credibility and reliability of the model.

Conformal Prediction on Quantifying Uncertainty of Dynamic Systems

Uncertainty Quantification of Surrogate Models using Conformal Prediction

Conformal Prediction in Dynamic Biological Systems

Calibrated Uncertainty Quantification for Operator Learning via Conformal Prediction

Conformal Prediction: A Data Perspective

Quantifying Deep Learning Model Uncertainty in Conformal Prediction

Uncertainty quantification for probabilistic machine learning in earth observation using conformal prediction

Conformal Prediction for Uncertainty-Aware Planning with Diffusion Dynamics Model

Conformal Prediction: A Gentle Introduction

A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification

A comparative study of conformal prediction methods for valid uncertainty quantification in machine learning

Safety-Critical Control with Uncertainty Quantification using Adaptive Conformal Prediction

Multi-model Ensemble Conformal Prediction in Dynamic Environments

Quantum Conformal Prediction for Reliable Uncertainty Quantification in Quantum Machine Learning

Conformal Prediction for Natural Language Processing: A Survey

An Information Theoretic Perspective on Conformal Prediction

Quantifying Aleatoric and Epistemic Dynamics Uncertainty via Local Conformal Calibration

Conformalized-DeepONet: A Distribution-Free Framework for Uncertainty Quantification in Deep Operator Networks

DOA Uncertainty Quantification With Conformal Prediction