Abstract:Conformal prediction is a statistical tool for producing prediction regions of machine learning models that are valid with high probability. However, applying conformal prediction to time series data leads to conservative prediction regions. In fact, to obtain prediction regions over $T$ time steps with confidence $1-\delta$, {previous works require that each individual prediction region is valid} with confidence $1-\delta/T$. We propose an optimization-based method for reducing this conservatism to enable long horizon planning and verification when using learning-enabled time series predictors. Instead of considering prediction errors individually at each time step, we consider a parameterized prediction error over multiple time steps. By optimizing the parameters over an additional dataset, we find prediction regions that are not conservative. We show that this problem can be cast as a mixed integer linear complementarity program (MILCP), which we then relax into a linear complementarity program (LCP). Additionally, we prove that the relaxed LP has the same optimal cost as the original MILCP. Finally, we demonstrate the efficacy of our method on case studies using pedestrian trajectory predictors and F16 fighter jet altitude predictors.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to reduce conservatism in order to achieve more accurate long - term planning and verification when using the Conformal Prediction (CP) method in time - series data. Specifically, the paper points out that applying the standard conformal prediction method in time - series data will lead to overly conservative prediction regions. In order to obtain a prediction region with a confidence level of $1 - \delta$ over all $T$ time steps, previous methods require that the prediction region at each individual time step must have a confidence level of $1-\frac{\delta}{T}$. This results in conservative prediction regions, limiting their effectiveness in practical applications, especially in scenarios where accurate prediction is required, such as the safety tasks of self - driving cars. To solve this problem, the paper proposes an optimization method based on Linear Complementarity Programming (LCP). By considering the parameterized prediction errors over multiple time steps instead of considering the prediction error at each time step separately, this conservatism is reduced. Specifically, the paper introduces a parameterized non - conformity score function $R$, which is defined as: \[R := \max(\alpha_1 R_1,\ldots,\alpha_T R_T)\] where $R_t$ is the prediction error at the $t$ - th time step and $\alpha_t$ is a parameter. By optimizing these parameters $\alpha_t$, the paper aims to find non - conservative prediction regions. In addition, the paper also proves that this problem can be transformed into a Mixed Integer Linear Complementarity Programming (MILCP) problem and further relaxed into a Linear Complementarity Programming (LCP) problem, thus improving the computational efficiency. Finally, the paper demonstrates the effectiveness of this method through two case studies (pedestrian trajectory prediction and F16 fighter altitude prediction), proving that compared with existing methods, the proposed method can generate smaller and effective conformal prediction regions.

Conformal Prediction Regions for Time Series using Linear Complementarity Programming

Multi-Modal Conformal Prediction Regions by Optimizing Convex Shape Templates

Multi-Modal Conformal Prediction Regions with Simple Structures by Optimizing Convex Shape Templates

Conformal prediction for multi-dimensional time series by ellipsoidal sets

A novel Deep Learning approach for one-step Conformal Prediction approximation

Conformal Prediction with Learned Features

Conformal Loss-Controlling Prediction

Conformal Predictions for Longitudinal Data

Conformal Prediction Intervals for Remaining Useful Lifetime Estimation

Conformal Prediction Intervals with Temporal Dependence

Conformalization of Sparse Generalized Linear Models

Conformal Predictive Programming for Chance Constrained Optimization

Safe Planning in Dynamic Environments using Conformal Prediction

Conformal PID Control for Time Series Prediction

Efficient and minimal length parametric conformal prediction regions

Conformal Prediction via Regression-as-Classification

Efficient Conformal Prediction via Cascaded Inference with Expanded Admission

From Conformal Predictions to Confidence Regions

Conformal Prediction for Time Series with Modern Hopfield Networks

Length Optimization in Conformal Prediction