Abstract:Uncertainty in the predictions of learning-enabled components hinders their deployment in safety-critical cyber-physical systems (CPS). A shift from the training distribution of a learning-enabled component (LEC) is one source of uncertainty in the LEC's predictions. Detection of this shift or out-of-distribution (OOD) detection on individual datapoints has therefore gained attention recently. But in many applications, inputs to CPS form a temporal sequence. Existing techniques for OOD detection in time-series data for CPS either do not exploit temporal relationships in the sequence or do not provide any guarantees on detection. We propose using deviation from the in-distribution temporal equivariance as the non-conformity measure in conformal anomaly detection framework for OOD detection in time-series data for CPS. Computing independent predictions from multiple conformal detectors based on the proposed measure and combining these predictions by Fisher's method leads to the proposed detector CODiT with bounded false alarms. CODiT performs OOD detection on fixed-length windows of consecutive time-series datapoints by using Fisher value of the input window. We further propose performing OOD detection on real-time time-series traces of variable lengths with bounded false alarms. This can be done by using CODiT to compute Fisher values of the sliding windows in the input trace and combining these values by a merging function. Merging functions such as Harmonic Mean, Arithmetic Mean, Geometric Mean, and Bonferroni Method, etc. can be used to combine Fisher values of the sliding windows in the input trace, and the combined value can be used for OOD detection on the trace with bounded false alarm rate guarantees. We illustrate the efficacy of CODiT by achieving state-of-the-art results in two case studies for OOD detection on fixed-length windows. The first one is on an autonomous driving system with perception (or vision) LEC. The second case study is on a medical CPS for walking pattern or GAIT analysis where physiological (non-vision) data is collected with force-sensitive resistors attached to the subject's body. For OOD detection on variable length traces, we consider the same case studies on the autonomous driving system and medical CPS for GAIT analysis. We report our results with four merging functions on the Fisher values computed by CODiT on the sliding windows of the input trace. We also compare the false alarm rate guarantees by these four merging functions in the autonomous driving system case study. Code, data, and trained models are available at https://github.com/kaustubhsridhar/time-series-OOD.

Towards Rigorous Design of OoD Detectors

Co-Design of Out-of-Distribution Detectors for Autonomous Emergency Braking Systems

Deciphering the Definition of Adversarial Robustness for post-hoc OOD Detectors

Rethinking Out-of-Distribution Detection From a Human-Centric Perspective

Recent Advances in OOD Detection: Problems and Approaches

Concept-based Explanations for Out-Of-Distribution Detectors

EPA: Neural Collapse Inspired Robust Out-of-Distribution Detector

Towards Building Self-Aware Object Detectors via Reliable Uncertainty Quantification and Calibration

Out-Of-Distribution Detection Is Not All You Need

Out-of-Distribution Detection in Dependent Data for Cyber-Physical Systems with Conformal Guarantees

Out-of-Distribution Detection for LiDAR-based 3D Object Detection

Interpretable Out-Of-Distribution Detection Using Pattern Identification

Generalized Out-of-Distribution Detection: A Survey

Meta OOD Learning for Continuously Adaptive OOD Detection

Advancing Out-of-Distribution Detection through Data Purification and Dynamic Activation Function Design

Continual Unsupervised Out-of-Distribution Detection

Re-Benchmarking Out-of-Distribution Detection in Deep Neural Networks

Out-of-Distribution (OOD) Detection Based on Deep Learning: A Review

Toward a Realistic Benchmark for Out-of-Distribution Detection

OpenOOD v1.5: Enhanced Benchmark for Out-of-Distribution Detection