Abstract:The ability to evaluate uncertainties in evolving data streams has become equally, if not more, crucial than building a static predictor. For instance, during the pandemic, a model should consider possible uncertainties such as governmental policies, meteorological features, and vaccination schedules. Neural process families (NPFs) have recently shone a light on predicting such uncertainties by bridging Gaussian processes (GPs) and neural networks (NNs). Their abilities to output average predictions and the acceptable variances, i.e., uncertainties, made them suitable for predictions with insufficient data, such as meta-learning or few-shot learning. However, existing models have not addressed continual learning which imposes a stricter constraint on the data access. Regarding this, we introduce a member meta-continual learning with neural process (MCLNP) for uncertainty estimation. We enable two levels of uncertainty estimations: the local uncertainty on certain points and the global uncertainty p(z) that represents the function evolution in dynamic environments. To facilitate continual learning, we hypothesize that the previous knowledge can be applied to the current task, hence adopt a coreset as a memory buffer to alleviate catastrophic forgetting. The relationships between the degree of global uncertainties with the intratask diversity and model complexity are discussed. We have estimated prediction uncertainties with multiple evolving types including abrupt/gradual/recurrent shifts. The applications encompass meta-continual learning in the 1-D, 2-D datasets, and a novel spatial-temporal COVID dataset. The results show that our method outperforms the baselines on the likelihood and can rebound quickly even for heavily evolved data streams.

N A ] 2 6 Se p 20 20 Machine Learning from a Continuous Viewpoint

Machine learning from a continuous viewpoint, I

Machine Learning from a Continuous Viewpoint,i

PDE problems that arise from machine learning

Uncertainty Estimation With Neural Processes for Meta-Continual Learning

A Proposal on Machine Learning Via Dynamical Systems

A Deep Learning Approach to Analyzing Continuous-Time Systems

Dynamical Perspective of Machine Learning: an Introductory Review

A Temporal Kernel Approach for Deep Learning with Continuous-time Information

Some Theoretical Issues on Continuous Process Neural Networks

A Continuous Volatility Forecasting Model Based on Neural Differential Equations and Scale-Similarity

Machine learning and computational mathematics

Configuration of Continuous Piecewise-Linear Neural Networks

Continuously Additive Models for Nonlinear Functional Regression

Space and Time Continuous Physics Simulation From Partial Observations

Continual Neural Mapping: Learning an Implicit Scene Representation from Sequential Observations

Continuous Management of Machine Learning-Based Application Behavior