Abstract:Being the most classical generative model for serial data, state-space models (SSM) are fundamental in AI and statistical machine learning. In SSM, any form of parameter learning or latent state inference typically involves the computation of complex latent-state posteriors. In this work, we build upon the variational sequential Monte Carlo (VSMC) method, which provides computationally efficient and accurate model parameter estimation and Bayesian latent-state inference by combining particle methods and variational inference. While standard VSMC operates in the offline mode, by re-processing repeatedly a given batch of data, we distribute the approximation of the gradient of the VSMC surrogate ELBO in time using stochastic approximation, allowing for online learning in the presence of streams of data. This results in an algorithm, online VSMC, that is capable of performing efficiently, entirely on-the-fly, both parameter estimation and particle proposal adaptation. In addition, we provide rigorous theoretical results describing the algorithm's convergence properties as the number of data tends to infinity as well as numerical illustrations of its excellent convergence properties and usefulness also in batch-processing settings.
What problem does this paper attempt to address?
### Problems the paper attempts to solve
This paper aims to solve the problems of parameter learning and hidden state inference in state - space models (SSM). Specifically, the paper proposes an online variational sequential Monte Carlo method (OVSMC), which can perform parameter estimation and particle proposal adaptation in real - time in data streams, thus overcoming the limitation of traditional offline methods that need to process the entire data batch.
### Background and problem description
1. **State - Space Models (SSM)**
- SSM is a classic generative model, widely used in artificial intelligence and statistical machine learning.
- In SSM, parameter learning or state inference usually involves the calculation of complex joint posterior distributions, which are often difficult to solve analytically, especially in the case of non - linear Gaussian models or infinite state spaces.
2. **Limitations of existing methods**
- **Offline methods**: The traditional variational sequential Monte Carlo (VSMC) method is a technique that combines particle methods and variational inference, and can efficiently perform model parameter estimation and Bayesian hidden state inference. However, the standard VSMC method is offline and needs to repeatedly process a given data batch, which is inefficient when dealing with large - scale data streams.
- **Online methods**: Existing online parameter learning methods (such as particle - based recursive maximum likelihood estimation RML and online EM algorithms) rely on particle smoothing techniques, which may lead to the degeneration of particle lineages and high computational costs.
### Proposed method
The paper proposes an online variational sequential Monte Carlo method (OVSMC), with the following main features:
1. **Online learning**
- OVSMC realizes online learning by stochastically approximating the gradient of the variational sequential Monte Carlo evidence lower bound (ELBO) in time.
- This enables OVSMC to perform parameter estimation and particle proposal adaptation in real - time in data streams without re - processing the entire data batch.
2. **Theoretical guarantees**
- The paper provides strict theoretical results, describing the convergence properties of OVSMC. As the amount of data increases, the performance of OVSMC gradually approaches the idealized "asymptotic" VSMC.
- Specifically, the goal of OVSMC is to make the gradient of the parameter sequence \(\theta_t\) arbitrarily close to zero (in the \(L^2\) sense).
3. **Numerical experiments**
- The paper shows the fast parameter learning and efficient particle proposal adaptation capabilities of OVSMC in classical SSMs and more complex generative models through numerical experiments.
- The experimental results show that OVSMC can also be comparable to VSMC in batch processing problems.
### Conclusion
The online variational sequential Monte Carlo method (OVSMC) proposed in the paper provides an effective method for performing parameter estimation and particle proposal adaptation in real - time in data streams. Through strict theoretical analysis and numerical experiments, it is proved that OVSMC is efficient and robust when dealing with large - scale data streams.