Abstract:This article addresses the challenge of parameter calibration in stochastic models where the likelihood function is not analytically available. We propose a gradient-based simulated parameter estimation framework, leveraging a multi-time scale algorithm that tackles the issue of ratio bias in both maximum likelihood estimation and posterior density estimation problems. Additionally, we introduce a nested simulation optimization structure, providing theoretical analyses including strong convergence, asymptotic normality, convergence rate, and budget allocation strategies for the proposed algorithm. The framework is further extended to neural network training, offering a novel perspective on stochastic approximation in machine learning. Numerical experiments show that our algorithm can improve the estimation accuracy and save computational costs.
What problem does this paper attempt to address?
This paper attempts to address the challenges encountered in parameter calibration in stochastic models, especially when the likelihood function cannot be obtained in an analytical form. Specifically, the paper focuses on the ratio bias problem in maximum likelihood estimation (MLE) and posterior density estimation (PDE) problems. These problems are very common in many practical applications, such as in the fields of financial risk assessment and medical diagnosis.
### Main Problem Description
1. **Ratio Bias in Maximum Likelihood Estimation (MLE)**:
- In MLE, when the likelihood function has no analytical form, gradient estimation is usually used to optimize parameters. However, the form of gradient estimation involves a ratio:
\[
\nabla_\theta L_T(\theta) = \sum_{t = 1}^T \frac{\nabla_\theta p(Y_t; \theta)}{p(Y_t; \theta)}
\]
Even if the estimators of the numerator and denominator are unbiased, their ratio may still be biased. This will lead to the introduction of bias in the iterative process and may cause numerical instability.
2. **Ratio Bias in Posterior Density Estimation (PDE)**:
- In PDE, when the likelihood function is not analytically tractable, variational inference methods are usually used to approximate the posterior distribution. At this time, the ELBO (Evidence Lower Bound) needs to be estimated, and its gradient also involves a similar ratio form:
\[
\nabla_\lambda L(\lambda) = E_u[\nabla_\lambda \theta(u; \lambda) \cdot (\nabla_\theta \log p(y | \theta) + \nabla_\theta \log p(\theta) - \nabla_\theta \log q_\lambda(\theta))]
\]
The ratio form here will also introduce bias.
### Solution
To solve these problems, the authors propose a gradient - simulated parameter estimation (GSPE) framework based on the multi - time - scale (MTS) stochastic approximation (SA) algorithm. The main contributions of this framework include:
- **Eliminating Ratio Bias**: By introducing two coupled iterative processes, one of which updates at a faster speed and the other at a slower speed, the bias caused by the ratio form is eliminated.
- **Theoretical Analysis**: Provide theoretical results such as strong convergence, asymptotic normality, and convergence rate to ensure the effectiveness and stability of the algorithm.
- **Nested Simulation Optimization Structure**: For PDE problems, a nested MTS algorithm is designed to solve the nested simulation optimization problem in variational inference.
- **Extension to Neural Network Training**: Apply the MTS framework to neural network training, demonstrating its compatibility and extensibility in complex simulators and complex posterior distributions.
In summary, this paper aims to solve the ratio bias problem in parameter estimation in stochastic models through an innovative algorithm framework, improve the estimation accuracy and reduce the computational cost.