Abstract:This paper investigates the problem of online statistical inference of model parameters in stochastic optimization problems via the Kiefer-Wolfowitz algorithm with random search directions. We first present the asymptotic distribution for the Polyak-Ruppert-averaging type Kiefer-Wolfowitz (AKW) estimators, whose asymptotic covariance matrices depend on the distribution of search directions and the function-value query complexity. The distributional result reflects the trade-off between statistical efficiency and function query complexity. We further analyze the choice of random search directions to minimize certain summary statistics of the asymptotic covariance matrix. Based on the asymptotic distribution, we conduct online statistical inference by providing two construction procedures of valid confidence intervals.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is to conduct online statistical inference in stochastic optimization problems by using the Kiefer - Wolfowitz algorithm with random search directions. Specifically, the paper focuses on how to effectively estimate model parameters without directly obtaining gradient information, and on this basis, construct effective confidence intervals to achieve online statistical inference. The main contributions of the paper are as follows:
1. **Quantification of Asymptotic Covariance Structure**: The paper first quantifies the asymptotic distribution of the Polyak - Ruppert - averaging type of Kiefer - Wolfowitz (AKW) estimator. Its asymptotic covariance matrix depends on the distribution of search directions and the function - value query complexity. This result reflects the trade - off between statistical efficiency and function query complexity.
2. **Analysis of the Selection of Random Search Directions**: The paper further analyzes the impact of the selection of different random search directions on the asymptotic covariance matrix, aiming to minimize some summary statistics of the asymptotic covariance matrix.
3. **Extension in the Multi - Query Setting**: The paper also studies the AKW estimator in the case where multiple function values can be evaluated at each step, showing that as the number of queries increases, the asymptotic covariance matrix decreases, and when the number of queries approaches infinity, the AKW estimator can achieve asymptotic statistical efficiency.
4. **Online Statistical Inference Methods**: Based on the asymptotic distribution results, the paper proposes two online statistical inference methods for constructing effective confidence intervals. One method is to use a plug - in estimator of the asymptotic covariance matrix, and the other method is to construct an asymptotically pivotal statistic by normalizing the AKW estimator without directly estimating the covariance matrix.
### Formula Presentation
1. **Objective Function**:
\[
\theta^{\star}=\arg\min F(\theta), \quad \text{where} \quad F(\theta)=\mathbb{E}_{\zeta}[f(\theta; \zeta)]=\int f(\theta; \zeta) dP_{\zeta}
\]
2. **Kiefer - Wolfowitz (KW) Algorithm**:
\[
\theta_{n}^{(KW)}=\theta_{n - 1}^{(KW)}-\eta_{n} \hat{g}(h_{n}, v_{n}; \theta_{n - 1}, \zeta_{n}), \quad \text{where} \quad \hat{g}(h_{n}, v_{n}; \theta_{n - 1}, \zeta_{n})=\frac{f(\theta_{n - 1}+h_{n} v_{n}; \zeta_{n})-f(\theta_{n - 1}; \zeta_{n})}{h_{n} v_{n}}
\]
3. **Asymptotic Distribution**:
\[
\sqrt{n}(\theta_{n}-\theta^{\star}) \xrightarrow{d} \mathcal{N}(0, H^{-1} Q H^{-1})
\]
where \(H = \nabla^{2} F(\theta^{\star})\) is the Hessian matrix of the population loss function \(F(\theta)\) at \(\theta=\theta^{\star}\), \(Q=\mathbb{E}[v v^{\top} S v v^{\top}]\), \(S=\mathbb{E}[\nabla f(\theta^{\star}; \zeta) \nabla f(\theta^{\star}; \zeta)^{\top}]\).
4. **Asymptotic Distribution in the Multi - Query Setting**:
\[
\sqrt{n}(\theta_{n}^{(m)}-\theta^{\star}) \xrightarrow{d} \mathcal{N}(0, H^{-1} Q_{m} H^{-1})