Abstract:We consider the problem of sampling from a $d$-dimensional log-concave distribution $\pi(\theta) \propto \exp(-f(\theta))$ for $L$-Lipschitz $f$, constrained to a convex body with an efficiently computable self-concordant barrier function, contained in a ball of radius $R$ with a $w$-warm start. We propose a \emph{robust} sampling framework that computes spectral approximations to the Hessian of the barrier functions in each iteration. We prove that for polytopes that are described by $n$ hyperplanes, sampling with the Lee-Sidford barrier function mixes within $\widetilde O((d^2+dL^2R^2)\log(w/\delta))$ steps with a per step cost of $\widetilde O(nd^{\omega-1})$, where $\omega\approx 2.37$ is the fast matrix multiplication exponent. Compared to the prior work of Mangoubi and Vishnoi, our approach gives faster mixing time as we are able to design a generalized soft-threshold Dikin walk beyond log-barrier. We further extend our result to show how to sample from a $d$-dimensional spectrahedron, the constrained set of a semidefinite program, specified by the set $\{x\in \mathbb{R}^d: \sum_{i=1}^d x_i A_i \succeq C \}$ where $A_1,\ldots,A_d, C$ are $n\times n$ real symmetric matrices. We design a walk that mixes in $\widetilde O((nd+dL^2R^2)\log(w/\delta))$ steps with a per iteration cost of $\widetilde O(n^\omega+n^2d^{3\omega-5})$. We improve the mixing time bound of prior best Dikin walk due to Narayanan and Rakhlin that mixes in $\widetilde O((n^2d^3+n^2dL^2R^2)\log(w/\delta))$ steps.

What problem does this paper attempt to address?

### What problem does this paper attempt to solve? This paper aims to solve the problem of efficient sampling from log - concave distributions restricted to convex bodies. Specifically, given a $d$-dimensional log - concave distribution $\pi(\theta)\propto\exp(-f(\theta))$, the author focuses on how to perform fast and accurate sampling within the convex body that satisfies the following conditions: 1. **L - Lipschitz function $f$**: The function $f$ is L - Lipschitz continuous. 2. **Efficiently computable self - concordant barrier function**: There is an efficiently computable self - concordant barrier function inside the convex body. 3. **Warm start of the initial point**: Start from an initial point with a warm start (w - warm start). #### Main problems and challenges - **Complexity and efficiency**: Existing methods have a high computational cost in each iteration step, especially for high - dimensional data and complex convex body structures (such as polyhedra and the constraint sets of semidefinite programming). The goal of the paper is to design a more efficient sampling framework to reduce the cost per iteration step while maintaining a fast mixing time. - **Extension to different types of convex bodies**: Besides polyhedra, the paper also considers more complex convex bodies, such as the spectrahedron defined by semidefinite programming (SDP), and proposes corresponding efficient sampling algorithms. #### Specific problem description The main contribution of the paper is to propose a robust sampling framework that achieves efficient sampling by computing the spectral approximation of the Hessian matrix of the barrier function in each iteration. Specifically: - **Polyhedron sampling**: For a polyhedron described by $n$ hyperplanes, sampling using the Lee - Sidford barrier function can mix in $\tilde{O}((d^{2}+dL^{2}R^{2})\log(w /\delta))$ steps, with a cost of $\tilde{O}(nd^{\omega - 1})$ per step, where $\omega\approx2.37$ is the exponent of fast matrix multiplication. - **Spectrahedron sampling**: For a spectrahedron defined by $d$-dimensional semidefinite programming, using the improved Dikin walk can mix in $\tilde{O}((nd + dL^{2}R^{2})\log(w /\delta))$ steps, with a cost of $\tilde{O}(n^{\omega}+n^{2}d^{3\omega - 5})$ per step. #### Key techniques of the solution - **Spectral approximation**: Efficiently compute the Hessian matrix through spectral approximation techniques, thereby reducing the cost per iteration step. - **Lee - Sidford barrier function**: Utilize the special properties of the Lee - Sidford barrier function to ensure fast convergence during the sampling process. - **Randomization techniques**: Use randomized sketching techniques and spectral sparsification to accelerate the calculation of key steps. Through these techniques, the paper successfully improves the efficiency of the sampling algorithm and achieves a faster mixing time on multiple convex bodies. ### Summary This paper mainly solves the problem of efficient sampling from log - concave distributions restricted to convex bodies, especially the fast sampling on polyhedra and spectrahedra. By introducing a robust sampling framework and spectral approximation techniques, the author significantly reduces the cost per iteration step while maintaining a fast mixing time.

Log-concave Sampling from a Convex Body with a Barrier: a Robust and Unified Dikin Walk

Faster Sampling from Log-Concave Densities over Polytopes via Efficient Linear Solvers

Gaussian Cooling and Dikin Walks: The Interior-Point Method for Logconcave Sampling

Sampling with Barriers: Faster Mixing via Lewis Weights

Efficient Sampling from Time-Varying Log-Concave Distributions

Sampling from convex sets with a cold start using multiscale decompositions

Double Randomized Underdamped Langevin with Dimension-Independent Convergence Guarantee.

Faster high-accuracy log-concave sampling via algorithmic warm starts

In-and-Out: Algorithmic Diffusion for Sampling Convex Bodies

Log-Concave Sampling on Compact Supports: A Versatile Proximal Framework

Adaptive complexity of log-concave sampling

Penalized Overdamped and Underdamped Langevin Monte Carlo Algorithms for Constrained Sampling

Query lower bounds for log-concave sampling

Rényi-infinity constrained sampling with $d^3$ membership queries

Sampling numbers of smoothness classes via $\ell^1$-minimization

Constrained Sampling with Primal-Dual Langevin Monte Carlo

Faster Diffusion Sampling with Randomized Midpoints: Sequential and Parallel

Kinetic Langevin MCMC Sampling Without Gradient Lipschitz Continuity - the Strongly Convex Case

Faster Sampling without Isoperimetry via Diffusion-based Monte Carlo

On theoretical guarantees and a blessing of dimensionality for nonconvex sampling

Convergence of Unadjusted Langevin in High Dimensions: Delocalization of Bias