Abstract:In this letter, we study the proximal gradient dynamics. This recently-proposed continuous-time dynamics solves optimization problems whose cost functions are separable into a nonsmooth convex and a smooth component. First, we show that the cost function decreases monotonically along the trajectories of the proximal gradient dynamics. We then introduce a new condition that guarantees exponential convergence of the cost function to its optimal value, and show that this condition implies the proximal Polyak-Łojasiewicz condition. We also show that the proximal Polyak-Łojasiewicz condition guarantees exponential convergence of the cost function. Moreover, we extend these results to time-varying optimization problems, providing bounds for equilibrium tracking. Finally, we discuss applications of these findings, including the LASSO problem, quadratic optimization with polytopic constraints, and certain matrix based problems.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is the convergence properties of cost functions in optimization problems and their applications. Specifically, the paper studies Proximal Gradient Dynamics, which is a continuous - time dynamical system for solving optimization problems where the cost function can be decomposed into a non - smooth convex part and a smooth part. The main contributions of the paper include:
1. **Monotonicity**: It is proved that under proximal gradient dynamics, the cost function is monotonically decreasing along the trajectory.
2. **Exponential Convergence**: A new condition is introduced, which ensures that the cost function converges to its optimal value at an exponential rate, and this condition implies the proximal Polyak - Łojasiewicz (PL) condition.
3. **Time - Varying Optimization Problems**: These results are extended to time - varying optimization problems, providing bounds for equilibrium tracking.
4. **Applications**: The applications of these findings are discussed, including the LASSO problem, quadratic optimization problems with polyhedral constraints, and certain matrix problems.
### Main Contributions
1. **Monotonicity**:
- It is proved that under proximal gradient dynamics, the cost function \( V(x)=f(x)+g(x)-F^{*} \) is non - increasing along the system trajectory, and its upper right Dini derivative \( D^{+}V(x)\leq-\frac{1}{\alpha}\left\|F_{\text{prox}}(x)\right\|^{2} \).
2. **Exponential Convergence**:
- A new condition is introduced, which ensures that the cost function converges to its optimal value \( F^{*} \) at an exponential rate, and this condition implies the proximal Polyak - Łojasiewicz (PL) condition.
- It is proved that if there exists a constant \( \mu>0 \) such that for each \( x \), \(\frac{1}{2}\left\|F_{\text{prox}}(x)\right\|^{2}\geq\mu\alpha^{2}(F(x)-F^{*})\), then each trajectory \( x(t) \) satisfies \( F(x(t))-F^{*}\leq e^{-2\mu\alpha t}(F(x(0))-F^{*}) \).
3. **Time - Varying Optimization Problems**:
- For the time - varying optimization problem \( \min_{x}f(x,\theta(t))+g(x,\theta(t)) = F(x,\theta(t)) \), under Lipschitz constraints, the bound of the difference between the current cost and the minimum cost is defined.
- By the Grönwall inequality, the bound of cost function tracking \( V(t)\leq e^{-\mu\alpha t}V(0)+\frac{2\ell_{\theta}}{\mu\alpha}\int_{0}^{t}e^{-\mu\alpha(t-\tau)}\left\|\dot{\theta}(\tau)\right\|d\tau \) is given.
4. **Applications**:
- **LASSO Problem**: The \( \ell_{1} \) - regularized least - squares problem (LASSO) is solved by proximal gradient dynamics, and it is proved that the cost function converges to the optimal value at an exponential rate.
- **Quadratic Optimization Problems with Polyhedral Constraints**: The quadratic optimization problem with linear constraints is considered, and it is proved that when the initial point is in the feasible set and \( \alpha=\frac{1}{\lambda_{\min}(Q)} \), the cost function converges to the optimal value at an exponential rate.
- **Matrix Problems**: Low - rank matrix recovery and matrix completion problems, as well as matrix factorization problems, are discussed, and the corresponding proximal gradient dynamics are given.
### Formula Summary
- **Proximal Operator**:
\[
\text{prox}_{\alpha g}(x)=\arg\min_{u\in\mathbb{R}^{n}}\left(\al