Abstract:Alternating direction method of multipliers (ADMM) has received tremendous interest for solving numerous problems in machine learning, statistics and signal processing. However, it is known that the performance of ADMM and many of its variants is very sensitive to the penalty parameter of a quadratic penalty applied to the equality constraints. Although several approaches have been proposed for dynamically changing this parameter during the course of optimization, they do not yield theoretical improvement in the convergence rate and are not directly applicable to stochastic ADMM. In this paper, we develop a new ADMM and its linearized variant with a new adaptive scheme to update the penalty parameter. Our methods can be applied under both deterministic and stochastic optimization settings for structured non-smooth objective function. The novelty of the proposed scheme lies at that it is adaptive to a local sharpness property of the objective function, which marks the key difference from previous adaptive scheme that adjusts the penalty parameter per-iteration based on certain conditions on iterates. On theoretical side, given the local sharpness characterized by an exponent $\theta\in(0, 1]$, we show that the proposed ADMM enjoys an improved iteration complexity of $\widetilde O(1/\epsilon^{1-\theta})$\footnote{$\widetilde O()$ suppresses a logarithmic factor.} in the deterministic setting and an iteration complexity of $\widetilde O(1/\epsilon^{2(1-\theta)})$ in the stochastic setting without smoothness and strong convexity assumptions. The complexity in either setting improves that of the standard ADMM which only uses a fixed penalty parameter. On the practical side, we demonstrate that the proposed algorithms converge comparably to, if not much faster than, ADMM with a fine-tuned fixed penalty parameter.

Finding the Near Optimal Policy via Adaptive Reduced Regularization in MDPs

Finding Near Optimal Policies via Reducive Regularization in Markov Decision Processes

A Regularized Approach to Sparse Optimal Policy in Reinforcement Learning.

Efficient Policy Iteration for Robust Markov Decision Processes via Regularization

An Aggressive Reduction on the Complexity of Optimization for Non-Strongly Convex Objectives

Policy Optimization with Advantage Regularization for Long-Term Fairness in Decision Systems

Bridging the Gap between Newton-Raphson Method and Regularized Policy Iteration

Landscape of Policy Optimization for Finite Horizon MDPs with General State and Action

A Dynamically Adaptive Approach to Reducing Strategic Interference for Multi-agent Systems

Near-Optimal Dynamic Regret for Adversarial Linear Mixture MDPs

Optimal Strong Regret and Violation in Constrained MDPs via Policy Optimization

Improved Regret Bounds for Linear Adversarial MDPs via Linear Optimization

Policy Mirror Descent for Regularized Reinforcement Learning: A Generalized Framework with Linear Convergence

Narrowing the Gap between Adversarial and Stochastic MDPs via Policy Optimization

Regularization and Variance-Weighted Regression Achieves Minimax Optimality in Linear MDPs: Theory and Practice

Optimizing Norm-Bounded Weighted Ambiguity Sets for Robust MDPs

Homotopic Policy Mirror Descent: Policy Convergence, Implicit Regularization, and Improved Sample Complexity

ADMM Without a Fixed Penalty Parameter: Faster Convergence with New Adaptive Penalization.

Adaptive Lightweight Regularization Tool for Complex Analytics

Policy Optimization finds Nash Equilibrium in Regularized General-Sum LQ Games

Near-Optimal Reinforcement Learning with Self-Play under Adaptivity Constraints