Decomposition Method Using Augmented Lagrangian for Solving Systems of Variational Inequalities

LU Qi-hui,ZHU Dao-li

DOI: https://doi.org/10.3969/j.issn.1001-9847.2005.04.022

2005-01-01

Abstract:In this paper, we consider the system of variational inequalities with constraints. We give a decomposition algorithm based on the augmented Lagrangian duality. In this paper we main consider the iterative algorithm′s convergence analysis.

What problem does this paper attempt to address?

Augmented Lagrangian Theory, Duality and Decomposition Methods for Variational Inequality Problems

D.L. Zhu

DOI: https://doi.org/10.1023/A:1023664709604

2003-01-01

Journal of Optimization Theory and Applications

Abstract:In this paper, we develop the augmented Lagrangian theory and duality theory for variational inequality problems. We propose also decomposition methods based on the augmented Lagrangian for solving complex variational inequality problems with coupling constraints.
Convergence Analysis of General Decomposition Algorithm for Solving General Constrained Variational Inequality Problems

Qi-hui LU,Dao-li ZHU

DOI: https://doi.org/10.3969/j.issn.1001-9847.2006.02.004

2006-01-01

Abstract:In this paper, we generalize the augmented Lagrangian duality theory for the general varψ(x* ,y)-ψ(x* ,x*)≥0,()y∈Γ, whereΓ= {y∈ X| ()(y) ∈-C}. We proposeageneraldecomposition algorithm based on the augmented Lagrangian for solving this complex general variational inequality problems with the coupling constraints.
A New Decomposition Method for Variational Inequalities with Linear Constraints

Min Zhang,Deren Han,Gang Qian,Xihong Yan

DOI: https://doi.org/10.1007/s10957-011-9931-2

2011-01-01

Journal of Optimization Theory and Applications

Abstract:We propose a new decomposition method for solving a class of monotone variational inequalities with linear constraints. The proposed method needs only to solve a well-conditioned system of nonlinear equations, which is much easier than a variational inequality, the subproblem in the classic alternating direction methods. To make the method more flexible and practical, we solve the sub-problems approximately. We adopt a self-adaptive rule to adjust the parameter, which can improve the numerical performance of the algorithm. Under mild conditions, the underlying mapping be monotone and the solution set of the problem be nonempty, we prove the global convergence of the proposed algorithm. Finally, we report some preliminary computational results, which demonstrate the promising performance of the new algorithm.
A DECOMPOSITION AUGMENTED LAGRANGIAN METHOD FOR LOW-RANK SEMIDEFINITE PROGRAMMING

Yifei Wang,Kangkang Deng,Haoyang Liu,Zaiwen Wen

DOI: https://doi.org/10.1137/22m1474539

IF: 2.763

2023-01-01

SIAM Journal on Optimization

Abstract:We develop a decomposition method based on the augmented Lagrangian framework to solve a broad family of semidefinite programming problems, possibly with nonlinear objective functions, nonsmooth regularization, and general linear equality/inequality constraints. In particular, the positive semidefinite variable along with a group of linear constraints can be transformed into a variable on a smooth manifold via matrix factorization. The nonsmooth regularization and other general linear constraints are handled by the augmented Lagrangian method. Therefore, each subproblem can be solved by a semismooth Newton method on a manifold. Theoretically, we show that the first and second-order necessary optimality conditions for the factorized subproblem are also sufficient for the original subproblem under certain conditions. Convergence analysis is established for the Riemannian subproblem and the augmented Lagrangian method. Extensive numerical experiments on large-scale semidefinite programming problems such as max-cut, nearest correlation estimation, clustering, and sparse principal component analysis demonstrate the strength of our proposed method compared to other state-of-the-art methods.
Parallel inertial forward–backward splitting methods for solving variational inequality problems with variational inclusion constraints

Tran Van Thang,Ha Manh Tien

DOI: https://doi.org/10.1002/mma.10356

2024-09-09

Mathematical Methods in the Applied Sciences

Abstract:The inertial forward–backward splitting algorithm can be considered as a modified form of the forward–backward algorithm for variational inequality problems with monotone and Lipschitz continuous cost mappings. By using parallel and inertial techniques and the forward–backward splitting algorithm, in this paper, we propose a new parallel inertial forward–backward splitting algorithm for solving variational inequality problems, where the constraints are the intersection of common solution sets of a finite family of variational inclusion problems. Then, strong convergence of proposed iteration sequences is showed under standard assumptions imposed on cost mappings in a real Hilbert space. Finally, some numerical experiments demonstrate the reliability and benefits of the proposed algorithm.

mathematics, applied
Variance reduced forward-reflected-backward algorithm for solving nonconvex finite-sum mixed variational inequalities

Mengjun Wang,Fuquan Xia

DOI: https://doi.org/10.3934/jimo.2024021

2024-02-28

Journal of Industrial and Management Optimization

Abstract:In this paper, we investigate a variance reduced forward-reflected-backward algorithm for solving nonconvex finite-sum mixed variational inequalities, where is Lipschitz continuous, is proper lower semicontinuous prox-convex, and they satisfy a generalized monotonicity condition on . We prove that the algorithm is almost surely convergent and exhibits -linear rate when is strongly monotone.

engineering, multidisciplinary,operations research & management science,mathematics, interdisciplinary applications
An Operator Splitting Method for Variational Inequalities with Partially Unknown Mappings

Deren Han,Wei Xu,Hai Yang

DOI: https://doi.org/10.1007/s00211-008-0181-7

IF: 2.5

2008-01-01

Numerische Mathematik

Abstract:In this paper, we propose a new operator splitting method for solving a class of variational inequality problems in which part of the underlying mappings are unknown. This class of problems arises frequently from engineering, economics and transportation equilibrium problems. At each iteration, by using the information observed from the system, the method solves a system of nonlinear equations, which is well-defined. Under mild assumptions, the global convergence of the method is proved, and its efficiency is demonstrated with numerical examples.
Some iterative algorithms for solving mixed variational inequalities

A. Bnouhachem,M. Noor,A. Massaq,Zhaohan Sheng

2014-01-01

Abstract:In this paper, we propose two new methods for solving mixed quasi variational inequalities by using the resolvent operator technique. Under certain conditions, the global convergence of the both methods is proved. The skew-symmetry of the nonlinear bifunction plays a crucial part in the convergence analysis of these new iterative methods. The comparison of these methods with other methods for solving the mixed quasi variational inequalities is an open interesting problem.
Variable dimensional algorithm for variational inequalities

Xiwen Lu,Quan Zheng

1995-01-01

Abstract:The existence of solution of VI(K,F) was studied. A variable dimensional algorithm for solving VI was given. Convergence of this algorithm was discussed. Numerical evaluation results showed that this algorithm is efficient.
An Augmented Lagrangian Deep Learning Method for Variational Problems with Essential Boundary Conditions

Jianguo Huang,Haoqin Wang,Tao Zhou

DOI: https://doi.org/10.4208/cicp.OA-2021-0176

IF: 3.791

2022-01-01

Communications in Computational Physics

Abstract:This paper is concerned with a novel deep learning method for variational problems with essential boundary conditions. To this end, we first reformulate the original problem into a minimax problem corresponding to a feasible augmented Lagrangian, which can be solved by the augmented Lagrangian method in an infinite dimensional setting. Based on this, by expressing the primal and dual variables with two individual deep neural network functions, we present an augmented Lagrangian deep learning method for which the parameters are trained by the stochastic optimization method together with a projection technique. Compared to the traditional penalty method, the new method admits two main advantages: i) the choice of the penalty parameter is flexible and robust, and ii) the numerical solution is more accurate in the same magnitude of computational cost. As typical applications, we apply the new approach to solve elliptic problems and (nonlinear) eigenvalue problems with essential boundary conditions, and numerical experiments are presented to show the effectiveness of the new method.
A New Penalty Dual-Primal Augmented Lagrangian Method and Its Extensions

Jie Liu,Xiaoqing Ou,Jiawei Chen

2023-05-06

Abstract:In this paper, we propose a penalty dual-primal augmented lagrangian method for solving convex minimization problems under linear equality or inequality constraints. The proposed method combines a novel penalty technique with updates the new iterates in a dual-primal order, and then be extended to solve multiple-block separable convex programming problems with splitting version and partial splitting version. We establish the convergence analysis for all the introduced algorithm in the lens of variational analysis. Numerical results on the basic pursuit problem and the lasso model are presented to illustrate the efficiency of the proposed method.

Optimization and Control
A novel method for hierarchical variational inequality with split common fixed point constraint

Mohammad Eslamian,Ahmad Kamandi

DOI: https://doi.org/10.1007/s12190-024-02024-4

2024-03-16

Journal of Applied Mathematics and Computing

Abstract:We address the strongly monotone variational inequality problem over the solution set of the split common fixed point problem with demimetric mappings in real Hilbert spaces. In order to solve this problem, we propose a new method that makes use of the inertial method with a correction term and a self-adaptive step size strategy. To demonstrate the effectiveness and performance of our proposed algorithm, we present two numerical examples, where one is related to the constrained convex minimization problem and the other is related to an application that performs binary classification based on support vector machines.

mathematics, applied
Two Novel Algorithms for Solving Variational Inequality Problems Governed by Fixed Point Problems and Their Applications

Shanshan Xu,Bing Tan,Songxiao Li

DOI: https://doi.org/10.24193/fpt-ro.2024.2.20

2024-01-01

Fixed Point Theory

Abstract:We study the problem of finding a common solution to the variational inequality problem with a pseudomonotone and Lipschitz continuous operator and the fixed point problem with a demicontractive mapping in real Hilbert spaces. Inspired by the inertial method and the subgradient extragradient method, two improved viscosity-type efficient iterative methods with a new adaptive non-monotonic step size criterion are proposed. We prove that the strong convergence theorems of these new methods hold under some standard and mild conditions. Numerical examples in finite- and infinite-dimensional spaces are provided to illustrate the effectiveness and potential applicability of the suggested iterative methods compared to some known ones.
A New Insight on Augmented Lagrangian Method with Applications in Machine Learning

Jianchao Bai,Linyuan Jia,Zheng Peng

DOI: https://doi.org/10.1007/s10915-024-02518-0

2024-04-14

Journal of Scientific Computing

Abstract:By exploiting double-penalty terms for the primal subproblem, we develop a novel relaxed augmented Lagrangian method for solving a family of convex optimization problems subject to equality or inequality constraints. The method is then extended to solve a general multi-block separable convex optimization problem, and two related primal-dual hybrid gradient algorithms are also discussed. Convergence results about the sublinear and linear convergence rates are established by variational characterizations for both the saddle-point of the problem and the first-order optimality conditions of involved subproblems. A large number of experiments on testing the linear support vector machine problem and the robust principal component analysis problem arising from machine learning indicate that our proposed algorithms perform much better than several state-of-the-art algorithms.

mathematics, applied
An Augmented Lagrangian Approach to Conically Constrained Non-monotone Variational Inequality Problems

Lei Zhao,Daoli Zhu,Shuzhong Zhang

2023-06-02

Abstract:In this paper we consider a non-monotone (mixed) variational inequality model with (nonlinear) convex conic constraints. Through developing an equivalent Lagrangian function-like primal-dual saddle-point system for the VI model in question, we introduce an augmented Lagrangian primal-dual method, to be called ALAVI in the current paper, for solving a general constrained VI model. Under an assumption, to be called the primal-dual variational coherence condition in the paper, we prove the convergence of ALAVI. Next, we show that many existing generalized monotonicity properties are sufficient -- though by no means necessary -- to imply the above mentioned coherence condition, thus are sufficient to ensure convergence of ALAVI. Under that assumption, we further show that ALAVI has in fact an $o(1/\sqrt{k})$ global rate of convergence where $k$ is the iteration count. By introducing a new gap function, this rate further improves to be $O(1/k)$ if the mapping is monotone. Finally, we show that under a metric subregularity condition, even if the VI model may be non-monotone the local convergence rate of ALAVI improves to be linear. Numerical experiments on some randomly generated highly nonlinear and non-monotone VI problems show practical efficacy of the newly proposed method.

Optimization and Control,Machine Learning
Iterative Algorithms for Variational Inequality and Equilibrium Problems with Applications.

Xiaolong Qin,Sun Young Cho,Shin Min Kang

DOI: https://doi.org/10.1007/s10898-009-9498-8

2009-01-01

Journal of Global Optimization

Abstract:In this paper, we introduce an iterative method for finding a common element of the set of solutions of equilibrium problems, of the set of variational inequalities and of the set of common fixed points of an infinite family of nonexpansive mappings in the framework of real Hilbert spaces. Strong convergence of the proposed iterative algorithm is obtained. As an application, we utilize the main results which improve the corresponding results announced in Chang et al. (Nonlinear Anal, 70:3307-3319, 2009), Colao et al. (J Math Anal Appl, 344:340-352, 2008), Plubtieng and Punpaeng (Appl Math Comput, 197:548-558, 2008) to study the optimization problem.
Global Convergence of an Inexact Operator Splitting Method for Monotone Variational Inequalities

Zhili Ge,Gang Qian,Deren Han

DOI: https://doi.org/10.3934/jimo.2011.7.1013

2011-01-01

Journal of Industrial and Management Optimization

Abstract:Recently, Han (Han D, Inexact operator splitting methods with self-adaptive strategy for variational inequality problems, Journal of Optimization Theory and Applications 132, 227-243 (2007)) proposed an inexact operator splitting method for solving variational inequality problems. It has advantage over the classical operator splitting method of Douglas-Peaceman-Rachford-Varga operator splitting methods (DPRV methods) and some of their variants, since it adopts a very flexible approximate rule in solving the subproblem in each iteration. However, its convergence is established under some what stringent condition that the underlying mapping F is strongly monotone. In this paper, we mainly establish the global convergence of the method under weaker condition that the underlying mapping F is monotone, which extends the fields of applications of the method relatively. Some numerical results are also presented to illustrate the method.
Alternating Direction Augmented Lagrangian Methods for Semidefinite Programming

Zaiwen Wen,Donald Goldfarb,Wotao Yin

DOI: https://doi.org/10.1007/s12532-010-0017-1

2010-01-01

Mathematical Programming Computation

Abstract:We present an alternating direction dual augmented Lagrangian method for solving semidefinite programming (SDP) problems in standard form. At each iteration, our basic algorithm minimizes the augmented Lagrangian function for the dual SDP problem sequentially, first with respect to the dual variables corresponding to the linear constraints, and then with respect to the dual slack variables, while in each minimization keeping the other variables fixed, and then finally it updates the Lagrange multipliers (i.e., primal variables). Convergence is proved by using a fixed-point argument. For SDPs with inequality constraints and positivity constraints, our algorithm is extended to separately minimize the dual augmented Lagrangian function over four sets of variables. Numerical results for frequency assignment, maximum stable set and binary integer quadratic programming problems demonstrate that our algorithms are robust and very efficient due to their ability or exploit special structures, such as sparsity and constraint orthogonality in these problems.
An Iterative Algorithm for Solving Generalized Variational Inequalities and Fixed Points Problems

Yonghong Yao,J. Yao,M. Postolache

DOI: https://doi.org/10.3390/MATH7010061

IF: 2.4

2019-01-08

Mathematics

Abstract:In this paper, a generalized variational inequality and fixed points problem is presented. An iterative algorithm is introduced for finding a solution of the generalized variational inequalities and fixed point of two quasi-pseudocontractive operators under a nonlinear transformation. Strong convergence of the suggested algorithm is demonstrated.

Mathematics
Modified Extragradient Methods for Solving Variational Inequalities

Abdellah Bnouhachem,M. H. Xu,Xiao-Ling Fu,Sheng Zhaohan

DOI: https://doi.org/10.1016/j.camwa.2008.10.065

IF: 3.218

2008-01-01

Computers & Mathematics with Applications

Abstract:In this paper, we propose two methods for solving variational inequalities. In the first method, we modified the extragradient method by using a new step size while the second method can be viewed as an extension of the first one by performing an additional projection step at each iteration and another optimal step length is employed to reach substantial progress in each iteration. Under certain conditions, the global convergence of two methods is proved. Preliminary numerical experiments are included to illustrate the efficiency of the proposed methods.

Decomposition Method Using Augmented Lagrangian for Solving Systems of Variational Inequalities

Augmented Lagrangian Theory, Duality and Decomposition Methods for Variational Inequality Problems

Convergence Analysis of General Decomposition Algorithm for Solving General Constrained Variational Inequality Problems

A New Decomposition Method for Variational Inequalities with Linear Constraints

A DECOMPOSITION AUGMENTED LAGRANGIAN METHOD FOR LOW-RANK SEMIDEFINITE PROGRAMMING

Parallel inertial forward–backward splitting methods for solving variational inequality problems with variational inclusion constraints

Variance reduced forward-reflected-backward algorithm for solving nonconvex finite-sum mixed variational inequalities

An Operator Splitting Method for Variational Inequalities with Partially Unknown Mappings

Some iterative algorithms for solving mixed variational inequalities

Variable dimensional algorithm for variational inequalities

An Augmented Lagrangian Deep Learning Method for Variational Problems with Essential Boundary Conditions

A New Penalty Dual-Primal Augmented Lagrangian Method and Its Extensions

A novel method for hierarchical variational inequality with split common fixed point constraint

Two Novel Algorithms for Solving Variational Inequality Problems Governed by Fixed Point Problems and Their Applications

A New Insight on Augmented Lagrangian Method with Applications in Machine Learning

An Augmented Lagrangian Approach to Conically Constrained Non-monotone Variational Inequality Problems

Iterative Algorithms for Variational Inequality and Equilibrium Problems with Applications.

Global Convergence of an Inexact Operator Splitting Method for Monotone Variational Inequalities

Alternating Direction Augmented Lagrangian Methods for Semidefinite Programming

An Iterative Algorithm for Solving Generalized Variational Inequalities and Fixed Points Problems

Modified Extragradient Methods for Solving Variational Inequalities