Block Pivot Method Based on Kernel Matrix in LP

姜波,蓝伯雄

DOI: https://doi.org/10.3969/j.issn.1007-3221.2008.02.001

2008-01-01

Abstract:Based on the Kernel matrix in linear programming,this paper deeply researches the block pivot in the simplex method.We prove the feasibility of the block pivot on the theory of Kuhn-Tucker condition in LP problem and present a general block pivot algorithm,which finds a new way for the simplex method.

What problem does this paper attempt to address?

A Primal-Dual Algorithm Based on Kernel Matrix in LP

JIANG Bo,LAN Bo-xiong

DOI: https://doi.org/10.3969/j.issn.1007-3221.2008.05.001

2008-01-01

Abstract:This paper analyzes the kernel matrix in the linear programming problem and proposes a new revised prime-dual algorithm based on the kernel matrix.Both low memory size and small-scale computation are showed in the new algorithm,on the other hand,a new pivot rule,which is an exterior point method,also makes the new algorithm more efficient.According to the experiment result,this algorithm is efficient in pivot efficiency and memory consuming.
The LU Decomposition of Dynamic Kernel Matrix

姜波,蓝伯雄

DOI: https://doi.org/10.3969/j.issn.1006-3897.2008.06.011

2008-01-01

Abstract:With the development of linear programming,the decomposition technology has been playing a significant role in the implementation of the algorithm.In the traditional simplex method,product form of inverse(PFI) and LU decomposition keep the sparsity of the A matrix and decrease the round-off error.Decomposition of dynamic matrix becomes a new research topic as soon as the LP dynamic factorization and kernel matrix came out.This paper discusses the decomposition and storage of kernel matrix by applying the LU decomposition method,which makes a good result on matrix sparsity and round-off error.Additionally,the permutation elimination method proposed in this paper makes the LU update process more efficient.
Sparsity Prevention Pivoting Method for Linear Programming.

Peiqiang Li,Qiyuan Li,Canbing Li,Bin Zhou,Yijia Cao,Qiuwei Wu,Baling Fang

DOI: https://doi.org/10.1109/access.2018.2817571

IF: 3.9

2018-01-01

IEEE Access

Abstract:When the simplex algorithm is used to calculate a linear programming (LP) problem, if the matrix is a sparse matrix, it will be possible to lead to many zero-length calculation steps, and even iterative cycle will appear. To deal with the problem, a new pivoting method is proposed in this paper. The principle of this method is to avoid choosing the row which the value of the element in the right side of constraint expression for LP in this row is zero as the row of the pivot element to make the matrix in LP density and ensure that most subsequent steps will improve the value of the objective function. One step following this principle is inserted in the existing LP algorithm to reselect the pivot element. Both the conditions for inserting this step and the maximum number of allowed insertion steps are determined. In the case study, taking several numbers of LP problems as examples, the results indicate that this method can effectively improve the efficiency of LP for the sparse matrix.
A double-pivot simplex algorithm and its upper bounds of the iteration numbers

Yaguang Yang

DOI: https://doi.org/10.1007/s40687-020-00235-2

2020-05-01

Abstract:In this paper, a double-pivot simplex method is proposed. Two upper bounds of iteration numbers are derived. Applying one of the bounds to some special linear programming (LP) problems, such as LP with a totally unimodular matrix and Markov Decision Problem (MDP) with a fixed discount rate, indicates that the double-pivot simplex method solves these problems in a strongly polynomial time. A variant of Klee-Minty cube is used to show that the estimated bounds of the iteration numbers are very tight. Numerical test on three variants of Klee-Minty cubes is performed for the problems with sizes as big as $200$ constraints and $400$ variables. Dantzig's simplex method cannot handle Klee-Minty cube problem with $200$ constraints because it needs about $2^{200} \approx 10^{60}$ iterations. But the proposed algorithm performs extremely good for all three variants.

Optimization and Control
Pivoting in Linear Complementarity: Two Polynomial-Time Cases

Jan Foniok,Komei Fukuda,Bernd Gärtner,Hans-Jakob Lüthi

DOI: https://doi.org/10.1007/s00454-009-9182-2

2009-03-29

Abstract:We study the behavior of simple principal pivoting methods for the P-matrix linear complementarity problem (P-LCP). We solve an open problem of Morris by showing that Murty's least-index pivot rule (under any fixed index order) leads to a quadratic number of iterations on Morris's highly cyclic P-LCP examples. We then show that on K-matrix LCP instances, all pivot rules require only a linear number of iterations. As the main tool, we employ unique-sink orientations of cubes, a useful combinatorial abstraction of the P-LCP.

Optimization and Control
Learning to Pivot as a Smart Expert

Tianhao Liu,Shanwen Pu,Dongdong Ge,Yinyu Ye

DOI: https://doi.org/10.1609/aaai.v38i8.28646

2023-08-31

Abstract:Linear programming has been practically solved mainly by simplex and interior point methods. Compared with the weakly polynomial complexity obtained by the interior point methods, the existence of strongly polynomial bounds for the length of the pivot path generated by the simplex methods remains a mystery. In this paper, we propose two novel pivot experts that leverage both global and local information of the linear programming instances for the primal simplex method and show their excellent performance numerically. The experts can be regarded as a benchmark to evaluate the performance of classical pivot rules, although they are hard to directly implement. To tackle this challenge, we employ a graph convolutional neural network model, trained via imitation learning, to mimic the behavior of the pivot expert. Our pivot rule, learned empirically, displays a significant advantage over conventional methods in various linear programming problems, as demonstrated through a series of rigorous experiments.

Optimization and Control
A Monotonic Build-Up Simplex Algorithm for Linear Programming

Kurt M. Anstreicher,Tamás Terlaky

DOI: https://doi.org/10.1287/opre.42.3.556

IF: 2.7

1994-06-01

Operations Research

Abstract:We devise a new simplex pivot rule which has interesting theoretical properties. Beginning with a basic feasible solution, and any nonbasic variable having a negative reduced cost the pivot rule produces a sequence of pivots such that ultimately the originally chosen nonbasic variable enters the basis, and all reduced costs which were originally nonnegative remain nonnegative. The pivot rule thus monotonically builds up to a dual feasible, and hence optimal, basis. A surprising property is that the pivot sequence results in intermediate bases which are neither primal nor dual feasible. We prove the correctness of the procedure, and relate it to other pivoting rules for linear programming.

management,operations research & management science
Paved with good intentions: Analysis of a randomized block Kaczmarz method

Deanna Needell,Joel A. Tropp

DOI: https://doi.org/10.1016/j.laa.2012.12.022

IF: 1.307

2014-01-01

Linear Algebra and its Applications

Abstract:The block Kaczmarz method is an iterative scheme for solving overdetermined least-squares problems. At each step, the algorithm projects the current iterate onto the solution space of a subset of the constraints. This paper describes a block Kaczmarz algorithm that uses a randomized control scheme to choose the subset at each step. This algorithm is the first block Kaczmarz method with an (expected) linear rate of convergence that can be expressed in terms of the geometric properties of the matrix and its submatrices. The analysis reveals that the algorithm is most effective when it is given a good row paving of the matrix, a partition of the rows into well-conditioned blocks. The operator theory literature provides detailed information about the existence and construction of good row pavings. Together, these results yield an efficient block Kaczmarz scheme that applies to many overdetermined least-squares problem.

mathematics, applied
Exponential Lower Bounds for Many Pivot Rules for the Simplex Method

Alexander E. Black

2024-05-08

Abstract:The existence of a pivot rule for the simplex method that guarantees a strongly polynomial run-time is a longstanding, fundamental open problem in the theory of linear programming. The leading pivot rule in theory is the shadow pivot rule, which solves a linear program by projecting the feasible region onto a polygon. It has been shown to perform in expected strongly polynomial time on uniformly random instances and in smoothed analysis. In practice, the pivot rule of choice is the steepest edge rule, which normalizes the set of improving neighbors and then chooses a maximally improving normalized neighbor. Exponential lower bounds are known for both rules in worst-case analysis. However, for the shadow simplex method, all exponential examples were only proven for one choice of projection, and for the steepest edge rule, the lower bounds were only proven for the Euclidean norm. In this work, we construct linear programs for which any choice of projection for shadow rule variants will lead to an exponential run-time and exponential examples for any choice of norm for a steepest edge variant.

Optimization and Control,Combinatorics
On the Number of Degenerate Simplex Pivots

Kirill Kukharenko,Laura Sanità

2024-08-07

Abstract:The simplex algorithm is one of the most popular algorithms to solve linear programs (LPs). Starting at an extreme point solution of an LP, it performs a sequence of basis exchanges (called pivots) that allows one to move to a better extreme point along an improving edge-direction of the underlying polyhedron. A key issue in the simplex algorithm's performance is degeneracy, which may lead to a (potentially long) sequence of basis exchanges which do not change the current extreme point solution. In this paper, we prove that it is always possible to limit the number of consecutive degenerate pivots that the simplex algorithm performs to $n-m-1$, where $n$ is the number of variables and $m$ is the number of equality constraints of a given LP in standard equality form.

Optimization and Control,Combinatorics
Simplex Method for Solving Linear Programs

SHEN Mao-xing,XU Jin

DOI: https://doi.org/10.3321/j.issn:1002-8331.2007.30.030

2007-01-01

Computer Engineering and Applications Journal

Abstract:An association between the elementary transformation of matrix and the concept of initial feasible basis is established,and then,a determination way for initial feasible basis of linear programming is presented.A direct way of the simplex method for solving a linear programming problem is presented,this scheme is concisely,convenience and making the solving procedure clearly in practice.Some examples are given to demonstrate these good characteristics.
On a Fast Deterministic Block Kaczmarz Method for Solving Large-Scale Linear Systems.

Chen Jia-Qi,Huang Zheng-Da

DOI: https://doi.org/10.1007/s11075-021-01143-4

IF: 2.37

2021-01-01

Numerical Algorithms

Abstract:For solving large-scale consistent systems of linear equations by iterative methods, a fast block Kaczmarz method based on a greedy criterion of the row selections is proposed. The method is deterministic and needs not compute the pseudoinverses of submatrices or solve subsystems. It is proved that the method will converge linearly to the unique least-norm solutions of the linear systems. Numerical experiments are given to illustrate that the method is more efficient and yields a significant acceleration in convergence for the tested data.
On the alternating randomized block Kaczmarz method

Nian-Ci Wu,Yang Zhou,Zhaolu Tian

2023-11-01

Abstract:The block Kaczmarz method and its variants are designed for solving the over-determined linear system. They involve iteratively projecting the current point onto the solution space of a subset of constraints. In this work, by alternately dealing with two subproblems (i.e., linear system with multiple right-hand sides) using the block Kaczmarz method, we propose the {\it Alternating Randomized Block Kaczmarz} (ARBK) method to solve the linear matrix equation $AXB=F$, which incorporates a randomized index selection scheme to determine the subset of constraints. The convergence analysis reveals that the ARBK method has a linear convergence rate bounded by an explicit expression. Several numerical studies have been conducted to validate the theoretical findings.

Numerical Analysis
Randomized Complete Pivoting for Solving Symmetric Indefinite Linear Systems

Yuehua Feng,Jianwei Xiao,Ming Gu

DOI: https://doi.org/10.1137/17M1150013

2017-09-30

Abstract:The Bunch-Kaufman algorithm and Aasen's algorithm are two of the most widely used methods for solving symmetric indefinite linear systems, yet they both are known to suffer from occasional numerical instability due to potentially exponential element growth or unbounded entries in the matrix factorization. In this work, we develop a randomized complete pivoting (RCP) algorithm for solving symmetric indefinite linear systems. RCP is comparable to the Bunch-Kaufman algorithm and Aasen's algorithm in computational efficiency, yet enjoys theoretical element growth and bounded entries in the factorization comparable to that of complete-pivoting, up to a theoretical failure probability that exponentially decays with an oversampling parameter. Our finite precision analysis shows that RCP is as numerically stable as Gaussian elimination with complete pivoting, and RCP has been observed to be numerically stable in our extensive numerical experiments.

Numerical Analysis
Orthogonal Block Kaczmarz Algorithm Based on Preprocessing Technology

Yu-Fang Liang,Hou-Biao Li

2024-08-04

Abstract:In this paper, a new simple and fast orthogonal block Kaczmarz (POBK) algorithm based on preprocessing techniques is proposed to solve large-scale sparse linear systems Ax = f. Firstly, the sparse Reverse Cuthill-McKee Algorithm (RCM) is used to preprocess the linear system, and then a new partitioning strategy is proposed to divide the mutually orthogonal blocks into one category, which greatly reduces computational complexity. In addition, we provide a theoretical proof of the convergence of the POBK algorithm and analyze its faster convergence. Finally, numerical simulations demonstrate that our method outperforms the latest Kaczmarz methods such as GRBK, RBK(k), GREBK(k), and aRBK in terms of CPU time.

Numerical Analysis
Pseudo Principal Pivot Transform: The Group Inverse Case

Kavita Bisht,K.C. Sivakumar

DOI: https://doi.org/10.48550/arXiv.1605.01970

2016-05-05

Abstract:In this short note, we prove a formula for the group inverse of a block matrix and consider the pseudo principal pivot transform expressed in terms of group inverses. Extensions of the usual principal pivot transform, where the usual inverse is replaced by the Moore-Penrose inverse, were considered in the literature. The objective here is to derive the corresponding formulae for the group generalized inverse. These are expected to be useful in our future work.

Rings and Algebras
On the optimal pivot path of simplex method for linear programming based on reinforcement learning

Anqi Li,Tiande Guo,Congying Han,Bonan Li,Haoran Li

DOI: https://doi.org/10.1007/s11425-022-2259-1

2023-10-12

Abstract:Based on the existing pivot rules, the simplex method for linear programming is not polynomial in the worst case. Therefore the optimal pivot of the simplex method is crucial. This study proposes the optimal rule to find all shortest pivot paths of the simplex method for linear programming problems based on Monte Carlo tree search (MCTS). Specifically, we first propose the SimplexPseudoTree to transfer the simplex method into tree search mode while avoiding repeated basis variables. Secondly, we propose four reinforcement learning (RL) models with two actions and two rewards to make the Monte Carlo tree search suitable for the simplex method. Thirdly, we set a new action selection criterion to ameliorate the inaccurate evaluation in the initial exploration. It is proved that when the number of vertices in the feasible region is $C_n^m$, our method can generate all the shortest pivot paths, which is the polynomial of the number of variables. In addition, we experimentally validate that the proposed schedule can avoid unnecessary search and provide the optimal pivot path. Furthermore, this method can provide the best pivot labels for all kinds of supervised learning methods to solve linear programming problems.

Optimization and Control
An efficient variant of the greedy block Kaczmarz algorithm for solving large linear systems

Ke Zhang,Hong-Yan Yin,Xiang-Long Jiang

DOI: https://doi.org/10.3934/math.2024122

2023-01-01

AIMS Mathematics

Abstract:By exploiting the concept of row partitioning, we propose an efficient variant of the greedy block Kaczmarz algorithm for solving consistent large linear systems. The number of blocks is determined a priori through numerical experiments. The new algorithm works with a reduced linear system, which dramatically diminishes the computational overhead per iteration. The theoretical result validates that this method converges to the unique least-norm solution of the linear system. The effectiveness of the proposed algorithm is also justified by comparing it with some block Kaczmarz algorithms in extensive numerical experiments.

mathematics, applied
A greedy block Kaczmarz algorithm for solving large-scale linear systems

Yu-Qi Niu,Bing Zheng

DOI: https://doi.org/10.1016/j.aml.2020.106294

IF: 4.294

2020-06-01

Applied Mathematics Letters

Abstract:<p>The block Kaczmarz algorithm is an iterative scheme for solving large-scale linear systems. At each iteration, the process of the algorithm is to orthogonally project the current iteration point onto the solution space of a subset of the constraints. In this paper, we propose a greedy block Kaczmarz algorithm by using a greedy strategy to construct control index set and choosing row submatrix in each iteration. This algorithm has an linear rate of convergence that can be expressed in terms of the geometric properties of the matrix and its row submatrices. The theoretical analysis and numerical results show that the developed algorithm can be more efficient than the greedy randomized Kaczmarz algorithm if parameter <span class="math"><math>η</math></span> is chosen appropriately.</p>

mathematics, applied
Block Matrix and Tensor Randomized Kaczmarz Methods for Linear Feasibility Problems

Minxin Zhang,Jamie Haddock,Deanna Needell

2024-06-18

Abstract:The randomized Kaczmarz methods are a popular and effective family of iterative methods for solving large-scale linear systems of equations, which have also been applied to linear feasibility problems. In this work, we propose a new block variant of the randomized Kaczmarz method, B-MRK, for solving linear feasibility problems defined by matrices. We show that B-MRK converges linearly in expectation to the feasible region.Furthermore, we extend the method to solve tensor linear feasibility problems defined under the tensor t-product. A tensor randomized Kaczmarz (TRK) method, TRK-L, is proposed for solving linear feasibility problems that involve mixed equality and inequality constraints. Additionally, we introduce another TRK method, TRK-LB, specifically tailored for cases where the feasible region is defined by linear equality constraints coupled with bound constraints on the variables. We show that both of the TRK methods converge linearly in expectation to the feasible region. Moreover, the effectiveness of our methods is demonstrated through numerical experiments on various Gaussian random data and applications in image deblurring.

Optimization and Control,Numerical Analysis

Block Pivot Method Based on Kernel Matrix in LP

A Primal-Dual Algorithm Based on Kernel Matrix in LP

The LU Decomposition of Dynamic Kernel Matrix

Sparsity Prevention Pivoting Method for Linear Programming.

A double-pivot simplex algorithm and its upper bounds of the iteration numbers

Pivoting in Linear Complementarity: Two Polynomial-Time Cases

Learning to Pivot as a Smart Expert

A Monotonic Build-Up Simplex Algorithm for Linear Programming

Paved with good intentions: Analysis of a randomized block Kaczmarz method

Exponential Lower Bounds for Many Pivot Rules for the Simplex Method

On the Number of Degenerate Simplex Pivots

Simplex Method for Solving Linear Programs

On a Fast Deterministic Block Kaczmarz Method for Solving Large-Scale Linear Systems.

On the alternating randomized block Kaczmarz method

Randomized Complete Pivoting for Solving Symmetric Indefinite Linear Systems

Orthogonal Block Kaczmarz Algorithm Based on Preprocessing Technology

Pseudo Principal Pivot Transform: The Group Inverse Case

On the optimal pivot path of simplex method for linear programming based on reinforcement learning

An efficient variant of the greedy block Kaczmarz algorithm for solving large linear systems

A greedy block Kaczmarz algorithm for solving large-scale linear systems

Block Matrix and Tensor Randomized Kaczmarz Methods for Linear Feasibility Problems