Parallel Branch and Bound Algorithm for Computing Maximal Structured Singular Value

X Chen,K Zhou

DOI: https://doi.org/10.1049/ip-cta:20030632

2003-01-01

IEE Proceedings - Control Theory and Applications

Abstract:In this paper, we have developed a parallel branch and bound algorithm which computes the maximal structured singular value μ without tightly bounding μ for each frequency and thus significantly reduce the computational complexity.

What problem does this paper attempt to address?

A parallel frequency sweeping algorithm for computing maximal structured singular value μ1

Xinjia Chen,Kemin Zhou

DOI: https://doi.org/10.1109/CDC.1999.830906

1999-01-01

Abstract:Proposes a parallel frequency sweeping algorithm which computes the maximal structured singular value μ without tightly bounding μ for each frequency and thus significantly reduces the computational complexity
Improved upper bounds for the structured singular value

Packard, A.,Kemin Zhou

DOI: https://doi.org/10.1109/CDC.1989.70262

1989-01-01

Abstract:The computation of the structured singular value, μ, a useful tool for matrix perturbation problems, is addressed. Computation of μ is difficult, and usually upper and lower bounds are all that can be reliably computed. In general, there is a gap between the computable upper bound and the structured singular value. An intuitive fix aimed at reducing this gap is presented
A Graphic Structure Based Branch-and-bound Algorithm for Complex Quadratic Optimization and Applications to Magnitude Least-Square Problem

Cheng Lu,Jitao Ma,Zhibin Deng,Wenxun Xing

DOI: https://doi.org/10.1007/s10898-023-01305-9

2024-01-01

Journal of Global Optimization

Abstract:In this paper, we propose a semidefinite relaxation based branch-and-bound algorithm to the unit-modulus constrained complex quadratic programming problem, which has broad applications in signal processing and wireless communications. Our research is motivated from the potential application of the magnitude least-square problem, which possesses the so-called block-arrow sparsity pattern. We discuss how to reduce the worst-case complexity of the proposed branch-and-bound algorithm by exploiting this special sparsity pattern. Numerical results are presented to show the effectiveness of the proposed algorithm.
A Parallel Method With Hybrid Algorithms For Mixed Integer Nonlinear Programming

Kai Zhou,Wei Wan,Xi Chen,Zhijiang Shao,Lorenz T. Biegler

DOI: https://doi.org/10.1016/B978-0-444-63234-0.50046-4

2013-01-01

Abstract:This study aims at improving the solution efficiency of Mixed Integer Nonlinear Programming (MINLP) through parallelism. Unlike most conventional parallel implementations of MINLP solvers, which utilize multi-threads to share the burden in the serial mode, the proposed method combines hybrid algorithms running on different threads. Two types of algorithms are designed in a parallel structure. One is the Quesada and Grossman's LP/NLP based branch and bound algorithm (QG); the other is Tabu Search (TS). The proposed method attempts to minimize the search space through continuous communication and exchange of intermediate results from each thread. Three kinds of information are exchanged between the two threads. First, the best solution in TS, if feasible, serves as a valid upper bound for QG. Second, new approximations which can further tighten the lower bound of QG can be generated at nodes provided by the TS. Third, strong branching in QG may fix some integer variables, which can help reduce the search space of TS. Both threads can thus benefit from the exchanged information in the hybrid method. Numerical results show that solution time can be greatly reduced for the tested MINLP. In addition, complexity analysis of the parallel approach suggests that the proposed method has the potential for superlinear speedup.
An optimal algorithm to compute maximal relaxations in parallel

Mingyue Wang,Na Wang,Chi Zhang,Dehong Ma,Zhanshan Li

DOI: https://doi.org/10.1109/ICNC.2014.6975865

2014-01-01

Abstract:The study of constraint satisfaction problems touches many aspects of artificial intelligence. It is the premise of dominant interactive constraint satisfaction algorithms that users' preference has complete order. To accord more with practical situation, Haijiao Shen [3] did some research on the situation in which users' preferences have partial order and put forward the related algorithm in 2011. By introducing a constraint set M to decrease the spread of redundancy, we optimize her algorithm. We also prove the validity of our new algorithm and test it on some benchmarks. It is indicated by test result that the optimized algorithm MulExp can save up to 18% of examine times and up to 12% of solving time, which greatly increases the efficiency of solving process.
Limited Memory Block Krylov Subspace Optimization for Computing Dominant Singular Value Decompositions.

Xin Liu,Zaiwen Wen,Yin Zhang

DOI: https://doi.org/10.1137/120871328

IF: 2.968

2013-01-01

SIAM Journal on Scientific Computing

Abstract:In many data-intensive applications, the use of principal component analysis and other related techniques is ubiquitous for dimension reduction, data mining, or other transformational purposes. Such transformations often require efficiently, reliably, and accurately computing dominant singular value decompositions (SVDs) of large and dense matrices. In this paper, we propose and study a subspace optimization technique for significantly accelerating the classic simultaneous iteration method. We analyze the convergence of the proposed algorithm and numerically compare it with several state-of-the-art SVD solvers under the MATLAB environment. Extensive computational results show that on a wide range of large unstructured dense matrices, the proposed algorithm can often provide improved efficiency or robustness over existing algorithms.
An Enhanced Branch-and-bound Algorithm for Bilevel Integer Linear Programming.

Shaonan Liu,Mingzheng Wang,Nan Kong,Xiangpei Hu

DOI: https://doi.org/10.1016/j.ejor.2020.10.002

IF: 6.4

2020-01-01

European Journal of Operational Research

Abstract:Bilevel integer linear programming (BILP) problems have been studied for decades. Many exact algorithms have been proposed in recent years for small- or medium-sized instances. However, few of these algorithms were shown to be efficient on large-sized instances. In this paper, we present an enhanced branch-and-bound algorithm for a class of BILP problems, which can discard a subspace from the search space in each iteration larger than that in a benchmark branch-and-bound algorithm. The corresponding enhanced branching rule can efficiently slow down the creation of new node problems so as to significantly reduce the computation time. Our scheme may be suboptimal if the lower-level problem is not unique optimal as the enhanced branching rule may discard bilevel feasible solutions that may turn out to be optimal to the bilevel programming. We present computational studies to evaluate the algorithm speedup and solution quality of our algorithm, compared with state-of-the-art algorithms from the literature on a large testbed of general BILP instances, some of which are still unsolved. The computational results show that our enhanced branching rule can achieve significant speedup on the benchmark branching rule with satisfying solution quality. In particular, our algorithm shows superior performance on large-sized BILP instances with a relatively complex lower-level problem. (C) 2020 Elsevier B.V. All rights reserved.
A Novel Fully Hardware-Implemented SVD Solver Based on Ultra-Parallel BCV Jacobi Algorithm

Tang Hu,Xiangdi Li,Xiao Yu,Songnan Ren,Li Yan,Xuyang Bai,Zhiwei Xu,Shiqiang Zhu

DOI: https://doi.org/10.1109/tcsii.2022.3200750

2022-01-01

IEEE Transactions on Circuits & Systems II Express Briefs

Abstract:Efficient FPGA-based floating-point singular value decomposition (SVD) is challenging for its enormous complexity with the rapid growth of the matrix dimension. Numerous hardware architectures have been proposed to improve the performance of SVD by increasing capacity of computation units, reusing data, and enhancing bandwidth. These designs, however, are not optimum due to their low parallelism, poor data access efficiency, and inferior iterations scheduling. In this express, we propose a block column vector Hestenes-Jacobi (BCV Jacobi) algorithm that decomposes an arbitrary large matrix into several blocks, enhances the access efficiency by customizing the distinctive data structure, and improves the system-level parallelism by simplifying the iteration scheduling. The proposed BCV Jacobi algorithm also achieves better scalability and efficiency. Experimental results show that the performance of the proposed FPGA based SVD processor is superior to other SVD implementations in terms of parallelism, data access efficiency, matrix size, and execution time. When compared with state of the art SVD accelerator engine, the proposed algorithm speeds up the runtime over $2{\times }$ on average.
A New Global Algorithm for Max-Cut Problem with Chordal Sparsity

Cheng Lu,Zhibin Deng,Shu-Cherng Fang,Wenxun Xing

DOI: https://doi.org/10.1007/s10957-023-02195-3

2023-01-01

Journal of Optimization Theory and Applications

Abstract:In this paper, we develop a semidefinite relaxation-based branch-and-bound algorithm that exploits the chordal sparsity patterns of the max-cut problem. We first study how the chordal sparsity pattern affects the hardness of a max-cut problem. To do this, we derive a polyhedral relaxation based on the clique decomposition of the chordal sparsity patterns and prove some sufficient conditions for the tightness of this polyhedral relaxation. The theoretical results show that the max-cut problem is easy to solve when the sparsity pattern embedded in the problem has a small treewidth and the number of vertices in the intersection of maximal cliques is small. Based on the theoretical results, we propose a new branching rule called hierarchy branching rule, which utilizes the tree decomposition of the sparsity patterns. We also analyze how the proposed branching rule affects the chordal sparsity patterns embedded in the problem, and explain why it can be effective. The numerical experiments show that the proposed algorithm is superior to those known algorithms using classical branching rules and the state-of-the-art solver BiqCrunch on most instances with sparsity patterns arisen in practical applications.
Practical Parallel Algorithms for Non-Monotone Submodular Maximization

Shuang Cui,Kai Han,Jing Tang,He Huang,Xueying Li,Aakas Zhiyuli,Hanxiao Li

2023-08-21

Abstract:Submodular maximization has found extensive applications in various domains within the field of artificial intelligence, including but not limited to machine learning, computer vision, and natural language processing. With the increasing size of datasets in these domains, there is a pressing need to develop efficient and parallelizable algorithms for submodular maximization. One measure of the parallelizability of a submodular maximization algorithm is its adaptive complexity, which indicates the number of sequential rounds where a polynomial number of queries to the objective function can be executed in parallel. In this paper, we study the problem of non-monotone submodular maximization subject to a knapsack constraint, and propose the first combinatorial algorithm achieving an $(8+\epsilon)$-approximation under $\mathcal{O}(\log n)$ adaptive complexity, which is \textit{optimal} up to a factor of $\mathcal{O}(\log\log n)$. Moreover, we also propose the first algorithm with both provable approximation ratio and sublinear adaptive complexity for the problem of non-monotone submodular maximization subject to a $k$-system constraint. As a by-product, we show that our two algorithms can also be applied to the special case of submodular maximization subject to a cardinality constraint, and achieve performance bounds comparable with those of state-of-the-art algorithms. Finally, the effectiveness of our approach is demonstrated by extensive experiments on real-world applications.

Data Structures and Algorithms,Machine Learning
23 European Symposium on Computer Aided Process Engineering: A Parallel Method with Hybrid Algorithms for Mixed Integer Nonlinear Programming

Kai Zhou,Wei Wan,Xi Chen,Zhijiang Shao,Lorenz T Biegler

2013-01-01

Abstract:This study aims at improving the solution efficiency of Mixed Integer Nonlinear Programming (MINLP) through parallelism. Unlike most conventional parallel implementations of MINLP solvers, which utilize multi-threads to share the burden in the serial mode, the proposed method combines hybrid algorithms running on different threads. Two types of algorithms are designed in a parallel structure. One is the Quesada and Grossman's LP/NLP based branch and bound algorithm (QG); the other is Tabu Search (TS). The proposed method attempts to minimize the search space through continuous communication and exchange of intermediate results from each thread. Three kinds of information are exchanged between the two threads. First, the best solution in TS, if feasible, serves as a valid upper bound for QG. Second, new approximations which can further tighten the lower bound of QG can be generated at nodes provided by the TS. Third, strong branching in QG may fix some integer variables, which can help reduce the search space of TS. Both threads can thus benefit from the exchanged information in the hybrid method. Numerical results show that solution time can be greatly reduced for the tested MINLP. In addition, complexity analysis of the parallel approach suggests that the proposed method has the potential for superlinear speedup.
Best of Both Worlds: Practical and Theoretically Optimal Submodular Maximization in Parallel

Yixin Chen,Tonmoy Dey,Alan Kuhnle

2024-08-20

Abstract:For the problem of maximizing a monotone, submodular function with respect to a cardinality constraint $k$ on a ground set of size $n$, we provide an algorithm that achieves the state-of-the-art in both its empirical performance and its theoretical properties, in terms of adaptive complexity, query complexity, and approximation ratio; that is, it obtains, with high probability, query complexity of $O(n)$ in expectation, adaptivity of $O(\log(n))$, and approximation ratio of nearly $1-1/e$. The main algorithm is assembled from two components which may be of independent interest. The first component of our algorithm, LINEARSEQ, is useful as a preprocessing algorithm to improve the query complexity of many algorithms. Moreover, a variant of LINEARSEQ is shown to have adaptive complexity of $O( \log (n / k) )$ which is smaller than that of any previous algorithm in the literature. The second component is a parallelizable thresholding procedure THRESHOLDSEQ for adding elements with gain above a constant threshold. Finally, we demonstrate that our main algorithm empirically outperforms, in terms of runtime, adaptive rounds, total queries, and objective values, the previous state-of-the-art algorithm FAST in a comprehensive evaluation with six submodular objective functions.

Data Structures and Algorithms,Machine Learning
An Effective Branch-and-Bound Algorithm with New Bounding Methods for the Maximum $s$-Bundle Problem

Jinghui Xue,Jiongzhi Zheng,Mingming Jin,Kun He

2024-02-06

Abstract:The Maximum s-Bundle Problem (MBP) addresses the task of identifying a maximum s-bundle in a given graph. A graph G=(V, E) is called an s-bundle if its vertex connectivity is at least |V|-s, where the vertex connectivity equals the minimum number of vertices whose deletion yields a disconnected or trivial graph. MBP is NP-hard and holds relevance in numerous realworld scenarios emphasizing the vertex connectivity. Exact algorithms for MBP mainly follow the branch-and-bound (BnB) framework, whose performance heavily depends on the quality of the upper bound on the cardinality of a maximum s-bundle and the initial lower bound with graph reduction. In this work, we introduce a novel Partition-based Upper Bound (PUB) that leverages the graph partitioning technique to achieve a tighter upper bound compared to existing ones. To increase the lower bound, we propose to do short random walks on a clique to generate larger initial solutions. Then, we propose a new BnB algorithm that uses the initial lower bound and PUB in preprocessing for graph reduction, and uses PUB in the BnB search process for branch pruning. Extensive experiments with diverse s values demonstrate the significant progress of our algorithm over state-of-the-art BnB MBP algorithms. Moreover, our initial lower bound can also be generalized to other relaxation clique problems.

Data Structures and Algorithms,Discrete Mathematics,Machine Learning
Practical and Parallelizable Algorithms for Non-Monotone Submodular Maximization with Size Constraint

Yixin Chen,Alan Kuhnle

DOI: https://doi.org/10.1613/jair.1.14323

2024-02-20

Abstract:We present combinatorial and parallelizable algorithms for maximization of a submodular function, not necessarily monotone, with respect to a size constraint. We improve the best approximation factor achieved by an algorithm that has optimal adaptivity and nearly optimal query complexity to $0.193 - \varepsilon$. The conference version of this work mistakenly employed a subroutine that does not work for non-monotone, submodular functions. In this version, we propose a fixed and improved subroutine to add a set with high average marginal gain, ThreshSeq, which returns a solution in $O( \log(n) )$ adaptive rounds with high probability. Moreover, we provide two approximation algorithms. The first has approximation ratio $1/6 - \varepsilon$, adaptivity $O( \log (n) )$, and query complexity $O( n \log (k) )$, while the second has approximation ratio $0.193 - \varepsilon$, adaptivity $O( \log^2 (n) )$, and query complexity $O(n \log (k))$. Our algorithms are empirically validated to use a low number of adaptive rounds and total queries while obtaining solutions with high objective value in comparison with state-of-the-art approximation algorithms, including continuous algorithms that use the multilinear extension.

Data Structures and Algorithms,Machine Learning
Average Optimal Branch-and-Bound Algorithm on Distributed Memory Systems

Jigang Wu,Xianchao Zhang,Xing Xie,Guoliang Chen

DOI: https://doi.org/10.1109/hpc.2000.843507

2000-01-01

Abstract:In this paper, a new data structure called string-queue is proposed in order to implement more efficiently the selection rule and the elimination rule of the general branch-and-bound algorithm. A new general parallel branch-and-bound algorithm on a distributed memory multiprocessor system is also presented. Its communication complexity in single iteration is down to its lower bound O(p) on a 2D mesh network. Both theoretical analysis and experimental results show that its average computational complexity is nearly in its lower bound.
Maximal Clique Enumeration with Hybrid Branching and Early Termination

Kaixin Wang,Kaiqiang Yu,Cheng Long

2024-12-11

Abstract:Maximal clique enumeration (MCE) is crucial for tasks like community detection and biological network analysis. Existing algorithms typically adopt the branch-and-bound framework with the vertex-oriented Bron-Kerbosch (BK) branching strategy, which forms the sub-branches by expanding the partial clique with a vertex. In this paper, we present a novel approach called HBBMC, a hybrid framework combining vertex-oriented BK branching and edge-oriented BK branching, where the latter adopts a branch-and-bound framework which forms the sub-branches by expanding the partial clique with an edge. This hybrid strategy enables more effective pruning and helps achieve a worst-case time complexity better than the best known one under a condition that holds for the majority of real-world graphs. To further enhance efficiency, we introduce an early termination technique, which leverages the topological information of the graphs and constructs the maximal cliques directly without branching. Our early termination technique is applicable to all branch-and-bound frameworks. Extensive experiments demonstrate the superior performance of our techniques.

Databases
The Maximum Common Subgraph Problem: A Parallel and Multi-Engine Approach

Stefano Quer,Andrea Marcelli,Giovanni Squillero

DOI: https://doi.org/10.3390/computation8020048

2020-05-18

Computation

Abstract:The maximum common subgraph of two graphs is the largest possible common subgraph, i.e., the common subgraph with as many vertices as possible. Even if this problem is very challenging, as it has been long proven NP-hard, its countless practical applications still motivates searching for exact solutions. This work discusses the possibility to extend an existing, very effective branch-and-bound procedure on parallel multi-core and many-core architectures. We analyze a parallel multi-core implementation that exploits a divide-and-conquer approach based on a thread pool, which does not deteriorate the original algorithmic efficiency and it minimizes data structure repetitions. We also extend the original algorithm to parallel many-core GPU architectures adopting the CUDA programming framework, and we show how to handle the heavily workload-unbalance and the massive data dependency. Then, we suggest new heuristics to reorder the adjacency matrix, to deal with “dead-ends”, and to randomize the search with automatic restarts. These heuristics can achieve significant speed-ups on specific instances, even if they may not be competitive with the original strategy on average. Finally, we propose a portfolio approach, which integrates all the different local search algorithms as component tools; such portfolio, rather than choosing the best tool for a given instance up-front, takes the decision on-line. The proposed approach drastically limits memory bandwidth constraints and avoids other typical portfolio fragility as CPU and GPU versions often show a complementary efficiency and run on separated platforms. Experimental results support the claims and motivate further research to better exploit GPUs in embedded task-intensive and multi-engine parallel applications.
A Criterion Space Branch-and-Cut Algorithm for Mixed Integer Bilinear Maximum Multiplicative Programs

Vahid Mahmoodian,Iman Dayarian,Payman Ghasemi Saghand,Yu Zhang,Hadi Charkhgard

DOI: https://doi.org/10.1287/ijoc.2021.1097

IF: 3.288

2022-01-04

INFORMS Journal on Computing

Abstract:This study introduces a branch-and-bound algorithm to solve mixed-integer bilinear maximum multiplicative programs (MIBL-MMPs). This class of optimization problems arises in many applications, such as finding a Nash bargaining solution (Nash social welfare optimization), capacity allocation markets, reliability optimization, etc. The proposed algorithm applies multiobjective optimization principles to solve MIBL-MMPs exploiting a special characteristic in these problems. That is, taking each multiplicative term in the objective function as a dummy objective function, the projection of an optimal solution of MIBL-MMPs is a nondominated point in the space of dummy objectives. Moreover, several enhancements are applied and adjusted to tighten the bounds and improve the performance of the algorithm. The performance of the algorithm is investigated by 400 randomly generated sample instances of MIBL-MMPs. The obtained result is compared against the outputs of the mixed-integer second order cone programming (SOCP) solver in CPLEX and a state-of-the-art algorithm in the literature for this problem. Our analysis on this comparison shows that the proposed algorithm outperforms the fastest existing method, that is, the SOCP solver, by a factor of 6.54 on average. Summary of Contribution: The scope of this paper is defined over a class of mixed-integer programs, the so-called mixed-integer bilinear maximum multiplicative programs (MIBL-MMPs). The importance of MIBL-MMPs is highlighted by the fact that they are encountered in applications, such as Nash bargaining, capacity allocation markets, reliability optimization, etc. The mission of the paper is to introduce a novel and effective criterion space branch-and-cut algorithm to solve MIBL-MMPs by solving a finite number of single-objective mixed-integer linear programs. Starting with an initial set of primal and dual bounds, our proposed approach explores the efficient set of the multiobjective problem counterpart of the MIBL-MMP through a criterion space–based branch-and-cut paradigm and iteratively improves the bounds using a branch-and-bound scheme. The bounds are obtained using novel operations developed based on Chebyshev distance and piecewise McCormick envelopes. An extensive computational study demonstrates the efficacy of the proposed algorithm.

computer science, interdisciplinary applications,operations research & management science
A Branch and Bound Algorithm for Discrete Linear Portfolio Selection Problems

Yi Jun,Sun Xiaoling

DOI: https://doi.org/10.3969/j.issn.1007-6093.2008.04.013

2008-01-01

Abstract:In this paper we introduce a class of new discrete mean-absolute devi- ation model and minmax model with concave transaction cost.A branch and bound algorithm is proposed for solving these models.Numerical experiment is carried out for testing problems with data from randomly generated and U.S.Nasdaq stock market.The numerical analysis indicates that for a given level of return,the discrete mean-absolute deviation model outperforms the discrete minmax model in term of risk control while the discrete minmax model is preferable when computing efficiency is concerned.

Parallel Branch and Bound Algorithm for Computing Maximal Structured Singular Value

A parallel frequency sweeping algorithm for computing maximal structured singular value μ1

Improved upper bounds for the structured singular value

A Graphic Structure Based Branch-and-bound Algorithm for Complex Quadratic Optimization and Applications to Magnitude Least-Square Problem

A Parallel Method With Hybrid Algorithms For Mixed Integer Nonlinear Programming

An optimal algorithm to compute maximal relaxations in parallel

Limited Memory Block Krylov Subspace Optimization for Computing Dominant Singular Value Decompositions.

An Enhanced Branch-and-bound Algorithm for Bilevel Integer Linear Programming.

A Novel Fully Hardware-Implemented SVD Solver Based on Ultra-Parallel BCV Jacobi Algorithm

A New Global Algorithm for Max-Cut Problem with Chordal Sparsity

Practical Parallel Algorithms for Non-Monotone Submodular Maximization

23 European Symposium on Computer Aided Process Engineering: A Parallel Method with Hybrid Algorithms for Mixed Integer Nonlinear Programming

Best of Both Worlds: Practical and Theoretically Optimal Submodular Maximization in Parallel

An Effective Branch-and-Bound Algorithm with New Bounding Methods for the Maximum $s$-Bundle Problem

Practical and Parallelizable Algorithms for Non-Monotone Submodular Maximization with Size Constraint

Average Optimal Branch-and-Bound Algorithm on Distributed Memory Systems

Maximal Clique Enumeration with Hybrid Branching and Early Termination

The Maximum Common Subgraph Problem: A Parallel and Multi-Engine Approach

A Criterion Space Branch-and-Cut Algorithm for Mixed Integer Bilinear Maximum Multiplicative Programs

A Branch and Bound Algorithm for Discrete Linear Portfolio Selection Problems