Algebraic Approach to Approximation

Libor Barto,Silvia Butti,Alexandr Kazda,Caterina Viola,Stanislav Živný

2024-01-27

Abstract:Following the success of the so-called algebraic approach to the study of decision constraint satisfaction problems (CSPs), exact optimization of valued CSPs, and most recently promise CSPs, we propose an algebraic framework for valued promise CSPs.

Computational Complexity,Discrete Mathematics,Logic in Computer Science

What problem does this paper attempt to address?

Algorithms for stochastic optimization with functional or expectation constraints

Guanghui Lan,Zhiqiang Zhou

DOI: https://doi.org/10.48550/arXiv.1604.03887

2016-04-13

Optimization and Control

Abstract:This paper considers the problem of minimizing an expectation function over a closed convex set, coupled with a {\color{black} functional or expectation} constraint on either decision variables or problem parameters. We first present a new stochastic approximation (SA) type algorithm, namely the cooperative SA (CSA), to handle problems with the constraint on devision variables. We show that this algorithm exhibits the optimal ${\cal O}(1/\epsilon^2)$ rate of convergence, in terms of both optimality gap and constraint violation, when the objective and constraint functions are generally convex, where $\epsilon$ denotes the optimality gap and infeasibility. Moreover, we show that this rate of convergence can be improved to ${\cal O}(1/\epsilon)$ if the objective and constraint functions are strongly convex. We then present a variant of CSA, namely the cooperative stochastic parameter approximation (CSPA) algorithm, to deal with the situation when the constraint is defined over problem parameters and show that it exhibits similar optimal rate of convergence to CSA. It is worth noting that CSA and CSPA are primal methods which do not require the iterations on the dual space and/or the estimation on the size of the dual variables. To the best of our knowledge, this is the first time that such optimal SA methods for solving functional or expectation constrained stochastic optimization are presented in the literature.
An Algebraic Approach for Approximity

Yong Wang

DOI: https://doi.org/10.48550/arXiv.1511.09186

2015-11-30

Abstract:Comparison to traditionally accurate computing, approximate computing focuses on the rapidity of the satisfactory solution, but not the unnecessary accuracy of the solution. Approximate bisimularity is the approximate one corresponding to traditionally accurate bisimilarity. Based on the work of distances between basic processes, we propose an algebraic approach for distances between processes to support a whole process calculus CCS, which contains prefix, sum, composition, restriction, relabeling and recursion.

Logic in Computer Science
Approximate algebraic structure

Ben Green

DOI: https://doi.org/10.48550/arXiv.1404.0093

2014-04-01

Abstract:We discuss a selection of recent developments in arithmetic combinatorics having to do with ``approximate algebraic structure'' together with some of their applications.

Number Theory
On Approximability of Satisfiable k-CSPs: V

Amey Bhangale,Subhash Khot,Dor Minzer

2024-08-28

Abstract:We propose a framework of algorithm vs. hardness for all Max-CSPs and demonstrate it for a large class of predicates. This framework extends the work of Raghavendra [STOC, 2008], who showed a similar result for almost satisfiable Max-CSPs. Our framework is based on a new hybrid approximation algorithm, which uses a combination of the Gaussian elimination technique (i.e., solving a system of linear equations over an Abelian group) and the semidefinite programming relaxation. We complement our algorithm with a matching dictator vs. quasirandom test that has perfect completeness. The analysis of our dictator vs. quasirandom test is based on a novel invariance principle, which we call the mixed invariance principle. Our mixed invariance principle is an extension of the invariance principle of Mossel, O'Donnell and Oleszkiewicz [Annals of Mathematics, 2010] which plays a crucial role in Raghavendra's work. The mixed invariance principle allows one to relate 3-wise correlations over discrete probability spaces with expectations over spaces that are a mixture of Guassian spaces and Abelian groups, and may be of independent interest.

Computational Complexity
Smooth approximations: An algebraic approach to CSPs over finitely bounded homogeneous structures

Antoine Mottet,Michael Pinsker

DOI: https://doi.org/10.1145/3689207

IF: 2.269

2024-10-06

Journal of the ACM

Abstract:We introduce the novel machinery of smooth approximations to provide a systematic algebraic approach to the complexity of CSPs over finitely bounded homogeneous structures. We apply smooth approximations to confirm the CSP dichotomy conjecture for first-order reducts of the random tournament and to give new short proofs of the conjecture for various homogeneous graphs including the random graph (STOC’11, ICALP’16, JACM 2015, SICOMP 2019), and for expansions of the order of the rationals (STOC’08, JACM 2009). Apart from obtaining these dichotomy results, we show how our new proof technique allows one to unify and significantly simplify the previous results from the literature. For all but the last structure, we moreover characterize for the first time those CSPs that are solvable by local consistency methods, again using the same machinery.

computer science, information systems, theory & methods, software engineering, hardware & architecture
Sketching approximability of all finite CSPs

Chi-Ning Chou,Alexander Golovnev,Madhu Sudan,Santhoshini Velusamy

DOI: https://doi.org/10.1145/3649435

IF: 2.269

2024-02-29

Journal of the ACM

Abstract:A constraint satisfaction problem (CSP), $\mathsf {Max-CSP}(\mathcal {F}) $ , is specified by a finite set of constraints $\mathcal {F} \subseteq \lbrace [q]^k \rightarrow \lbrace 0,1\rbrace \rbrace $ for positive integers q and k . An instance of the problem on n variables is given by m applications of constraints from $\mathcal {F} $ to subsequences of the n variables, and the goal is to find an assignment to the variables that satisfies the maximum number of constraints. In the ( γ , β )-approximation version of the problem for parameters 0 ≤ β < γ ≤ 1, the goal is to distinguish instances where at least γ fraction of the constraints can be satisfied from instances where at most β fraction of the constraints can be satisfied. In this work, we consider the approximability of this problem in the context of sketching algorithms and give a dichotomy result. Specifically, for every family $\mathcal {F} $ and every β < γ , we show that either a linear sketching algorithm solves the problem in polylogarithmic space, or the problem is not solvable by any sketching algorithm in $o(\sqrt {n}) $ space. In particular, we give non-trivial approximation algorithms using polylogarithmic space for infinitely many constraint satisfaction problems. We also extend previously known lower bounds for general streaming algorithms to a wide variety of problems, and in particular the case of q = k = 2, where we get a dichotomy, and the case when the satisfying assignments of the constraints of $\mathcal {F} $ support a distribution on [ q ] k with uniform marginals. Prior to this work, other than sporadic examples, the only systematic classes of CSPs that were analyzed considered the setting of Boolean variables q = 2, binary constraints k = 2, singleton families $|\mathcal {F}|=1 $ and only considered the setting where constraints are placed on literals rather than variables. Our positive results show wide applicability of bias-based algorithms used previously by [47] and [41], which we extend to include richer norm estimation algorithms, by giving a systematic way to discover biases. Our negative results combine the Fourier analytic methods of [56], which we extend to a wider class of CSPs, with a rich collection of reductions among communication complexity problems that lie at the heart of the negative results. In particular, previous works used Fourier analysis over the Boolean cube to initiate their results and the results seemed particularly tailored to functions on Boolean literals (i.e., with negations). Our techniques surprisingly allow us to get to general q -ary CSPs without negations by appealing to the same Fourier analytic starting point over Boolean hypercubes.

computer science, information systems, theory & methods, software engineering, hardware & architecture
Approximation Algorithms for Combinatorial Optimization with Predictions

Antonios Antoniadis,Marek Eliáš,Adam Polak,Moritz Venzin

2024-11-26

Abstract:We initiate a systematic study of utilizing predictions to improve over approximation guarantees of classic algorithms, without increasing the running time. We propose a systematic method for a wide class of optimization problems that ask to select a feasible subset of input items of minimal (or maximal) total weight. This gives simple (near-)linear time algorithms for, e.g., Vertex Cover, Steiner Tree, Min-Weight Perfect Matching, Knapsack, and Clique. Our algorithms produce optimal solutions when provided with perfect predictions and their approximation ratios smoothly degrade with increasing prediction error. With small enough prediction error we achieve approximation guarantees that are beyond reach without predictions in the given time bounds, as exemplified by the NP-hardness and APX-hardness of many of the above problems. Although we show our approach to be optimal for this class of problems as a whole, there is a potential for exploiting specific structural properties of individual problems to obtain improved bounds; we demonstrate this on the Steiner Tree problem. We conclude with an empirical evaluation of our approach.

Data Structures and Algorithms,Machine Learning
Approximation algorithms for noncommutative CSPs

Eric Culf,Hamoon Mousavi,Taro Spirig

2024-09-29

Abstract:Noncommutative constraint satisfaction problems (NC-CSPs) are higher-dimensional operator extensions of classical CSPs. Despite their significance in quantum information, their approximability remains largely unexplored. A notable example of a noncommutative CSP that is not solvable in polynomial time is NC-Max-$3$-Cut. We present a $0.864$-approximation algorithm for this problem. Our approach extends to a broader class of both classical and noncommutative CSPs. We introduce three key concepts: approximate isometry, relative distribution, and $\ast$-anticommutation, which may be of independent interest.

Quantum Physics,Computational Complexity
An approximation algorithm for a general class of multi-parametric optimization problems

Stephan Helfrich,Arne Herzel,Stefan Ruzika,Clemens Thielen,Helfrich, Stephan,Herzel, Arne,Ruzika, Stefan,Thielen, Clemens

DOI: https://doi.org/10.1007/s10878-022-00902-w

2022-08-30

Journal of Combinatorial Optimization

Abstract:In a widely-studied class of multi-parametric optimization problems, the objective value of each solution is an affine function of real-valued parameters. Then, the goal is to provide an optimal solution set, i.e., a set containing an optimal solution for each non-parametric problem obtained by fixing a parameter vector. For many multi-parametric optimization problems, however, an optimal solution set of minimum cardinality can contain super-polynomially many solutions. Consequently, no polynomial-time exact algorithms can exist for these problems even if $$\textsf {P}=\textsf {NP}$$ . We propose an approximation method that is applicable to a general class of multi-parametric optimization problems and outputs a set of solutions with cardinality polynomial in the instance size and the inverse of the approximation guarantee. This method lifts approximation algorithms for non-parametric optimization problems to their parametric version and provides an approximation guarantee that is arbitrarily close to the approximation guarantee of the approximation algorithm for the non-parametric problem. If the non-parametric problem can be solved exactly in polynomial time or if an FPTAS is available, our algorithm is an FPTAS. Further, we show that, for any given approximation guarantee, the minimum cardinality of an approximation set is, in general, not $$\ell $$ -approximable for any natural number $$\ell $$ less or equal to the number of parameters, and we discuss applications of our results to classical multi-parametric combinatorial optimizations problems. In particular, we obtain an FPTAS for the multi-parametric minimum s-t-cut problem, an FPTAS for the multi-parametric knapsack problem, as well as an approximation algorithm for the multi-parametric maximization of independence systems problem.

mathematics, applied,computer science, interdisciplinary applications
An invitation to the promise constraint satisfaction problem

Andrei Krokhin,Jakub Opršal

DOI: https://doi.org/10.1145/3559736.3559740

2022-08-29

Abstract:The study of the complexity of the constraint satisfaction problem (CSP), centred around the Feder-Vardi Dichotomy Conjecture, has been very prominent in the last two decades. After a long concerted effort and many partial results, the Dichotomy Conjecture has been proved in 2017 independently by Bulatov and Zhuk. At about the same time, a vast generalisation of CSP, called promise CSP, has started to gain prominence. In this survey, we explain the importance of promise CSP and highlight many new very interesting features that the study of promise CSP has brought to light. The complexity classification quest for the promise CSP is wide open, and we argue that, despite the promise CSP being more general, this quest is rather more accessible to a wide range of researchers than the dichotomy-led study of the CSP has been.

Computational Complexity,Logic in Computer Science,Logic
Approximation Schemes Via Sherali-Adams Hierarchy for Dense Constraint Satisfaction Problems and Assignment Problems

Yuichi Yoshida,Yuan Zhou

DOI: https://doi.org/10.1145/2554797.2554836

2014-01-01

Abstract:We consider approximation schemes for the maximum constraint satisfaction problems and the maximum assignment problems. Though they are NP-Hard in general, if the instance is "dense" or "locally dense", then they are known to have approximation schemes that run in polynomial time or quasi-polynomial time. In this paper, we give a unified method of showing these approximation schemes based on the Sherali-Adams linear programming relaxation hierarchy. We also use our linear programming-based framework to show new algorithmic results on the optimization version of the hypergraph isomorphism problem.
Approximating Bounded Occurrence Ordering CSPs

Venkatesan Guruswami,Yuan Zhou

DOI: https://doi.org/10.1007/978-3-642-32512-0_14

2012-01-01

Abstract:A theorem of Håstad shows that for every constraint satisfaction problem (CSP) over a fixed size domain, instances where each variable appears in at most O(1) constraints admit a non-trivial approximation algorithm, in the sense that one can beat (by an additive constant) the approximation ratio achieved by the naive algorithm that simply picks a random assignment. We consider the analogous question for ordering CSPs, where the goal is to find a linear ordering of the variables to maximize the number of satisfied constraints, each of which stipulates some restriction on the local order of the involved variables. It was shown recently that without the bounded occurrence restriction, for every ordering CSP it is Unique Games-hard to beat the naive random ordering algorithm.
Pliability and Approximating Max-CSPs

Miguel Romero,Marcin Wrochna,Stanislav Živný

DOI: https://doi.org/10.1145/3626515

2023-09-17

Abstract:We identify a sufficient condition, treewidth-pliability, that gives a polynomial-time algorithm for an arbitrarily good approximation of the optimal value in a large class of Max-2-CSPs parameterised by the class of allowed constraint graphs (with arbitrary constraints on an unbounded alphabet). Our result applies more generally to the maximum homomorphism problem between two rational-valued structures. The condition unifies the two main approaches for designing a polynomial-time approximation scheme. One is Baker's layering technique, which applies to sparse graphs such as planar or excluded-minor graphs. The other is based on Szemerédi's regularity lemma and applies to dense graphs. We extend the applicability of both techniques to new classes of Max-CSPs. On the other hand, we prove that the condition cannot be used to find solutions (as opposed to approximating the optimal value) in general. Treewidth-pliability turns out to be a robust notion that can be defined in several equivalent ways, including characterisations via size, treedepth, or the Hadwiger number. We show connections to the notions of fractional-treewidth-fragility from structural graph theory, hyperfiniteness from the area of property testing, and regularity partitions from the theory of dense graph limits. These may be of independent interest. In particular we show that a monotone class of graphs is hyperfinite if and only if it is fractionally-treewidth-fragile and has bounded degree.

Discrete Mathematics,Data Structures and Algorithms,Combinatorics
Approximate Evaluation of Quantitative Second Order Queries

Jan Dreier,Robert Ganian,Thekla Hamm

2023-05-03

Abstract:Courcelle's theorem and its adaptations to cliquewidth have shaped the field of exact parameterized algorithms and are widely considered the archetype of algorithmic meta-theorems. In the past decade, there has been growing interest in developing parameterized approximation algorithms for problems which are not captured by Courcelle's theorem and, in particular, are considered not fixed-parameter tractable under the associated widths. We develop a generalization of Courcelle's theorem that yields efficient approximation schemes for any problem that can be captured by an expanded logic we call Blocked CMSO, capable of making logical statements about the sizes of set variables via so-called weight comparisons. The logic controls weight comparisons via the quantifier-alternation depth of the involved variables, allowing full comparisons for zero-alternation variables and limited comparisons for one-alternation variables. We show that the developed framework threads the very needle of tractability: on one hand it can describe a broad range of approximable problems, while on the other hand we show that the restrictions of our logic cannot be relaxed under well-established complexity assumptions. The running time of our approximation scheme is polynomial in $1/\varepsilon$, allowing us to fully interpolate between faster approximate algorithms and slower exact algorithms. This provides a unified framework to explain the tractability landscape of graph problems parameterized by treewidth and cliquewidth, as well as classical non-graph problems such as Subset Sum and Knapsack.

Computational Complexity,Discrete Mathematics,Data Structures and Algorithms,Logic in Computer Science
SDPs and Robust Satisfiability of Promise CSP

Joshua Brakensiek,Venkatesan Guruswami,Sai Sandeep

DOI: https://doi.org/10.48550/arXiv.2211.08373

2023-06-08

Abstract:For a constraint satisfaction problem (CSP), a robust satisfaction algorithm is one that outputs an assignment satisfying most of the constraints on instances that are near-satisfiable. It is known that the CSPs that admit efficient robust satisfaction algorithms are precisely those of bounded width, i.e., CSPs whose satisfiability can be checked by a simple local consistency algorithm (eg., 2-SAT or Horn-SAT in the Boolean case). While the exact satisfiability of a bounded width CSP can be checked by combinatorial algorithms, the robust algorithm is based on rounding a canonical Semidefinite programming(SDP) relaxation. In this work, we initiate the study of robust satisfaction algorithms for promise CSPs, which are a vast generalization of CSPs that have received much attention recently. The motivation is to extend the theory beyond CSPs, as well as to better understand the power of SDPs. We present robust SDP rounding algorithms under some general conditions, namely the existence of particular high-dimensional Boolean symmetries known as majority or alternating threshold polymorphisms. On the hardness front, we prove that the lack of such polymorphisms makes the PCSP hard for all pairs of symmetric Boolean predicates. Our method involves a novel method to argue SDP gaps via the absence of certain colorings of the sphere, with connections to sphere Ramsey theory. We conjecture that PCSPs with robust satisfaction algorithms are precisely those for which the feasibility of the canonical SDP implies (exact) satisfiability. We also give a precise algebraic condition, known as a minion characterization, of which PCSPs have the latter property.

Data Structures and Algorithms,Discrete Mathematics,Logic in Computer Science
Greedy Algorithms for the Maximum Satisfiability Problem: Simple Algorithms and Inapproximability Bounds

Matthias Poloczek,Georg Schnitger,David P. Williamson,Anke van Zuylen

DOI: https://doi.org/10.1137/15m1053369

2017-01-01

SIAM Journal on Computing

Abstract:We give a simple, randomized greedy algorithm for the maximum satisfiability problem (MAX SAT) that obtains a $\frac{3}{4}$-approximation in expectation. In contrast to previously known $\frac{3}{4}$-approximation algorithms, our algorithm does not use flows or linear programming. Hence we provide a positive answer to a question posed by Williamson in 1998 on whether such an algorithm exists. Moreover, we show that Johnson's greedy algorithm cannot guarantee a $\frac{3}{4}$-approximation, even if the variables are processed in a random order. Thereby we partially solve a problem posed by Chen, Friesen, and Zheng in 1999. In order to explore the limitations of the greedy paradigm, we use the model of priority algorithms of Borodin, Nielsen, and Rackoff. Since our greedy algorithm works in an online scenario where the variables arrive with their set of undecided clauses, we wonder if a better approximation ratio can be obtained by further fine-tuning its random decisions. For a particular information model we show that no priority algorithm can approximate Online MAX SAT within $\frac{3}{4} + \varepsilon$ (for any $\varepsilon > 0$). We further investigate the strength of deterministic greedy algorithms that may choose the variable ordering. Here we show that no adaptive priority algorithm can achieve approximation ratio $\frac{3}{4}$. We propose two ways in which this inapproximability result can be bypassed. First we show that if our greedy algorithm is additionally given the variable assignments of an optimal solution to the canonical LP relaxation, then we can derandomize its decisions while preserving the overall approximation guarantee. Second we give a simple, deterministic algorithm that performs an additional pass over the input. We show that this 2-pass algorithm satisfies clauses with a total weight of at least $\frac{3}{4} {OPT}_{LP}$, where ${OPT}_{LP}$ is the objective value of the canonical linear program. Moreover, we demonstrate that our analysis is tight and detail how each pass can be implemented in linear time.

computer science, theory & methods,mathematics, applied
Robust Stochastic Approximation Approach to Stochastic Programming

A. Nemirovski,A. Juditsky,G. Lan,A. Shapiro

DOI: https://doi.org/10.1137/070704277

IF: 2.763

2009-01-01

SIAM Journal on Optimization

Abstract:In this paper we consider optimization problems where the objective function is given in a form of the expectation. A basic difficulty of solving such stochastic optimization problems is that the involved multidimensional integrals (expectations) cannot be computed with high accuracy. The aim of this paper is to compare two computational approaches based on Monte Carlo sampling techniques, namely, the stochastic approximation (SA) and the sample average approximation (SAA) methods. Both approaches, the SA and SAA methods, have a long history. Current opinion is that the SAA method can efficiently use a specific (say, linear) structure of the considered problem, while the SA approach is a crude subgradient method, which often performs poorly in practice. We intend to demonstrate that a properly modified SA approach can be competitive and even significantly outperform the SAA method for a certain class of convex stochastic problems. We extend the analysis to the case of convex-concave stochastic saddle point problems and present (in our opinion highly encouraging) results of numerical experiments.

mathematics, applied
Sandwiches for Promise Constraint Satisfaction

Guofeng Deng,Ezzeddine El Sai,Trevor Manders,Peter Mayr,Poramate Nakkirt,Athena Sparks

DOI: https://doi.org/10.48550/arXiv.2003.07487

2020-03-17

Abstract:Promise Constraint Satisfaction Problems (PCSP) were proposed recently by Brakensiek and Guruswami <a class="link-https" data-arxiv-id="1704.01937" href="https://arxiv.org/abs/1704.01937">arXiv:1704.01937</a> as a framework to study approximations for Constraint Satisfaction Problems (CSP). Informally a PCSP asks to distinguish between whether a given instance of a CSP has a solution or not even a specified relaxation can be satisfied. All currently known tractable PCSPs can be reduced in a natural way to tractable CSPs. Barto <a class="link-https" data-arxiv-id="1909.04878" href="https://arxiv.org/abs/1909.04878">arXiv:1909.04878</a> presented an example of a PCSP over Boolean structures for which this reduction requires solving a CSP over an infinite structure. We give a first example of a PCSP over Boolean structures which reduces to a tractable CSP over a structure of size $3$ but not smaller. Further we investigate properties of PCSPs that reduce to systems of linear equations or to CSPs over structures with semilattice or majority polymorphism.

Computational Complexity
Sample Average Approximation for Stochastic Programming with Equality Constraints

Thomas Lew,Riccardo Bonalli,Marco Pavone

DOI: https://doi.org/10.1137/23m1573227

IF: 2.763

2024-10-15

SIAM Journal on Optimization

Abstract:SIAM Journal on Optimization, Volume 34, Issue 4, Page 3506-3533, December 2024. We revisit the sample average approximation (SAA) approach for nonconvex stochastic programming. We show that applying the SAA approach to problems with expected value equality constraints does not necessarily result in asymptotic optimality guarantees as the sample size increases. To address this issue, we relax the equality constraints. Then, we prove the asymptotic optimality of the modified SAA approach under mild smoothness and boundedness conditions on the equality constraint functions. Our analysis uses random set theory and concentration inequalities to characterize the approximation error from the sampling procedure. We apply our approach and analysis to the problem of stochastic optimal control for nonlinear dynamical systems under external disturbances modeled by a Wiener process. Numerical results on relevant stochastic programs show the reliability of the proposed approach. Results on a rocket-powered descent problem show that our computed solutions allow for significant uncertainty reduction compared to a deterministic baseline.

mathematics, applied

Algebraic Approach to Approximation

Algorithms for stochastic optimization with functional or expectation constraints

An Algebraic Approach for Approximity

Approximate algebraic structure

On Approximability of Satisfiable k-CSPs: V

Smooth approximations: An algebraic approach to CSPs over finitely bounded homogeneous structures

Sketching approximability of all finite CSPs

Approximation Algorithms for Combinatorial Optimization with Predictions

Approximation algorithms for noncommutative CSPs

An approximation algorithm for a general class of multi-parametric optimization problems

An invitation to the promise constraint satisfaction problem

Approximation Schemes Via Sherali-Adams Hierarchy for Dense Constraint Satisfaction Problems and Assignment Problems

Approximating Bounded Occurrence Ordering CSPs

Pliability and Approximating Max-CSPs

Approximate Evaluation of Quantitative Second Order Queries

SDPs and Robust Satisfiability of Promise CSP

Greedy Algorithms for the Maximum Satisfiability Problem: Simple Algorithms and Inapproximability Bounds

Robust Stochastic Approximation Approach to Stochastic Programming

Sandwiches for Promise Constraint Satisfaction

Sample Average Approximation for Stochastic Programming with Equality Constraints