A Note On Deterministic Submodular Maximization With Bounded Curvature

Wenxin Li

2024-08-25

Abstract:We show that the recent breakthrough result of [Buchbinder and Feldman, FOCS'24] could further lead to a deterministic $(1-\kappa_{f}/e-\varepsilon)$-approximate algorithm for maximizing a submodular function with curvature $\kappa_{f}$ under matroid constraint.

Data Structures and Algorithms,Machine Learning

What problem does this paper attempt to address?

Deterministic Algorithm and Faster Algorithm for Submodular Maximization subject to a Matroid Constraint

Niv Buchbinder,Moran Feldman

2024-09-30

Abstract:We study the problem of maximizing a monotone submodular function subject to a matroid constraint, and present for it a deterministic non-oblivious local search algorithm that has an approximation guarantee of $1 - 1/e - \varepsilon$ (for any $\varepsilon > 0$) and query complexity of $\tilde{O}_\varepsilon(nr)$, where $n$ is the size of the ground set and $r$ is the rank of the matroid. Our algorithm vastly improves over the previous state-of-the-art $0.5008$-approximation deterministic algorithm, and in fact, shows that there is no separation between the approximation guarantees that can be obtained by deterministic and randomized algorithms for the problem considered. The query complexity of our algorithm can be improved to $\tilde{O}_\varepsilon(n + r\sqrt{n})$ using randomization, which is nearly-linear for $r = O(\sqrt{n})$, and is always at least as good as the previous state-of-the-art algorithms.

Data Structures and Algorithms,Discrete Mathematics
Submodular Maximization with Limited Function Access

Andrew Downie,Bahman Gharesifard,Stephen L. Smith

DOI: https://doi.org/10.48550/arXiv.2201.00724

2022-02-08

Abstract:We consider a class of submodular maximization problems in which decision-makers have limited access to the objective function. We explore scenarios where the decision-maker can observe only pairwise information, i.e., can evaluate the objective function on sets of size two. We begin with a negative result that no algorithm using only $k$-wise information can guarantee performance better than $k/n$. We present two algorithms that utilize only pairwise information about the function and characterize their performance relative to the optimal, which depends on the curvature of the submodular function. Additionally, if the submodular function possess a property called supermodularity of conditioning, then we can provide a method to bound the performance based purely on pairwise information. The proposed algorithms offer significant computational speedups over a traditional greedy strategy. A by-product of our study is the introduction of two new notions of curvature, the $k$-Marginal Curvature and the $k$-Cardinality Curvature. Finally, we present experiments highlighting the performance of our proposed algorithms in terms of approximation and time complexity.

Data Structures and Algorithms,Systems and Control,Optimization and Control
Maximizing a Submodular Function with Bounded Curvature under an Unknown Knapsack Constraint

Max Klimm,Martin Knaack

2024-10-24

Abstract:This paper studies the problem of maximizing a monotone submodular function under an unknown knapsack constraint. A solution to this problem is a policy that decides which item to pack next based on the past packing history. The robustness factor of a policy is the worst case ratio of the solution obtained by following the policy and an optimal solution that knows the knapsack capacity. We develop a policy with a robustness factor that is decreasing in the curvature $c$ of the submodular function. For the extreme cases $c=0$ corresponding to an additive objective function, it matches a previously known and best possible robustness factor of $1/2$. For the other extreme case of $c=1$ it yields a robustness factor of $\approx 0.35$ improving over the best previously known robustness factor of $\approx 0.06$. The analysis of our policy relies on a greedy algorithm that is a slight modification of Wolsey's greedy algorithm for the submodular knapsack problem with a known knapsack constraint. We obtain tight approximation guarantees for both of these algorithms in the setting of a submodular objective function with curvature $c$.

Data Structures and Algorithms,Discrete Mathematics
Subquadratic Submodular Maximization with a General Matroid Constraint

Yusuke Kobayashi,Tatsuya Terao

2024-05-01

Abstract:We consider fast algorithms for monotone submodular maximization with a general matroid constraint. We present a randomized $(1 - 1/e - \epsilon)$-approximation algorithm that requires $\tilde{O}_{\epsilon}(\sqrt{r} n)$ independence oracle and value oracle queries, where $n$ is the number of elements in the matroid and $r \leq n$ is the rank of the matroid. This improves upon the previously best algorithm by Buchbinder-Feldman-Schwartz [Mathematics of Operations Research 2017] that requires $\tilde{O}_{\epsilon}(r^2 + \sqrt{r}n)$ queries.

Data Structures and Algorithms
Discretely Beyond $1/e$: Guided Combinatorial Algorithms for Submodular Maximization

Yixin Chen,Ankur Nath,Chunli Peng,Alan Kuhnle

2024-05-23

Abstract:For constrained, not necessarily monotone submodular maximization, all known approximation algorithms with ratio greater than $1/e$ require continuous ideas, such as queries to the multilinear extension of a submodular function and its gradient, which are typically expensive to simulate with the original set function. For combinatorial algorithms, the best known approximation ratios for both size and matroid constraint are obtained by a simple randomized greedy algorithm of Buchbinder et al. [9]: $1/e \approx 0.367$ for size constraint and $0.281$ for the matroid constraint in $\mathcal O (kn)$ queries, where $k$ is the rank of the matroid. In this work, we develop the first combinatorial algorithms to break the $1/e$ barrier: we obtain approximation ratio of $0.385$ in $\mathcal O (kn)$ queries to the submodular set function for size constraint, and $0.305$ for a general matroid constraint. These are achieved by guiding the randomized greedy algorithm with a fast local search algorithm. Further, we develop deterministic versions of these algorithms, maintaining the same ratio and asymptotic time complexity. Finally, we develop a deterministic, nearly linear time algorithm with ratio $0.377$.

Data Structures and Algorithms,Discrete Mathematics,Machine Learning
Submodular Maximization Beyond Non-negativity: Guarantees, Fast Algorithms, and Applications

Christopher Harshaw,Moran Feldman,Justin Ward,Amin Karbasi

DOI: https://doi.org/10.48550/arXiv.1904.09354

2019-04-20

Abstract:It is generally believed that submodular functions -- and the more general class of $\gamma$-weakly submodular functions -- may only be optimized under the non-negativity assumption $f(S) \geq 0$. In this paper, we show that once the function is expressed as the difference $f = g - c$, where $g$ is monotone, non-negative, and $\gamma$-weakly submodular and $c$ is non-negative modular, then strong approximation guarantees may be obtained. We present an algorithm for maximizing $g - c$ under a $k$-cardinality constraint which produces a random feasible set $S$ such that $\mathbb{E} \left[ g(S) - c(S) \right] \geq (1 - e^{-\gamma} - \epsilon) g(OPT) - c(OPT)$, whose running time is $O (\frac{n}{\epsilon} \log^2 \frac{1}{\epsilon})$, i.e., independent of $k$. We extend these results to the unconstrained setting by describing an algorithm with the same approximation guarantees and faster $O(\frac{n}{\epsilon} \log\frac{1}{\epsilon})$ runtime. The main techniques underlying our algorithms are two-fold: the use of a surrogate objective which varies the relative importance between $g$ and $c$ throughout the algorithm, and a geometric sweep over possible $\gamma$ values. Our algorithmic guarantees are complemented by a hardness result showing that no polynomial-time algorithm which accesses $g$ through a value oracle can do better. We empirically demonstrate the success of our algorithms by applying them to experimental design on the Boston Housing dataset and directed vertex cover on the Email EU dataset.

Data Structures and Algorithms,Machine Learning,Optimization and Control
Dynamic Algorithms for Matroid Submodular Maximization

Kiarash Banihashem,Leyla Biabani,Samira Goudarzi,MohammadTaghi Hajiaghayi,Peyman Jabbarzade,Morteza Monemizadeh

2023-12-26

Abstract:Submodular maximization under matroid and cardinality constraints are classical problems with a wide range of applications in machine learning, auction theory, and combinatorial optimization. In this paper, we consider these problems in the dynamic setting, where (1) we have oracle access to a monotone submodular function $f: 2^{V} \rightarrow \mathbb{R}^+$ and (2) we are given a sequence $\mathcal{S}$ of insertions and deletions of elements of an underlying ground set $V$. We develop the first fully dynamic $(4+\epsilon)$-approximation algorithm for the submodular maximization problem under the matroid constraint using an expected worst-case $O(k\log(k)\log^3{(k/\epsilon)})$ query complexity where $0 < \epsilon \le 1$. This resolves an open problem of Chen and Peng (STOC'22) and Lattanzi et al. (NeurIPS'20). As a byproduct, for the submodular maximization under the cardinality constraint $k$, we propose a parameterized (by the cardinality constraint $k$) dynamic algorithm that maintains a $(2+\epsilon)$-approximate solution of the sequence $\mathcal{S}$ at any time $t$ using an expected worst-case query complexity $O(k\epsilon^{-1}\log^2(k))$. This is the first dynamic algorithm for the problem that has a query complexity independent of the size of ground set $V$.

Data Structures and Algorithms,Machine Learning
Fully Dynamic Submodular Maximization over Matroids

Paul Dütting,Federico Fusco,Silvio Lattanzi,Ashkan Norouzi-Fard,Morteza Zadimoghaddam

2023-05-31

Abstract:Maximizing monotone submodular functions under a matroid constraint is a classic algorithmic problem with multiple applications in data mining and machine learning. We study this classic problem in the fully dynamic setting, where elements can be both inserted and deleted in real-time. Our main result is a randomized algorithm that maintains an efficient data structure with an $\tilde{O}(k^2)$ amortized update time (in the number of additions and deletions) and yields a $4$-approximate solution, where $k$ is the rank of the matroid.

Data Structures and Algorithms,Machine Learning
Approximation Algorithms for Maximization of -Submodular Function Under a Matroid Constraint

Yuezhu Liu,Yunjing Sun,Min Li

DOI: https://doi.org/10.26599/tst.2023.9010122

2024-06-21

Tsinghua Science & Technology

Abstract:In this paper, we design a deterministic 1/3-approximation algorithm for the problem of maximizing non-monotone -submodular function under a matroid constraint. In order to reduce the complexity of this algorithm, we also present a randomized 1/3-approximation algorithm with the probability of , where is the probability of algorithm failure. Moreover, we design a streaming algorithm for both monotone and non-monotone objective -submodular functions.

computer science, information systems,engineering, electrical & electronic, software engineering
Greedy Maximization of Functions with Bounded Curvature under Partition Matroid Constraints

Tobias Friedrich,Andreas Göbel,Frank Neumann,Francesco Quinzan,Ralf Rothenberger

DOI: https://doi.org/10.1609/aaai.v33i01.33012272

2019-07-17

Proceedings of the AAAI Conference on Artificial Intelligence

Abstract:We investigate the performance of a deterministic GREEDY algorithm for the problem of maximizing functions under a partition matroid constraint. We consider non-monotone submodular functions and monotone subadditive functions. Even though constrained maximization problems of monotone submodular functions have been extensively studied, little is known about greedy maximization of non-monotone submodular functions or monotone subadditive functions. We give approximation guarantees for GREEDY on these problems, in terms of the curvature. We find that this simple heuristic yields a strong approximation guarantee on a broad class of functions. We discuss the applicability of our results to three real-world problems: Maximizing the determinant function of a positive semidefinite matrix, and related problems such as the maximum entropy sampling problem, the constrained maximum cut problem on directed graphs, and combinatorial auction games. We conclude that GREEDY is well-suited to approach these problems. Overall, we present evidence to support the idea that, when dealing with constrained maximization problems with bounded curvature, one needs not search for (approximate) monotonicity to get good approximate solutions.
Extending the Extension: Deterministic Algorithm for Non-monotone Submodular Maximization

Niv Buchbinder,Moran Feldman

2024-09-22

Abstract:Maximization of submodular functions under various constraints is a fundamental problem that has been studied extensively. A powerful technique that has emerged and has been shown to be extremely effective for such problems is the following. First, a continues relaxation of the problem is obtained by relaxing the (discrete) set of feasible solutions to a convex body, and extending the discrete submodular function $f$ to a continuous function $F$ known as the multilinear extension. Then, two algorithmic steps are implemented. The first step approximately solves the relaxation by finding a fractional solution within the convex body that approximately maximizes $F$; and the second step rounds this fractional solution to a feasible integral solution. While this ``fractionally solve and then round'' approach has been a key technique for resolving many questions in the field, the main drawback of algorithms based on it is that evaluating the multilinear extension may require a number of value oracle queries to $f$ that is exponential in the size of $f$'s ground set. The only known way to tackle this issue is to approximate the value of $F$ via sampling, which makes all algorithms based on this approach inherently randomized and quite slow. In this work, we introduce a new tool, that we refer to as the extended multilinear extension, designed to derandomize submodular maximization algorithms that are based on the successful ``solve fractionally and then round'' approach. We demonstrate the effectiveness of this new tool on the fundamental problem of maximizing a submodular function subject to a matroid constraint, and show that it allows for a deterministic implementation of both the fractionally solving step and the rounding step of the above approach. As a bonus, we also get a randomized algorithm for the problem with an improved query complexity.

Data Structures and Algorithms,Discrete Mathematics
On maximizing a monotone $k$-submodular function under a knapsack constraint

Zhongzheng Tang,Chenhao Wang,Hau Chan

2023-08-03

Abstract:We study the problem of maximizing a non-negative monotone $k$-submodular function $f$ under a knapsack constraint, where a $k$-submodular function is a natural generalization of a submodular function to $k$ dimensions. We present a deterministic $(\frac12-\frac{1}{2e})\approx 0.316$-approximation algorithm that evaluates $f$ $O(n^4k^3)$ times, based on the result of Sviridenko (2004) on submodular knapsack maximization.

Data Structures and Algorithms,Discrete Mathematics,Optimization and Control
An Improved Greedy Curvature Bound in Finite-Horizon String Optimization with Application to a Sensor Coverage Problem

Brandon Van Over,Bowen Li,Edwin K.P. Chong,Ali Pezeshki

2023-09-07

Abstract:We study the optimization problem of choosing strings of finite length to maximize string submodular functions on string matroids, which is a broader class of problems than maximizing set submodular functions on set matroids. We provide a lower bound for the performance of the greedy algorithm in our problem, and then prove that our bound is superior to the greedy curvature bound of Conforti and Cornuejols. Our bound has lower computational complexity than most previously proposed curvature bounds. Finally, we demonstrate the strength of our result on a sensor coverage problem.

Data Structures and Algorithms,Discrete Mathematics,Systems and Control
Constrained Submodular Maximization via New Bounds for DR-Submodular Functions

Niv Buchbinder,Moran Feldman

2023-11-02

Abstract:Submodular maximization under various constraints is a fundamental problem studied continuously, in both computer science and operations research, since the late $1970$'s. A central technique in this field is to approximately optimize the multilinear extension of the submodular objective, and then round the solution. The use of this technique requires a solver able to approximately maximize multilinear extensions. Following a long line of work, Buchbinder and Feldman (2019) described such a solver guaranteeing $0.385$-approximation for down-closed constraints, while Oveis Gharan and Vondrák (2011) showed that no solver can guarantee better than $0.478$-approximation. In this paper, we present a solver guaranteeing $0.401$-approximation, which significantly reduces the gap between the best known solver and the inapproximability result. The design and analysis of our solver are based on a novel bound that we prove for DR-submodular functions. This bound improves over a previous bound due to Feldman et al. (2011) that is used by essentially all state-of-the-art results for constrained maximization of general submodular/DR-submodular functions. Hence, we believe that our new bound is likely to find many additional applications in related problems, and to be a key component for further improvement.

Data Structures and Algorithms,Discrete Mathematics
Weakly Submodular Maximization Beyond Cardinality Constraints: Does Randomization Help Greedy?

Lin Chen,Moran Feldman,Amin Karbasi

DOI: https://doi.org/10.48550/arXiv.1707.04347

IF: 0.961

2017-07-13

Discrete Mathematics

Abstract:Submodular functions are a broad class of set functions, which naturally arise in diverse areas. Many algorithms have been suggested for the maximization of these functions. Unfortunately, once the function deviates from submodularity, the known algorithms may perform arbitrarily poorly. Amending this issue, by obtaining approximation results for set functions generalizing submodular functions, has been the focus of recent works. One such class, known as weakly submodular functions, has received a lot of attention. A key result proved by Das and Kempe (2011) showed that the approximation ratio of the greedy algorithm for weakly submodular maximization subject to a cardinality constraint degrades smoothly with the distance from submodularity. However, no results have been obtained for maximization subject to constraints beyond cardinality. In particular, it is not known whether the greedy algorithm achieves any non-trivial approximation ratio for such constraints. In this paper, we prove that a randomized version of the greedy algorithm (previously used by Buchbinder et al. (2014) for a different problem) achieves an approximation ratio of $(1 + 1/\gamma)^{-2}$ for the maximization of a weakly submodular function subject to a general matroid constraint, where $\gamma$ is a parameter measuring the distance of the function from submodularity. Moreover, we also experimentally compare the performance of this version of the greedy algorithm on real world problems against natural benchmarks, and show that the algorithm we study performs well also in practice. To the best of our knowledge, this is the first algorithm with a non-trivial approximation guarantee for maximizing a weakly submodular function subject to a constraint other than the simple cardinality constraint. In particular, it is the first algorithm with such a guarantee for the important and broad class of matroid constraints.
Efficient Deterministic Algorithms for Maximizing Symmetric Submodular Functions

Zongqi Wan,Jialin Zhang,Xiaoming Sun,Zhijie Zhang

2024-06-20

Abstract:Symmetric submodular maximization is an important class of combinatorial optimization problems, including MAX-CUT on graphs and hyper-graphs. The state-of-the-art algorithm for the problem over general constraints has an approximation ratio of $0.432$. The algorithm applies the canonical continuous greedy technique that involves a sampling process. It, therefore, suffers from high query complexity and is inherently randomized. In this paper, we present several efficient deterministic algorithms for maximizing a symmetric submodular function under various constraints. Specifically, for the cardinality constraint, we design a deterministic algorithm that attains a $0.432$ ratio and uses $O(kn)$ queries. Previously, the best deterministic algorithm attains a $0.385-\epsilon$ ratio and uses $O\left(kn (\frac{10}{9\epsilon})^{\frac{20}{9\epsilon}-1}\right)$ queries. For the matroid constraint, we design a deterministic algorithm that attains a $1/3-\epsilon$ ratio and uses $O(kn\log \epsilon^{-1})$ queries. Previously, the best deterministic algorithm can also attain $1/3-\epsilon$ ratio but it uses much larger $O(\epsilon^{-1}n^4)$ queries. For the packing constraints with a large width, we design a deterministic algorithm that attains a $0.432-\epsilon$ ratio and uses $O(n^2)$ queries. To the best of our knowledge, there is no deterministic algorithm for the constraint previously. The last algorithm can be adapted to attain a $0.432$ ratio for single knapsack constraint using $O(n^4)$ queries. Previously, the best deterministic algorithm attains a $0.316-\epsilon$ ratio and uses $\widetilde{O}(n^3)$ queries.

Data Structures and Algorithms
An Approximation Algorithm for Risk-Averse Submodular Optimization

Lifeng Zhou,Pratap Tokekar

DOI: https://doi.org/10.1007/978-3-030-44051-0_9

2020-01-01

Abstract:We study the problem of incorporating risk while making combinatorial decisions under uncertainty. We formulate a discrete submodular maximization problem for selecting a set using Conditional-Value-at-Risk (CVaR), a risk metric commonly used in financial analysis. While CVaR has recently been used in optimization of linear cost functions in robotics, we take the first stages towards extending this to discrete submodular optimization and provide several positive results. Specifically, we propose the Sequential Greedy Algorithm that provides an approximation guarantee on finding the maxima of CVaR cost function under a matroidal constraint. The approximation guarantee shows that the solution produced by our algorithm is within a constant factor of the optimal and an additive term that depends on the optimal. Our analysis uses the curvature of the submodular set function, and proves that the algorithm runs in polynomial time. This formulates a number of combinatorial optimization problems that appear in robotics. We use two such problems, vehicle assignment under uncertainty for mobility-on-demand and sensor selection with failures for environmental monitoring, as case studies to demonstrate the efficacy of our formulation.
On Bounds for Greedy Schemes in String Optimization based on Greedy Curvatures

Bowen Li,Brandon Van Over,Edwin K. P. Chong,Ali Pezeshki

2024-04-10

Abstract:We consider the celebrated bound introduced by Conforti and Cornuéjols (1984) for greedy schemes in submodular optimization. The bound assumes a submodular function defined on a collection of sets forming a matroid and is based on greedy curvature. We show that the bound holds for a very general class of string problems that includes maximizing submodular functions over set matroids as a special case. We also derive a bound that is computable in the sense that they depend only on quantities along the greedy trajectory. We prove that our bound is superior to the greedy curvature bound of Conforti and Cornuéjols. In addition, our bound holds under a condition that is weaker than submodularity.

Systems and Control,Data Structures and Algorithms
Fast algorithms for k-submodular maximization subject to a matroid constraint

Shuxian Niu,Qian Liu,Yang Zhou,Min Li

2023-07-26

Abstract:In this paper, we apply a Threshold-Decreasing Algorithm to maximize $k$-submodular functions under a matroid constraint, which reduces the query complexity of the algorithm compared to the greedy algorithm with little loss in approximation ratio. We give a $(\frac{1}{2} - \epsilon)$-approximation algorithm for monotone $k$-submodular function maximization, and a $(\frac{1}{3} - \epsilon)$-approximation algorithm for non-monotone case, with complexity $O(\frac{n(k\cdot EO + IO)}{\epsilon} \log \frac{r}{\epsilon})$, where $r$ denotes the rank of the matroid, and $IO, EO$ denote the number of oracles to evaluate whether a subset is an independent set and to compute the function value of $f$, respectively. Since the constraint of total size can be looked as a special matroid, called uniform matroid, then we present the fast algorithm for maximizing $k$-submodular functions subject to a total size constraint as corollaries. corollaries.

Data Structures and Algorithms,Machine Learning
Practical $0.385$-Approximation for Submodular Maximization Subject to a Cardinality Constraint

Murad Tukan,Loay Mualem,Moran Feldman

2024-05-23

Abstract:Non-monotone constrained submodular maximization plays a crucial role in various machine learning applications. However, existing algorithms often struggle with a trade-off between approximation guarantees and practical efficiency. The current state-of-the-art is a recent $0.401$-approximation algorithm, but its computational complexity makes it highly impractical. The best practical algorithms for the problem only guarantee $1/e$-approximation. In this work, we present a novel algorithm for submodular maximization subject to a cardinality constraint that combines a guarantee of $0.385$-approximation with a low and practical query complexity of $O(n+k^2)$. Furthermore, we evaluate the empirical performance of our algorithm in experiments based on various machine learning applications, including Movie Recommendation, Image Summarization, and more. These experiments demonstrate the efficacy of our approach.

Machine Learning,Discrete Mathematics,Data Structures and Algorithms

A Note On Deterministic Submodular Maximization With Bounded Curvature

Deterministic Algorithm and Faster Algorithm for Submodular Maximization subject to a Matroid Constraint

Submodular Maximization with Limited Function Access

Maximizing a Submodular Function with Bounded Curvature under an Unknown Knapsack Constraint

Subquadratic Submodular Maximization with a General Matroid Constraint

Discretely Beyond $1/e$: Guided Combinatorial Algorithms for Submodular Maximization

Submodular Maximization Beyond Non-negativity: Guarantees, Fast Algorithms, and Applications

Dynamic Algorithms for Matroid Submodular Maximization

Fully Dynamic Submodular Maximization over Matroids

Approximation Algorithms for Maximization of -Submodular Function Under a Matroid Constraint

Greedy Maximization of Functions with Bounded Curvature under Partition Matroid Constraints

Extending the Extension: Deterministic Algorithm for Non-monotone Submodular Maximization

On maximizing a monotone $k$-submodular function under a knapsack constraint

An Improved Greedy Curvature Bound in Finite-Horizon String Optimization with Application to a Sensor Coverage Problem

Constrained Submodular Maximization via New Bounds for DR-Submodular Functions

Weakly Submodular Maximization Beyond Cardinality Constraints: Does Randomization Help Greedy?

Efficient Deterministic Algorithms for Maximizing Symmetric Submodular Functions

An Approximation Algorithm for Risk-Averse Submodular Optimization

On Bounds for Greedy Schemes in String Optimization based on Greedy Curvatures

Fast algorithms for k-submodular maximization subject to a matroid constraint

Practical $0.385$-Approximation for Submodular Maximization Subject to a Cardinality Constraint