Abstract:We study a class of optimization problems including matrix scaling, matrix balancing, multidimensional array scaling, operator scaling, and tensor scaling that arise frequently in theory and in practice. Some of these problems, such as matrix and array scaling, are convex in the Euclidean sense, but others such as operator scaling and tensor scaling are geodesically convex on a different Riemannian manifold. Trust region methods, which include box-constrained Newton's method, are known to produce high precision solutions very quickly for matrix scaling and matrix balancing (Cohen et. al., FOCS 2017, Allen-Zhu et. al. FOCS 2017), and result in polynomial time algorithms for some geodesically convex problems like operator scaling (Garg et. al. STOC 2018, Bürgisser et. al. FOCS 2019). One is led to ask whether these guarantees also hold for multidimensional array scaling and tensor scaling. We show that this is not the case by exhibiting instances with exponential diameter bound: we construct polynomial-size instances of 3-dimensional array scaling and 3-tensor scaling whose approximate solutions all have doubly exponential condition number. Moreover, we study convex-geometric notions of complexity known as margin and gap, which are used to bound the running times of all existing optimization algorithms for such problems. We show that margin and gap are exponentially small for several problems including array scaling, tensor scaling and polynomial scaling. Our results suggest that it is impossible to prove polynomial running time bounds for tensor scaling based on diameter bounds alone. Therefore, our work motivates the search for analogues of more sophisticated algorithms, such as interior point methods, for geodesically convex optimization that do not rely on polynomial diameter bounds.

Algorithm-independent bounds on complex optimization through the statistics of marginal optima

Conditioning the complexity of random landscapes on marginal optima

The Asymptotic Distribution and Berry–Esseen Bound of a New Test for Independence in High Dimension with an Application to Stochastic Optimization

Bound Analysis of Natural Gradient Descent in Stochastic Optimization Setting

A Bound on the Expected Optimality of Random Feasible Solutions to Combinatorial Optimization Problems

Random Gradient-Free Minimization of Convex Functions

Asymptotic Optimality in Stochastic Optimization

Near-optimal algorithms for minimax optimization

Tight Lower Complexity Bounds for Strongly Convex Finite-Sum Optimization

High Probability Complexity Bounds for Non-Smooth Stochastic Optimization with Heavy-Tailed Noise

High-Probability Bounds for Stochastic Optimization and Variational Inequalities: the Case of Unbounded Variance

Optimal rates for zero-order convex optimization: The power of two function evaluations

A Note on High-Probability Analysis of Algorithms with Exponential, Sub-Gaussian, and General Light Tails

Towards Optimal Problem Dependent Generalization Error Bounds in Statistical Learning Theory

A General Analysis Framework of Lower Complexity Bounds for Finite-Sum Optimization

Local Asymptotics for some Stochastic Optimization Problems: Optimality, Constraint Identification, and Dual Averaging

High-Probability Complexity Bounds for Non-smooth Stochastic Convex Optimization with Heavy-Tailed Noise

Barriers for recent methods in geodesic optimization

Limitations of Information-Theoretic Generalization Bounds for Gradient Descent Methods in Stochastic Convex Optimization

Optimal Time Complexities of Parallel Stochastic Optimization Methods Under a Fixed Computation Model

Bounds on the infimum of polynomials over a generic semi-algebraic set using asymptotic critical values