Abstract:The sum of ratios problem has a variety of important applications in economics and management science, but it is difficult to globally solve this problem. In this paper, we consider the minimization problem of a sum of a number of nondifferentiable quasi-convex component functions over a closed and convex set, which includes the sum of ratios problem as a special case. The sum of quasi-convex component functions is not necessarily to be quasi-convex, and so, this study goes beyond quasi-convex optimization. Exploiting the structure of the sum-minimization problem, we propose a new incremental subgradient method for this problem and investigate its convergence properties to a global optimal solution when using the constant, diminishing or dynamic stepsize rules and under a homogeneous assumption and the Hölder condition of order $p$. To economize on the computation cost of subgradients of a large number of component functions, we further propose a randomized incremental subgradient method, in which only one component function is randomly selected to construct the subgradient direction at each iteration. The convergence properties are obtained in terms of function values and distances of iterates from the optimal solution set with probability 1. The proposed incremental subgradient methods are applied to solve the sum of ratios problem, as well as the multiple Cobb-Douglas productions efficiency problem, and the numerical results show that the proposed methods are efficient for solving the large sum of ratios problem.

Incremental Methods for Weakly Convex Optimization

An Incremental Updating Method for Support Vector Machines

Last Iterate Convergence of Incremental Methods and Applications in Continual Learning

Incremental Majorization-Minimization Optimization with Application to Large-Scale Machine Learning

Linearized Proximal Algorithms with Adaptive Stepsizes for Convex Composite Optimization with Applications

An Aggressive Reduction on the Complexity of Optimization for Non-Strongly Convex Objectives

On Convergence of Incremental Gradient for Non-Convex Smooth Functions

A Population-Based Incremental Learning Method for Robust Optimal Solutions

Decentralized Weakly Convex Optimization Over the Stiefel Manifold

Weakly-Convex Concave Min-Max Optimization: Provable Algorithms and Applications in Machine Learning

Incremental Quasi-Newton Methods with Faster Superlinear Convergence Rates

Error bounds, PL condition, and quadratic growth for weakly convex functions, and linear convergences of proximal point methods

An inexact variable metric variant of incremental aggregated Forward-Backward method for the large-scale composite optimization problem

Weak Sharp Minima for Convex Infinite Optimization Problems in Normed Linear Spaces

Proximal methods for structured nonsmooth optimization over Riemannian submanifolds

Methods for Convex $(L_0,L_1)$-Smooth Optimization: Clipping, Acceleration, and Adaptivity

Stochastic Weakly Convex Optimization Beyond Lipschitz Continuity

Incremental Subgradient Methods for Minimizing The Sum of Quasi-convex Functions

An Adaptive Incremental Gradient Method With Support for Non-Euclidean Norms

Convex and Non-convex Optimization Under Generalized Smoothness

Variational Analysis Perspective on Linear Convergence of Some First Order Methods for Nonsmooth Convex Optimization Problems