Max-convolution through numerics and tropical geometry

Taylor Brysiewicz,Jonathan D. Hauenstein,Caroline Hills
2023-06-20
Abstract:The maximum function, on vectors of real numbers, is not differentiable. Consequently, several differentiable approximations of this function are popular substitutes. We survey three smooth functions which approximate the maximum function and analyze their convergence rates. We interpret these functions through the lens of tropical geometry, where their performance differences are geometrically salient. As an application, we provide an algorithm which computes the max-convolution of two integer vectors in quasi-linear time. We show this algorithm's power in computing adjacent sums within a vector as well as computing service curves in a network analysis application.
Numerical Analysis
What problem does this paper attempt to address?
The main problem that this paper attempts to solve is how to approximately solve the max - function of vectors through numerical methods and tropical geometry. Specifically, the max - function is non - differentiable on the real elements of vectors. Therefore, the paper explores three smooth functions to approximate the max - function and analyzes their convergence rates. In addition, the author proposes an algorithm that can calculate the max - convolution of two integer vectors in quasi - linear time and demonstrates its powerful performance in network analysis applications. ### Main Content Overview 1. **Background and Motivation**: - The max - function \( M=\max_{1\leq i\leq n}v_i \) is a basic task, but this function is non - differentiable. - A common method in the fields of optimization and machine learning is to replace the exact max - value calculation with a differentiable approximation function. 2. **Three Approximation Methods**: - **LogSumExp (LSE)**: Defined as \( L_v(t)=\log\left(\sum_{j = 1}^n t^{v_j}\right) \), and \( M=\lim_{t\rightarrow\infty}L_v(t) \). - **Ratio**: Defined as \( R_v(t)=t\frac{F'_v(t)}{F_v(t)} \), and \( M=\lim_{t\rightarrow\infty}R_v(t) \). - **p - norm**: If each element of the vector is non - negative, then \( M=\lim_{p\rightarrow\infty}\|v\|_p = \|v\|_\infty \). 3. **Convergence Rate Analysis**: - Analyzed the convergence rates of the above three approximation methods. Especially for integer vectors, a lower bound \( t \) is provided such that the absolute error is less than a given \( \delta \). - In the case of high multiplicity, the Ratio approximation method is significantly better than other methods. 4. **Geometric Interpretation**: - The performance differences of these approximation methods are explained using the amoebas structure in tropical geometry. - The graph of \( u\mapsto L_v(e^u) \) is regarded as the upper boundary of some amoebas, and the performance differences of different approximation methods when the multiplicity of the maximum value is large are explained. 5. **Experimental Verification**: - The validity of the theoretical results is demonstrated through a series of experiments, and empirical data are provided to prove the compactness of the derived boundary conditions. - Experiments show that in the case where the maximum value has non - trivial multiplicity, the Ratio approximation method is significantly better than other methods. 6. **Applications**: - An algorithm for the max - convolution problem is proposed, which uses the fast Fourier transform (FFT) to complete the calculation in quasi - linear time. - This numerical method is applied to solve the problems of calculating the maximum continuous subsequence sum and the service curve constraint. ### Conclusion By combining numerical methods and tropical geometry, the paper not only deeply analyzes three methods for approximating the max - function and their convergence characteristics, but also proposes efficient algorithms to solve practical problems, such as max - convolution and service curve calculation in network analysis. These achievements provide new tools and perspectives for the fields of optimization and machine learning.