Abstract:The Sum-of-Squares (SOS) approximation method is a technique used in optimization problems to derive lower bounds on the optimal value of an objective function. By representing the objective function as a sum of squares in a feature space, the SOS method transforms non-convex global optimization problems into solvable semidefinite programs. This note presents an overview of the SOS method. We start with its application in finite-dimensional feature spaces and, subsequently, we extend it to infinite-dimensional feature spaces using reproducing kernels (k-SOS). Additionally, we highlight the utilization of SOS for estimating some relevant quantities in information theory, including the log-partition function.
What problem does this paper attempt to address?
The core problem that this paper attempts to solve is to transform non - convex global optimization problems into solvable semidefinite programming (SDP) problems. Specifically, by representing the objective function as a sum - of - squares (SOS) in the feature space, the original non - convex problem is thus converted into a convex optimization problem. This method not only simplifies the solving process but also provides an effective means of computing non - negative functions.
### Main content of the paper
1. **Basic principles of the SOS method**:
- By representing a non - negative function in the form of a sum of squares, it can be transformed into a convex optimization problem.
- Specifically, given a feature mapping \(\phi: X \to \mathbb{C}^d\) and an objective function \(h(x)=\phi(x)^*H\phi(x)\), where \(H \in \mathcal{H}_d\) is a Hermitian matrix, if \(H \succeq 0\), then \(h(x)\) is a sum - of - squares function (SOS).
2. **SOS representation of non - negative functions**:
- In order to handle non - negative functions, they can be represented in the form of a sum of squares through feature mapping. For example, for polynomials, trigonometric polynomials, etc., corresponding feature mappings can be found.
- Through Proposition 1, we can verify whether a function is SOS: if \(h(x)=\phi(x)^*H\phi(x)\) and \(H \succeq 0\), then \(h(x)\) is SOS.
3. **SOS relaxation of optimization problems**:
- For the global minimization problem \(\inf_{x \in X}h(x)\), it can be relaxed into a semidefinite programming problem by introducing SOS constraints:
\[
\inf_{x \in X}h(x)=\sup_{c \in \mathbb{R}, A \succeq 0}c\quad\text{s.t.}\quad\forall x \in X,\;h(x)-c = \phi(x)^*A\phi(x)
\]
- This relaxation method not only makes the problem solvable but also ensures numerical stability.
4. **SOS relaxation of dual problems**:
- Through the analysis of dual problems, the effectiveness of SOS relaxation can be further understood. By introducing dual variables, the original problem can be transformed into a more easily solvable form.
5. **Applications of the SOS method**:
- This method is applicable not only to finite - dimensional feature spaces but also can be extended to infinite - dimensional feature spaces, which are processed using reproducing kernels.
- In addition, the SOS method also has important applications in information theory, such as estimating the log - partition function, etc.
### Contributions of the paper
- Proposed the SOS method for transforming non - convex optimization problems into convex optimization problems.
- Discussed in detail the applications of the SOS method in different feature spaces and demonstrated its effectiveness in special cases such as polynomials and trigonometric polynomials.
- Explored the applications of the SOS method in information theory, especially its performance in estimating some key quantities.
In conclusion, this paper provides a new and effective way to solve non - convex optimization problems through the SOS method and shows its broad application potential in multiple fields.