Junyu Chen,Binh T. Nguyen,Shang Hui Koh,Yong Sheng Soh
Abstract:The Gromov-Wasserstein (GW) distance is an extension of the optimal transport problem that allows one to match objects between incomparable spaces. At its core, the GW distance is specified as the solution of a non-convex quadratic program and is not known to be tractable to solve. In particular, existing solvers for the GW distance are only able to find locally optimal solutions. In this work, we propose a semi-definite programming (SDP) relaxation of the GW distance. The relaxation can be viewed as the Lagrangian dual of the GW distance augmented with constraints that relate to the linear and quadratic terms of transportation plans. In particular, our relaxation provides a tractable (polynomial-time) algorithm to compute globally optimal transportation plans (in some instances) together with an accompanying proof of global optimality. Our numerical experiments suggest that the proposed relaxation is strong in that it frequently computes the globally optimal solution. Our Python implementation is available at <a class="link-external link-https" href="https://github.com/tbng/gwsdp" rel="external noopener nofollow">this https URL</a>.
What problem does this paper attempt to address?
### Problems the paper attempts to solve
The paper aims to solve the problem of calculating the Gromov - Wasserstein (GW) distance. The GW distance is an extended optimal transport problem, which allows matching objects between non - comparable spaces. However, the GW distance is essentially a non - convex quadratic programming problem, and current solvers can only find local optimal solutions and cannot guarantee global optimal solutions. To solve this problem, the author proposes a semi - definite programming (SDP) relaxation method, which can provide a polynomial - time algorithm to calculate the global optimal transport plan, along with a proof of global optimality.
### Main contributions
1. **Propose strong SDP relaxation**:
- The author proposes a strong relaxation method based on semi - definite programming, which can find global optimal solutions in many cases.
- This method enhances the GW problem through the Lagrangian dual method, adding constraints related to linear and quadratic term transport plans.
2. **Proof of global optimality**:
- Through the SDP relaxation method, a lower bound can be calculated, which can be used to verify the global optimality of a given transport plan.
- If the rank of the solution obtained by the SDP relaxation method is 1, it can be proved that the solution is globally optimal.
3. **Numerical experiments**:
- The author has carried out a large number of numerical experiments, and the results show that in many cases, the proposed SDP relaxation method can frequently find global optimal solutions.
- The experiments also show that compared with other existing methods, the SDP relaxation method has significant advantages in computational efficiency and solution quality.
### Mathematical formulas
- **Definition of GW distance**:
\[
\text{GW}(C, D, \alpha, \beta) \defeq \min_{\pi \in \Pi(\alpha, \beta)} \sum_{i,j,k,l} \ell(C_{i,k}, D_{j,l}) \pi_{i,j} \pi_{k,l} = \min_{\pi \in \Pi(\alpha, \beta)} \langle L(C, D) \otimes \pi, \pi \rangle
\]
where \( L(C, D)_{i,j,k,l} \defeq \ell(C_{i,k}, D_{j,l}) \) is a four - dimensional tensor, representing the difference between two metric spaces \( C \) and \( D \).
- **SDP relaxation**:
\[
\text{GW - SDP}(C, D, \alpha, \beta) \defeq \min_{\pi \in \mathbb{R}^{m \times n}, P \in \mathbb{R}^{mn \times mn}} \langle L, P \rangle
\]
\[
\text{s.t.} \quad \begin{pmatrix} P & \text{vec}(\pi) \\ \text{vec}(\pi)^T & 1 \end{pmatrix} \succeq 0, \quad \pi \in \Pi(\alpha, \beta)
\]
\[
P \text{vec}(e_i 1_n^T) = \alpha_i \text{vec}(\pi), \quad i \in [m]
\]
\[
P \text{vec}(1_m e_j^T) = \beta_j \text{vec}(\pi), \quad j \in [n]
\]
\[
P \geq 0
\]
### Conclusion
The paper successfully solves the global optimality problem in GW distance calculation by proposing a strong SDP relaxation method. The numerical experiment results show that this method can find global optimal solutions in many cases and is superior to existing methods in computational efficiency and solution quality. This provides new opportunities for the application of GW distance in machine learning and other fields.