Abstract:In recent years, deep clustering has achieved encouraging results. However, existing deep clustering methods work with the traditional Euclidean space and thus present deficiency on clustering complex structures. On the contrary, Riemannian geometry provides an elegant framework to model complex structures as well as a powerful tool for clustering, i.e., the Ricci flow. In this paper, we rethink the problem of deep clustering, and introduce the Riemannian geometry to deep clustering for the first time. Deep clustering in Riemannian manifold still faces significant challenges: (1) Ricci flow itself is unaware of cluster membership, (2) Ricci curvature prevents the gradient backpropagation, and (3) learning the flow largely remains open in the manifold. To bridge these gaps, we propose a novel Riemannian generative model (RicciNet), a neural Ricci flow with several theoretical guarantees. The novelty is that we model the dynamic self-clustering process of Ricci flow: data points move to the respective clusters in the manifold, influenced by Ricci curvatures. The point's trajectory is characterized by a parametric velocity, taking the form of Ordinary Differential Equation (ODE). Specifically, we encode data points as samples of Gaussian mixture in the manifold where we propose two types of reparameterization approaches: Gumbel reparameterization, and geometric trick. We formulate a differentiable Ricci curvature parameterized by a Riemannian graph convolution. Thereafter, we propose a geometric learning approach in which we study the geometric regularity of the point's trajectory, and learn the flow via distance matching and velocity matching. Consequently, data points go along the shortest Ricci flow to complete clustering. Extensive empirical results show RicciNet outperforms Euclidean deep methods.

Dynamically Stable Poincaré Embeddings for Neural Manifolds

Thoughts on the Consistency between Ricci Flow and Neural Network Behavior

Evolving Hypersurfaces by Their Mean Curvature in the Background Manifold Evolving by Ricci Flow

Deep Manifold Computing and Visualization

Deep Manifold Computing and Visualization Using Elastic Locally Isometric Smoothness

Learning Discretized Neural Networks under Ricci Flow

Neurodynamic Flow Approach for Convex and Quasi-Convex Optimization on Riemannian Manifolds with Diagonal Metrics

Deep learning as Ricci flow

Metric Flows with Neural Networks

RicciNet: Deep Clustering Via A Riemannian Generative Model

Physics-informed neural networks for transformed geometries and manifolds

Matrix Manifold Neural Networks++

Markov-Lipschitz Deep Learning

Riemannian Residual Neural Networks

Ricci flow-guided autoencoders in learning time-dependent dynamics

Neural Ordinary Differential Equations on Manifolds

Applying Ricci Flow to High Dimensional Manifold Learning

A Convergence Rate for Manifold Neural Networks

A Geometric Insight into Equivariant Message Passing Neural Networks on Riemannian Manifolds

Solving PDEs on unknown manifolds with machine learning