Learning Riemannian Metrics

Guy Lebanon
DOI: https://doi.org/10.48550/arXiv.1212.2474
2012-10-19
Abstract:We propose a solution to the problem of estimating a Riemannian metric associated with a given differentiable manifold. The metric learning problem is based on minimizing the relative volume of a given set of points. We derive the details for a family of metrics on the multinomial simplex. The resulting metric has applications in text classification and bears some similarity to TFIDF representation of text documents.
Machine Learning
What problem does this paper attempt to address?
The problem that this paper attempts to solve is how to learn a Riemannian metric from a given dataset in order to better capture the intrinsic geometric structure of the data, especially in the case of high - dimensional sparse data. Specifically, the author proposes a new framework to learn local Riemannian metrics instead of assuming a Euclidean metric structure. ### Core of the Problem 1. **Limitations of Existing Methods**: - Many machine - learning algorithms use the Euclidean metric structure by default, but this assumption is usually not supported by data or model arguments. - Existing metric - learning methods (such as the work of Saul and Jordan, Xing et al.) can capture non - Euclidean geometries, but their metrics in space are constant and cannot adapt well to local changes. - Other methods (such as kernel matrix learning proposed by Lanckriet et al.) can learn similarities from data, but the results cannot be generalized to unseen data points. 2. **The Proposed New Framework**: - The author proposes a method based on maximizing the inverse volume element to select the Riemannian metric. This method selects an optimal metric from a parameterized set of metric candidates. - In particular, on the multinomial simplex, the metric candidates are pull - back metrics of the Fisher information metric. - In this way, the learned metric can be defined in the entire embedding space and can capture local changes. ### Key Points of the Solution - **Maximizing the Inverse Volume Element**: By maximizing the inverse volume element, geodesics tend to pass through data - dense regions, thus better reflecting the intrinsic geometric structure of the data. - **Parameterized Transformation**: Use a family of parameterized transformations to specify the set of metric candidates. These transformations pull the Euclidean metric back onto the original manifold. - **Application to Text Classification**: In text - classification tasks, the geodesic distance is calculated using the learned Riemannian metric, and the results show that its performance is better than the traditional TFIDF cosine similarity and the TF - based L2 distance. ### Summary The main contribution of this paper is to propose a new framework for learning local Riemannian metrics from high - dimensional sparse data. This framework can not only better capture the intrinsic geometric structure of the data but also performs well in practical applications (such as text classification).