Multiresolution Kernels

Marco Cuturi,Kenji Fukumizu

DOI: https://doi.org/10.48550/arXiv.cs/0507033

2005-11-14

Abstract:We present in this work a new methodology to design kernels on data which is structured with smaller components, such as text, images or sequences. This methodology is a template procedure which can be applied on most kernels on measures and takes advantage of a more detailed "bag of components" representation of the objects. To obtain such a detailed description, we consider possible decompositions of the original bag into a collection of nested bags, following a prior knowledge on the objects' structure. We then consider these smaller bags to compare two objects both in a detailed perspective, stressing local matches between the smaller bags, and in a global or coarse perspective, by considering the entire bag. This multiresolution approach is likely to be best suited for tasks where the coarse approach is not precise enough, and where a more subtle mixture of both local and global similarities is necessary to compare objects. The approach presented here would not be computationally tractable without a factorization trick that we introduce before presenting promising results on an image retrieval task.

Machine Learning

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: when processing structured data (such as text, image or sequence), how to design a new kernel function method to more finely capture the local and global similarities between the internal components of an object. Traditional methods usually assume that the generation of components is independent and stationary, which may lead to an overly simplified representation and fail to fully capture complex conditional dependencies and global similarities.

Specifically, the paper points out the following problems:
1. **Limitations of existing methods**: Traditional methods based on histograms or "bag of components" ignore the context information in the component generation process, such as position information or transition information. These methods usually assume that each component is generated independently and only focus on the overall histogram representation.
2. **Requirements of complex tasks**: For some tasks (such as speech, language, time - series or image processing), relying solely on rough global similarity may not be sufficient to solve the problem. These tasks require a more detailed combination of local and global similarities to compare objects.

To solve these problems, the author proposes a method of multiresolution kernels, which can combine different levels of detail and consider both local matching and global similarity. By introducing a decomposition framework, the original object is decomposed into multiple nested sub - objects (bags), and the kernel function is applied at different levels, thereby achieving a more detailed description and comparison of the object.

### Formula summary

- **Definition of multiresolution kernel function**:
\[
k_{\pi}(\mu,\mu')=\sum_{P\in P_{D}}\pi(P)k_{P}(\mu,\mu')
\]
where \(k_{P}(\mu,\mu')\) is the kernel function based on partition \(P\), and \(\pi(P)\) is the prior probability of partition \(P\).

- **Recursive formula for partition generation**:
\[
K_{T}=(1 - \varepsilon_{T})k_{T}(\mu,\mu')+\varepsilon_{T}\prod_{U\in s(T)}K_{U}
\]
where \(K_{T}\) is the intermediate result of recursive calculation, \(s(T)\) is the set of siblings of \(T\), and \(\varepsilon_{T}\) is a binomial distribution parameter.

Through this method, the paper aims to provide a more flexible and powerful tool for dealing with complex similarity problems in structured data.