Abstract:In many applications, it is very expensive or time consuming to obtain a lot of labeled examples. One practically important problem is: can the labeled data from other related sources help predict the target task, even if they have 1) different feature spaces (e.g., image versus text data), 2) different data distributions, and 3) different output spaces? This paper proposes a solution and discusses the conditions where this is highly likely to produce better results. It first unifies the feature spaces of the target and source data sets by spectral embedding, even when they are with completely different feature spaces. The principle is to devise an optimization objective that preserves the original structure of the data, while at the same time, maximizes the similarity between the two. A linear projection model, as well as a nonlinear approach are derived on the basis of this principle with closed forms. Second, a judicious sample selection strategy is applied to select only those related source examples. At last, a Bayesian-based approach is applied to model the relationship between different output spaces. The three steps can bridge related heterogeneous sources in order to learn the target task. Among the 20 experiment data sets, for example, the images with wavelet-transformed-based features are used to predict another set of images whose features are constructed from color-histogram space; documents are used to help image classification, etc. By using these extracted examples from heterogeneous sources, the models can reduce the error rate by as much as 50 percent, compared with the methods using only the examples from the target task.

Wasserstein task embedding for measuring task similarities

Exploring Wasserstein Distance across Concept Embeddings for Ontology Matching

Deep Model Transferability from Attribution Maps

Distilled Wasserstein Learning for Word Embedding and Topic Modeling.

Task-oriented Word Embedding for Text Classification.

Gaussian Word Embedding with a Wasserstein Distance Loss

Adversarial task-specific learning.

Word Embedding based Edit Distance

Measuring similarity between embedding spaces using induced neighborhood graphs

Task-oriented Domain-specific Meta-Embedding for Text Classification

A Dataset Similarity Evaluation Framework for Wireless Communications and Sensing

Transfer Across Completely Different Feature Spaces Via Spectral Embedding

An Empirical Study of Self-Supervised Learning with Wasserstein Distance

Supervised Tree-Wasserstein Distance

Exploring the Effectiveness and Consistency of Task Selection in Intermediate-Task Transfer Learning

Wasserstein Distance-Based Auto-Encoder Tracking.

Semantics-assisted Wasserstein Learning for Topic and Word Embeddings

Task-aware Similarity Learning for Event-triggered Time Series

Task-Guided Context-Path Embedding in Temporal Heterogeneous Networks

Assignment Problem Based Deep Embedding

An Information-Theoretic Approach to Transferability in Task Transfer Learning