Abstract:Domain transfer learning aims to learn common data representations from a source domain and a target domain so that the source domain data can help the classification of the target domain. Conventional transfer representation learning imposes the distributions of source and target domain representations to be similar, which heavily relies on the characterization of the distributions of domains and the distribution matching criteria. In this paper, we proposed a novel framework for domain transfer representation learning. Our motive is to make the learned representations of data points independent from the domains which they belong to. In other words, from an optimal cross-domain representation of a data point, it is difficult to tell which domain it is from. In this way, the learned representations can be generalized to different domains. To measure the dependency between the representations and the corresponding domain which the data points belong to, we propose to use the mutual information between the representations and the domain-belonging indicators. By minimizing such mutual information, we learn the representations which are independent from domains. We build a classwise deep convolutional network model as a representation model and maximize the margin of each data point of the corresponding class, which is defined over the intraclass and interclass neighborhood. To learn the parameters of the model, we construct a unified minimization problem where the margins are maximized while the representation-domain mutual information is minimized. In this way, we learn representations which are not only discriminate but also independent from domains. An iterative algorithm based on the Adam optimization method is proposed to solve the minimization to learn the classwise deep model parameters and the cross-domain representations simultaneously. Extensive experiments over benchmark datasets show its effectiveness and advantage over existing domain transfer learning methods.

Learning deep representations by mutual information estimation and maximization

Learning Disentangled Representation for Cross-Modal Retrieval with Deep Mutual Information Estimation.

Efficient Distribution Matching of Representations via Noise-Injected Deep InfoMax

Stable and Fast Deep Mutual Information Maximization Based on Wasserstein Distance

Deep Representation Debiasing Via Mutual Information Minimization and Maximization (student Abstract).

Multimodal Representation Learning via Maximization of Local Mutual Information

Heterogeneous Deep Graph Infomax

Cross-modal Image Retrieval with Deep Mutual Information Maximization

Graph Representation Learning Via Graphical Mutual Information Maximization.

Maximizing Mutual Information Across Feature and Topology Views for Learning Graph Representations

Learning Disentangled Representations via Mutual Information Estimation

Variational Information Maximisation for Intrinsically Motivated Reinforcement Learning

Mutual Information Gradient Estimation for Representation Learning

Learning Efficient Representation for Intrinsic Motivation

Infrared and Visible Image Fusion Via Mutual Information Maximization

Learning Domain-Independent Deep Representations by Mutual Information Minimization

Unsupervised Deep Representation Learning for Motor Fault Diagnosis by Mutual Information Maximization

Learning Representations by Graphical Mutual Information Estimation and Maximization

A robust estimator of mutual information for deep learning interpretability

Cluster-aware Multiplex InfoMax for Unsupervised Graph Representation Learning

Deep Sufficient Representation Learning via Mutual Information