Abstract:Domain transfer learning aims to learn common data representations from a source domain and a target domain so that the source domain data can help the classification of the target domain. Conventional transfer representation learning imposes the distributions of source and target domain representations to be similar, which heavily relies on the characterization of the distributions of domains and the distribution matching criteria. In this paper, we proposed a novel framework for domain transfer representation learning. Our motive is to make the learned representations of data points independent from the domains which they belong to. In other words, from an optimal cross-domain representation of a data point, it is difficult to tell which domain it is from. In this way, the learned representations can be generalized to different domains. To measure the dependency between the representations and the corresponding domain which the data points belong to, we propose to use the mutual information between the representations and the domain-belonging indicators. By minimizing such mutual information, we learn the representations which are independent from domains. We build a classwise deep convolutional network model as a representation model and maximize the margin of each data point of the corresponding class, which is defined over the intraclass and interclass neighborhood. To learn the parameters of the model, we construct a unified minimization problem where the margins are maximized while the representation-domain mutual information is minimized. In this way, we learn representations which are not only discriminate but also independent from domains. An iterative algorithm based on the Adam optimization method is proposed to solve the minimization to learn the classwise deep model parameters and the cross-domain representations simultaneously. Extensive experiments over benchmark datasets show its effectiveness and advantage over existing domain transfer learning methods.

Preserving domain private information via mutual information maximization

Preserving Domain Private Representation via Mutual Information Maximization

Enhancing Evolving Domain Generalization through Dynamic Latent Representations

Information Maximizing Adaptation Network With Label Distribution Priors for Unsupervised Domain Adaptation

Cross-domain Mutual Information Adversarial Maximization

Domain Adaptation Via Maximizing Surrogate Mutual Information

Learning Domain-Independent Deep Representations by Mutual Information Minimization

Invariant Information Bottleneck for Domain Generalization.

Maximizing conditional independence for unsupervised domain adaptation

On Learning Invariant Representation for Domain Adaptation

INSURE: An Information Theory iNspired diSentanglement and pURification modEl for Domain Generalization

Unsupervised domain adaptation by learning using privileged information

Conditional Independence Induced Unsupervised Domain Adaptation

Constrained Maximum Cross-Domain Likelihood for Domain Generalization

Deep Representation Learning for Domain Generalization with Information Bottleneck Principle

Privacy-Preserving Domain Adaptation for Intracranial EEG Classification via Information Maximization and Gaussian Mixture Model

Model-Induced Generalization Error Bound for Information-Theoretic Representation Learning in Source-Data-Free Unsupervised Domain Adaptation

Improving Adversarial Robustness via Mutual Information Estimation

Domain Generalization with Small Data

Unsupervised Domain Adaption of Hyperspectral Images Based on Paring Domain Discrimination

MaNi: Maximizing Mutual Information for Nuclei Cross-Domain Unsupervised Segmentation