Abstract:Many machine learning problems concern with discovering or associating common patterns in data of multiple views or modalities. Multi-view learning is of the methods to achieve such goals. Recent methods propose deep multi-view networks via adaptation of generic Deep Neural Networks (DNNs), which concatenate features of individual views at intermediate network layers (i.e., fusion layers). In this work, we study the problem of multi-view learning in such end-to-end networks. We take a regularization approach via multi-view learning criteria, and propose a novel, effective, and efficient neuron-wise correlation-maximizing regularizer. We implement our proposed regularizers collectively as a correlation-regularized network layer (CorrReg). CorrReg can be applied to either fully-connected or convolutional fusion layers, simply by replacing them with their CorrReg counterparts. By partitioning neurons of a hidden layer in generic DNNs into multiple subsets, we also consider a multi-view feature learning perspective of generic DNNs. Such a perspective enables us to study deep multi-view learning in the context of regularized network training, for which we present control experiments of benchmark image classification to show the efficacy of our proposed CorrReg. To investigate how CorrReg is useful for practical multi-view learning problems, we conduct experiments of RGB-D object/scene recognition and multi-view based 3D object recognition, using networks with fusion layers that concatenate intermediate features of individual modalities or views for subsequent classification. Applying CorrReg to fusion layers of these networks consistently improves classification performance. In particular, we achieve the new state of the art on the benchmark RGB-D object and RGB-D scene datasets. We make the implementation of CorrReg publicly available.

A novel regularization method for decorrelation learning of non-parallel hyperplanes

On Feature Decorrelation in Self-Supervised Learning

Joint Domain Alignment and Discriminative Feature Learning for Unsupervised Deep Domain Adaptation

A Local Online Learning Approach for Non-linear Data.

Hyperspherical Consistency Regularization.

Nonlinear Regression via Deep Negative Correlation Learning

Improving Deep Neural Network Sparsity Through Decorrelation Regularization

Regularized Multinomial Regression Method for Hyperspectral Data Classification Via Pathwise Coordinate Optimization

Decorrelated Empirical Likelihood for Generalized Linear Models with High-Dimensional Longitudinal Data

A hybrid ensemble method with negative correlation learning for regression

Deep Multi-View Learning using Neuron-Wise Correlation-Maximizing Regularizers

Unified Locally Linear Classifiers with Diversity-Promoting Anchor Points

Partial Feature Decorrelation for Non-I.I.D Image classification

On the Versatile Uses of Partial Distance Correlation in Deep Learning

A Variance Minimization Criterion to Feature Selection Using Laplacian Regularization

Regularizing Deep Convolutional Neural Networks with a Structured Decorrelation Constraint.

Elastic Net Nonparallel Hyperplane Support Vector Machine and Its Geometrical Rationality

Regularized Negative Correlation Learning for Neural Network Ensembles

Graph-based regularization for regression problems with alignment and highly-correlated designs

Towards Better Orthogonality Regularization with Disentangled Norm in Training Deep CNNs

A unified consensus-based parallel ADMM algorithm for high-dimensional regression with combined regularizations